clock sync
TRANSCRIPT
-
7/31/2019 Clock Sync
1/59
Clock Synchronization
Topics in Multiprocessing 2004
Presented by: Alon Stern
-
7/31/2019 Clock Sync
2/59
Outline Introduction
Event Ordering Partial Ordering Total Ordering
Clock Synchronization
Model Overview The Basic Theorem
External Synchronization
-
7/31/2019 Clock Sync
3/59
Importance of clock
synchronization Communication is not enough. Cooperation
is needed synchronization.
Distributed synchronization is needed for:
Transactions (bank account via ATM)
Access to shared resources (networkprinter, databases)
Ordering of events (network gameswhere players have different ping times)
-
7/31/2019 Clock Sync
4/59
Clock Synchronization Problem
GoalObtain the tightest achievable estimateof the readings of a remote clock in any givenexecution of the system.
-
7/31/2019 Clock Sync
5/59
Outline Introduction
Event Ordering Partial Ordering Total Ordering
Clock Synchronization
Model Overview The Basic Theorem
External Synchronization
-
7/31/2019 Clock Sync
6/59
Event Ordering The concept of one event happening
before another in a distributed system
is examined.
Defining a partial ordering of events.
Defining total ordering of events.
-
7/31/2019 Clock Sync
7/59
Defining Our System The system is composed of a collection of
processes.
Each process consists of a sequence ofevents.
The events of a process form a sequence,where aoccurs before bin this sequence ifa
happened before b. Sending or receiving a message is an event in
a process.
-
7/31/2019 Clock Sync
8/59
Causality Event ordering linked with the concept
of causality:
Saying that event ahappened before eventbis the same as saying that it is possiblefor event ato causally effect event b.
If events aand bhappen on processes thatdo not exchange any data, their exactordering is not important.
-
7/31/2019 Clock Sync
9/59
Partial Ordering In a distributed system it is sometimes
impossible to say that one of two
events occurred first.
The relationhappened before is
therefore only a partial ordering of theevents in the system.
-
7/31/2019 Clock Sync
10/59
The Happened Before
Relation The happened-beforerelation in a distributed
system:
1. Ifaand bare two events in the same process,
and acomes before b, then a b.2. Ifais the sending of a message, and bis the
receipt of that message, then a b.
3. Ifa band b cthen a c.
Two distinct events aand bare said to be concurrentifa band b a.
-
7/31/2019 Clock Sync
11/59
q2 q4(rule #1)
q4 r3(rule #2)
r3 r4(rule #1)
Example p1 q2(rule #2)
p1 r4(rule #3)
p3,q3are concurrent(neither can causally effectthe other)
-
7/31/2019 Clock Sync
12/59
Logical Clocks Problem: How do we maintain a global
view on the systems behavior?
Solution: Use logical clocks on eachprocess.
-
7/31/2019 Clock Sync
13/59
Logical Clocks Define a Logical clockCifor each
process Pito be a function which
assigns a number Ci(a)to any event ain that process.
Monotonically increasing counter.
No relation with physical time: Just away of assigning a number to an event.
-
7/31/2019 Clock Sync
14/59
Logical Clocks The clock condition:
Ifa b then c(a) < c(b).
The clock condition is satisfied if:
Ifaand bare events in process Pi, and acomes before b, then Ci(a)
-
7/31/2019 Clock Sync
15/59
Logical Clocks Note: We can not expect
the converse condition.
Reason: That would
imply that any twoconcurrent events mustoccur at the same time.
Example: p2and p3are
both concurrent with q3.they must occur at thesame time as q3.C(p2)=C(q3)=C(p3).But, p2 p3.
-
7/31/2019 Clock Sync
16/59
Logical Clocks in Space-Time
Diagram There must be a
tick line between
any two events ona process line.
Every message line
must cross a tickline.
-
7/31/2019 Clock Sync
17/59
Satisfying the Clock Condition Each process Pi maintains a localcounter
Ci and adjusts this counter according to thefollowing rules:
1. Ci is incremented before each event. Ci=Ci+1.
2. If event ais the sending of a message m, thenthe message mcontains a timestamp Tm=Ci(a).
3. Whenever a message mis received by a processPj, Pj adjusts its local counter Cj:Cjmax{Cj+1,Tm+1}.
-
7/31/2019 Clock Sync
18/59
Satisfying the Clock Conditionpublicclass LamportClock {
int c;
public LamportClock() {
c = 1;
}
publicint getValue() {
return c;
}
publicvoid internalAction() {
c = c + 1;
//action execution
}
publicvoid sendAction() {
c = c + 1;
msg.header.timeStamp = c;
//send message
}
publicvoid receiveAction(int src, int sentValue) {
c = max(c, sentValue) + 1;
}
-
7/31/2019 Clock Sync
19/59
Total Ordering With Logical
ClocksProblem: it can still occur that two events
happen at the same time.
Solution: Attach a process number to anevent:
Pi timestamps event ewith {Ci(e),i}.
The relation defines a total ordering.abif and only if either:
Ci(a)
-
7/31/2019 Clock Sync
20/59
Example of Total Ordering Events (1,0) and
(1,1) were
considered to beconcurrent.
Now we can say
(1,0) (1,1)
(1,0)(1,1)
(2,1)
(3,1)
(4,1)
(2,0)
(5,0)
(6,0)
(7,0)
(5,1)
(6,1)
(7,1)
(1,2)
(2,2)
(4,2)
-
7/31/2019 Clock Sync
21/59
Anomalous Behavior Presence of outside interactions.
Carrying a diskette from one machine to
another. Dictating file changes over the phone.
Two ways to avoid such anomalous:
Explicitly introduce into the system the necessaryinformation about the ordering.
Strong clock condition: for any event a,bif abthen c(a)
-
7/31/2019 Clock Sync
22/59
Outline Introduction
Event Ordering Partial Ordering Total Ordering
Clock Synchronization
Model Overview The Basic Theorem
External Synchronization
-
7/31/2019 Clock Sync
23/59
Clock Synchronization Goal: Ensure that physically dispersed
processors will acquire a common notion of
time using: Local physical clocks (whose rates may vary).
Message exchange over a communication network(with uncertain transmission delay).
Question: Can we set all the clocks in adistributed system to have the same time?
-
7/31/2019 Clock Sync
24/59
Physical Clocks Every computer contains a physical
clock.
Aclockis an electronic device, whichcounts oscillations in a crystal at aparticular frequency.
Example: Oscillator frequency is 10MHz
increment clock by one second after counting
10M cycles.
-
7/31/2019 Clock Sync
25/59
Physical Clocks in Distributed
System Can we synchronize all the clocks to some known high
degree of accuracy?
Well, there are some problems
Its difficult to synchronize the clocks. Crystal based clocks tend to drift over time.
Physical variations in the crystals, temperaturevariations, aging, etc.
Drift is small, but adds up over time.
For quartz crystal time, maximum drift rate is 50 secondsevery 106 seconds=11.6days.
Best atomic clocks have drift rate of one second in 1013seconds = 300,000 years.
-
7/31/2019 Clock Sync
26/59
Real Time Exact time was computed by astronomers. Take
noon for two days, divide by 24*60*60 meansolar second.
At present, the real time is taken as the average ofsome 50 cesium-clocks around the world.
Introduces a leap second from time to time tocompensate that days are getting longer. This time
called Universal Coordinated Time (UTC).
UTC is broadcast through short wave radio andsatellite.
-
7/31/2019 Clock Sync
27/59
Does This Solve All Our
Problems?
Dont we now have some global timing
mechanism?
Not every machine has UTC receiver.
Suppose we have a distributed system with a UTC
receiver somewhere in it, we still have to distributeits time to each machine.
-
7/31/2019 Clock Sync
28/59
Clock Synchronization Variants External Synchronization
Get tight estimate on
some source clock whichshows real time.
Internal Synchronization
Real time is notavailable. Keep all clocksas close as possible.
-
7/31/2019 Clock Sync
29/59
Main Difficulties Synchronization tends to deteriorate over time and
space.
When the local clocks are not drift free, the
synchronization loosens timing informationshould be refreshed periodically.
Communication information to distant processors timing uncertainty due to unpredictable
message delays. Usually there exist some a-priory bounds on these
uncertainties.
-
7/31/2019 Clock Sync
30/59
Model Overview Local Clock The hardware
clock of the processor.
Network Connectsprocessors to their neighbors.
Send Module When andwhere to send a message.
Clock SynchronizationAlgorithm (CSA) Computesa correctness and an errormargin.
-
7/31/2019 Clock Sync
31/59
Local Clock Hardware clock, Physical clock.
Encodes at all states sof processor v a realnumber called local timedenoted LT(s).
Drift free or bounded drift clock. Drift freeRun at the rate of real time, but does
not necessary shows real time.
Bounded driftKnown lower and upper bounds onthe rate of its progress relative to real time,denoted +, -
-
7/31/2019 Clock Sync
32/59
Drift Bounds - ExampleTypical workstation - clock accuracy of50
parts per million (ppm).
If it shows that 1000000 time unitshave passed between event pand q,we are guaranteed that RT(p)-RT(q)
[999950,1000050].
-
7/31/2019 Clock Sync
33/59
Network Connects processors to their neighbors.
May lose, duplicate and deliver
messages out of order. The real time delay of each message m
is within some known bounds
0L(m)H(m). Bounds may vary from link to link and
from message to message.
-
7/31/2019 Clock Sync
34/59
Clock Synchronization Module
(CSA) Uses:
Local time
Received Messages
Specification of the system
To compute Correctness term D
Error margin
The logical time is (T,e)where T=LT+D. sticks extra bytes to an outgoing message and
strips the corresponding bytes of incomingmessage.
-
7/31/2019 Clock Sync
35/59
Before we start
Lets take a simpleexample:
Suppose u,vare drift freeclocks.
When the clock of node ushows 12, ucan safety
deduce that the clock at vshows something in[18,20].
-
7/31/2019 Clock Sync
36/59
System Execution A sequence of events, where each event p,
has: Real time of occurrence RT(p).
Local time of occurrence LT(p).
We shall think of an execution as a graph,whose nodes are the set of events, with anedge(p,q)if either: qis the receive event whose send event is p.
p,qoccur at the same processor and qis the firstevent following p.
-
7/31/2019 Clock Sync
37/59
Execution as a Graph Nodes:
{p,q,r,s,t}
Edges:{(p,q),(q,s),(r,s),(s,t)}
r
RT(r)
LT(r)
s
RT(s)
LT(s)
pRT(p)
LT(p)
q
RT(q)
LT(q)
t
RT(t)
LT(t)
-
7/31/2019 Clock Sync
38/59
View Given an execution, its view is obtained by omitting
the real time mapping.
View graphGb
V- The set of points
E- Two anti-symmetrical directed edges joining each pairof adjacent points.
An event of an execution or a view is called apoint.
The local view of a point is the view of all pointswhich happened before the point.
-
7/31/2019 Clock Sync
39/59
View Graph - Exampler
LT(r)
s
LT(s)
pLT(p)
q
LT(q)
t
LT(t)
Time
-
7/31/2019 Clock Sync
40/59
-
7/31/2019 Clock Sync
41/59
Real Time Specifications Bounds on the difference of real times
between pairs of events.
Use the following real time specifications: Message transmit bounds.
Clock drift bounds.
The bounds will be modeled uniformly
nodifference between the bounds oncommunication delay and the bounds ondrifting clocks.
-
7/31/2019 Clock Sync
42/59
Bounds MappingA function Bthat maps every pair p,qof
adjacent points to a number B(p,q)>-.
The interpretation of bounds mapping isonly upper bounds.
For every pair of adjacent points p,qwe
have that RT(p)-RT(q)B(p,q).
-
7/31/2019 Clock Sync
43/59
Bounds Mapping - Formulation State the bounds mapping using quantities
available to the processor:
L(m), H(m),+,-,virt_del
For a message mwith send point p, receivepoint qand delay in the range [L(m),H(m)]:B(q,p)=H(m)andB(p,q)=(-L(m)).
For two adjacent points p,qat a processorhaving drift bounds+,-
B(q,p)=virt_del(q,p)/-
B(p,q)=virt_del(p,q)/+
-
7/31/2019 Clock Sync
44/59
Bounds Mapping - ExampleGiven:
Processors clock may drift up to 100ppm.
Message delivery time is completely arbitrary
(between 0to ).The bounds mapping:
Ifpis the send event and qis the receive event ofthat message B(p,q)=0.
Ifpoccurs before qat the same processor,B(p,q)=(LT(p)-LT(q))/1.0001and
B(q,p)=(LT(q)-LT(p))/0.9999.
For all other cases B(p,q)=.
-
7/31/2019 Clock Sync
45/59
Relative Offset Ties together actual and virtual delays. Absolute offset of a point p:d(p)=RT(p)-LT(p).
The offset ofprelative to a point q:d(p,q)=d(p)-d(q).
Lemma: For any two points p,q of a pattern,d(p,q)=-d(q,p).
Lemma: For any three points p,q,r of apattern, d(p,q)=d(p,r)+d(r,q).
-
7/31/2019 Clock Sync
46/59
Pause What information do we have till now?
A view graph, containing the local time LTofeach event.
bounds mapping in terms of quantitiesavailable to the processor L(m), H(m),+,-,virt_del
Offset definition which ties together actual
and virtual delay. Using this information we can now define
a synchronization graph.
-
7/31/2019 Clock Sync
47/59
Synchronization GraphGiven: A viewb=(V,E). Bounds mapping B.
The synchronization graphGbB=(V,E,w)is: The view graph (V,E)where for each (p,q)E,
w(p,q)=B(p,q)-virt_del(p,q).
Lemma: For every pair (p,q)E, d(p,q)w(p,q). Explanation:act_del(p,q)-virt_del(p,q)B(p,q)-virt_del(p,q).
-
7/31/2019 Clock Sync
48/59
Synchronization Graph -
Example Nodes, points.
Two anti-symmetrical
directed edges joiningeach pair of adjacentpoints.
Weighted edges.
-
7/31/2019 Clock Sync
49/59
Synchronization Distance Strategy - Reduce synchronization problem
to distance computations in the
synchronization graph. synchronization distance d(p,q): Length
of the shortest directed path from pto qinthe synchronization graph.
Lemma: IfBis satisfied, then for every twopoints p,qGbB, d(p,q)d(p,q)
-
7/31/2019 Clock Sync
50/59
Synchronization Distance -
Example
d(p,q) = 10d(q,p) = -8
-
7/31/2019 Clock Sync
51/59
The Main Theorem Let p,qbe to points in a view b. In
any execution:
RT(p)-RT(q)[VD(p,q)-d(q,p),VD(p,q)+d(p,q)]
Furthermore, there exist executions
which meets the bounds. Interpretation: The synchronization
distances determine precisely theuncertainty bounds.
-
7/31/2019 Clock Sync
52/59
Main Theorem - Example Given
p is an event at node u
q,r are events at node v
The clocks are drift free The delay of the message from p to q is
in the range [0,2].
pLT=7
q
LT=15
rLT=17
Node u Node v
w=0w=0
drift free w(q,r)=w(r,q)=0
w=(-6)
B(q,p)=2 w(q,p)=2-(15-7)=(-6)
w=8
B(p,q)=0 w(p,q)=0-(7-15)=8
RT(r)-RT(p)[(17-7)-8, (17-7)+(-6)]
RT(r)-RT(p) [2,4]
Solution:
d(r,p)=(-6), d(p,r)=8
-
7/31/2019 Clock Sync
53/59
Main Theorem - Example Cont RT(p)=5, RT(q)=7, RT(r)=9
Real time specifications are satisfied.
RT(r)-RT(p)=4
pLT=7
q
LT=15
rLT=17
RT(p)=5, RT(q)=5, RT(r)=7
Real time specifications are satisfied.
RT(r)-RT(p)=2
RT(p)=5, RT(q)=10, RT(r)=12
Real time specifications are not satisfied.
X RT(r)-RT(p)=7
-
7/31/2019 Clock Sync
54/59
External Synchronization Each processor has a local clock.
Each clock has drift bounds.
Each message has latency bounds.
One of the clocks, Called source, shows real time.Task Processors should provide the best estimate of the
source clock.
An estimate is an interval which is guaranteed to containthe source time.
-
7/31/2019 Clock Sync
55/59
Synchronization Algorithms Efficient - If its complexity is
independent of the execution length.
OptimalIf it always outputs thetightest bounds.
GeneralIf it works for all possible
systems.
-
7/31/2019 Clock Sync
56/59
Space Lower BoundAny optimal and general algorithm for
external synchronization has high spacecomplexity is not efficient.
There is no algorithm which is general,efficient and optimal.
One of the three should be sacrificed.
-
7/31/2019 Clock Sync
57/59
Drift Free Clocks Lemma The distance between any two
points that occur at a drift free clock is 0.
Meaning All points associated with aprocessor with a perfect clock may becollapsed into a single supper point forsynchronization distance purposes.
Number ofnodes inthe graph
Number ofprocessors=
-
7/31/2019 Clock Sync
58/59
Drift Free Clocks - Algorithm
Idea Use distribution Bellman-Ford.
Algorithm message from uto vcontains:
timestampu, wu(v,u),wu(u,v),du(s,u),du(u,s)When v receives a message with delay bounds [L,H]:
vd = timestampv - timestampuwv(u,v) = min{H-vd, wu(u,v), wv(u,v)}
wv(v,u) = min{L+vd, wu(v,u), wv(v,u)}dv(v,s) = min{du(u,s)+wu(v,u), dv(v,s)}dv(s,v) = min{du(s,u)+wu(u,v), dv(s,v)}D= (dv(v,s) - dv(s,v))ev = (dv(v,s) + dv(s,v))
-
7/31/2019 Clock Sync
59/59
References
A Theory of Clock Synchronization, by Patt-Shamir and Rajsbaum (STOC 94).
Optimal and Efficient Clock SynchronizationUnder Drifting Clocks, by Ostrovsky and Patt-Shamir (PODC 99).
Time, Clocks, and the Ordering of Events in a
Distributed System, by Leslie Lamport.