unit7 clock synchronization.pdf

8/18/2019 UNIT7 clock synchronization.pdf

1/46

Time in Distributed Systems

Brian [email protected]


2/46

5-Oct-05 2

Needs for precision time

• Stock market buy and sell orders• Secure document timestamps (with

cryptographic certification)• Aviation traffic control and position reporting• Multimedia synchronization for real-time

teleconferencing• Event synchronization and ordering• Network monitoring, measurement and control• Distributed network gaming and training


3/46

The reversal of events by twodifferent observers cannotoccur if the two events arecausally dependent.

The physical effect follows thephysical cause for all observers.

The elapsed time between theoccurrence of events mayhowever vary.

The timing of physical events was hence show tobe relative to the observer, thereby discreditingNewton’s notion of absolute physical time !!

CONCLUSION: There is no special physical clock in the universe

to which we can appeal to precisely measure intervals of time !

Thanks Herr Einstein!!The Special Theory of Relativity : assumes that the speed of lightis constant for all observers, regardless of their relative speed.

Two events may appear in opposite order in two frames ofreferences that are in relative motion.


4/46

But Istill don’t

understandwomen


5/46

1. The sun

2. An Atom

Today: 1 sec ~ 1 day / 86400

but rotation of earth slows down

State transitions in atoms (defined by BIH in Paris)1 sec = time a cesium atom needs for 9 192631 770

state transitions*

* TAI (International Atomic Time)

Physical Time

• A TAI-day is about 3 msec shorter than a day

• BHI inserts 1 sec, if the difference between a day and aTAI-day is more than 800 msec•Definition of UTC = universal time coordinated, being thebase of any international time measure.


6/46

UTC• International standard for timekeeping.• Based on atomic time.• Synchronizes with astronomical time using leap

seconds.

• UTC signals are synchronized and broadcastregularly from land-based radio stations andsatellites.

• Computers with receivers attached cansynchronize their clocks with these timingsignals.


7/46

•UTC-signals come from radio broadcasting stationsor from satellites (GEOS, GPS) with an accuracy of:

• 1.0 msec (broadcasting station)

• 1.0 µsec (GPS)•>> 1ms (UTC available via phone line)

•Receivers are available commercialle and can beconnected to PCs

UTC broadcasts


8/46

Computer Clocks• Each node in the DS contains its own physical clock ! • Physical clocks are HW devices that count oscillations of

a quartz.• After a specified number of oscillations, the clock

increments a register, thereby adding one clock-tick to acounter the represents the passing of time: Hi(t).

• Resolution: Period between clock updates• The OS maintains SW Clock by scaling and adding anoffset to it:

C i(t) = H i(t) + .• C

i(t) approximates the physical time t at process p

i. C

i(t)

may be implemented by a 64-bit word, representingnanoseconds that have elapsed at time t.

• Successive events can be timed if the clock resolutionsis smaller that the time interval between the two events.


9/46

Network

Clock Skew

Drift and Skew

• Computer clocks, like any other clocks tend not to be inperfect agreement !!

• Clock skew (offset) : the difference between the timeson two clocks |C i(t) – C j(t)|

• Clock drift : they count time at different rates – Ordinary quartz clocks drift by ~ 1sec in 11-12 days. (10 -6

secs/sec). – High precision quartz clocks drift rate is ~ 10 -7 or 10 -8 secs/sec – Differences in material, Temperature variation.

C1 C2 C3 C4


10/46

Clock Synchronization• Clock makers specify a maximum drift rate ρ (rho)• By definition

1-ρ ≤ dC/dt ≤ 1+ ρwhere C(t) is the clock’s time as a function of the real time

• To keep them synchronized to within a time interval delta,δ, they must re-sync every δ/2ρ seconds.


11/46

Internal/External Synchronization

• synchronization of process’ clocks C iwith an authoritative external sourceS.

• Let D>0 be the synchronization

bound and S be the source of UTC.• Then |S(t) – C i(t)| < D for i=1,2,…,Nand for all real times t.

• We say that clocks C i are accuratewithin the bound of D

• synchronization of process’ clocksC i with each other.

•Let D>0 be the synchronizationbound and C i and C j are clocks at

processes p i and p j, respectively.•Then |C i(t) – C j(t)| < D for i,j=1,2,…,N and for all real times t.

•We say that clocks C i, C j agreewithin the bound of D

Note that clocks that are internally synchronized are not necessarily externallysynchronized. i.e., even though they agree with each other, the drift collectively from theexternal source of time.

External synchronization Internal synchronization


12/46

Synchronization in asynchronous systemWe have:

• known upper (max) and lower (min) bound forcommunication delay,

• known maximum clock drift,

• known maximum time taken for each computational step.

We synchronize by:• time server sends its local time t to a client,• and the client sets its local clock to t+(max+min)/2.


13/46

Synchronization in anasynchronous system• Christians algorithm.• The Berkeley algorithm.• Network time protocol (NTP).


14/46

Cristian’s Algorithm

• A node is a time server TS (presumablywith access to UTC). How can the othernodes be sync’ed?

• Periodically, at least every δ/2ρ seconds,each machine sends a message to the TSasking for the current time and the TSresponds.


15/46

Christians algorithm• Client p sends request (m r ) to time server S ,• S inserts its time t immediately before reply (m t ) is returned,

• p measures how long it takes (T round= T 1 - T 0 ) from m r is send to m t is received• p sets its local clock to t+T round /2.

t


16/46

Accuracy of Christians algorithm• Assume min = minimal message delay• t is in the interval [T 0+min, T 1-min]• Uncertainty on t = T round -2min• Estimated Accuracy= T round /2-min

tt


17/46

Cristian’s Algorithm• Monotonicity:

– Jumps in time backward not permitted – Jumps forward may be confusing – Receiver adjusts α : C i(t) = α Hi(t) + β .

• Improve precision – by taking several measurements and taking the

smallest round trip

– or use an average after throwing out the large values


18/46

The Berkeley algorithmDesigned for internal synchronization.1.One machine is chosen as the master M ,2. M request other machines p i to return their local

time t i .3. M calculates a ‘fault tolerant’ average t of

received t i ’s.4. M instructs each machine p i to offset its clock

with t-t i.

• Idea: averaging cancels individual clock drifterrors

• Do not include extreme deviations in average

(clock is assumed to be faulty)


19/46

The Berkeley Algorithm

a) The time daemon asks all the other machines for their clock valuesb) The machines answer c) The time daemon tells everyone how to adjust their clock


20/46

Network time protocol• Synchronization of clients relative to UTC

on an internet-wide scale• Reliable, even in the presence ofextensive loss of connectivity

• Allow frequent synchronization (relative toclock drift)• Protection of service against disturbance

•


21/46

An example synchronizationsubnet in an NTP implementation

1

2

3

2

3 3

Note: Arrows denote synchronization control, numbers denotestrata.

Stratum 1 servers

directly coupled UTC time


22/46

Reconfiguration

1

2

3

2

3 3

1

2

3

3

3 3

Server changes stratum

x


23/46

Distributed Systems

• Multicast (for quick LANs, low accuracy)- server sends its actual time to its leaves in theLAN

• Procedure-call (medium accuracy)- server responds to requests with its actualtimestamp- like Cristian’s algorithm

• Symmetric mode (high accuracy)- used to synchronize between the time servers

Remark: In all cases the UDP-transportation protocol is used, i.e. messages can get lost!

NTP-Modes


24/46

Messages exchangedbetween a pair of NTP peers

Ti

Ti-1Ti-2

Ti- 3

Server B

Server A

Time

m m'

Time

•Exchanges local timestamps to estimate offset o i and delay d i•NTP servere filters pairs , saves 8 latest•Use the o i, with smallest d i.(the smaller delay the better accuracy)

oi=(Ti-2-Ti-3+Ti-1-Ti)/2 d i=Ti-2-Ti-3+Ti-Ti-1


25/46

Theoretical Foundations• Inherent characterstic of a distributed system:

– Absence of a global clock: – Absence of 100% accurately synchronized clocks

– Impact: Due to the absence of global clock, it isdifficult to reason about the temporal order ofevents in distributed system, e.g. schedulingevents is more difficult.


26/46

Logical Clocks in a DS• What is important is usually not when things

happened but in what order they happened so theinteger counter works well in a centralizedsystem.

• However, in a DS, each system has its ownlogical clock, and you can run into problems if one“clock” gets ahead of others. (like with physicalclocks)

• RELIABLE WAY OF ORDERING EVENTS ISREQUIRED

• We need a rule to synchronize the logical clocks.


27/46

Event Ordering

• Database updates need to be performed in thesame order at all sites of a replicated database.


28/46

Events and Logical Clocks• Leslie Lamport’s 1978 paper: Time,

Clocks, and the Ordering of Events inDistributed Systems. – Theoretical Foundation – Logical Clocks – Partial and Total Event Ordering

– Towards distribute mutual exclusion – MUST KNOW FOR ANY COMPUTER

SCIENTIST


29/46

System Model• A distributed system is a collection P of sequential

processes p i, i= 1,2,…N.• A process p i has state s i• Each process p i executes a sequence of actions

• Sending a message;

• Receiving a message;• Performing an internal computation that alters its state s i;• The sequence of events within a single process p i are

totally ordered e je ’

• The history of process p i is the sequence of events thattakes place thereinhistory(p i ) = h i =

ei

0 ei

1 ei

2 ei

4ei

3


30/46

Happened Before Relation• The happened-before relation captures the causal

dependencies between events,1. a b if a and b are events in the same process and a

occurred before b.2. a b if a is the event of sending a message m in a

process and b is the event of receipt of the samemessage m by another process.3. If a b and b c, then a c, i.e. happened before

relation is transitive.

• That is, past events causal affects future events


31/46

Concurrent Events• Two distinct events a and b are

concurrent (a||b) if not (a b or b a).• We cannot say whether one event

happened-before• For any two events a and b in a distributed

system, either a b, b a or a||b.


32/46

Examplep1

p2

p3

a b

c d

e f

m1

m2

Physicaltime

• a b and a c and b f• b f and e f Does e b?


33/46

Logical Clocks• There is a clock C i at each process p i• The clock C i can be thought of as a function

that assigns a number C i(a) to any event a,called the timestamp of event a, at p i

• These clocks can be implemented bycounters and have no relation to physicaltime.


34/46

Conditions Satisfied by theSystem of Clocks

• For any events a and b: if a b , then C(a)


35/46

Implementation Rules• [IR1] Before P i timestamps an event

Ci:= C i+1• [IR2a] P i sends m: [IR1] and piggy-back

timestamp t=C i: m’=• [IR2b] P j receives m’=:

C j:=max(C j, t), followed by [IR1]


36/46

Lamport clocks example

b

2

a

1

1m

c

3

d

4

f

m 2

5

e

1

p1

p2

p3

Physicaltime

NOTE: C(e)


37/46

Total Ordering of Events• Lamport’s happened before relation defines an

irreflexive partial order among the events.• Total ordering (denoted by =>) can be obtained

by using process-id to break tie if timestamps

are equal:• a => b iff

– C i(a) < C j(b) or

– C i(a) = C j(b) and P i


38/46

Total Order Lamport Timestamps

• The order will be (1,1), (1,3), (2,1), (3,2) etc

( 1, 1) ( 2, 1)

( 3, 2)

( 4, 2)

( 1, 3) ( 5, 3)


39/46

Why Total Order?

• Database updates need to be performed in thesame order at all sites of a replicated database.


40/46

Exercise: Lamport Clocks

• Assuming the only events are message sendand receive, what are the clock values atevents a-g?

A

B

C

a b c

d e

f g


41/46

Vector Clocks• Lamport: e f implies C(e) < C(f)

• Vector clocks: e f iff C(e) < C(f)• vector timestamps : Each node maintains

an array of N counters• Vi[i] is the local clock for process p i• In general, Vi[j] is the latest info the node

has on what p j‘s local clock is.


42/46

Implementation Rules• [VC1] Initially Vi[j]=0 for i,j = 1…N• [VC2] Before P i timestamps an event:

Vi[i] := Vi [i] +1• [VC3] P

isends m: piggy-back timestamp t=V

i:

m’=• [VC4] P j receives m’=

Vi[i] :=max(Vi[i] , t i[i])


43/46

Comparison of Vector ClocksComparing vector clocks• V = V´ iff V[j] = V´[j] for all j=1,2,…,N.• V ≤ V´ iff V[j] ≤ V´[j] for all j=1,2,…,N.

• V < V´ iff V≤

V´ and V≠

V´.


44/46

Vector clocks illustrated

b

(2,0,0)

a

(1,0,0)

1m

c

(2,1,0)

d

(2,2,0)

f

m2

(2,2,2)

e

(0,0,1)

p1

p2

p3

Physicaltime

NOTE e and b are not related


45/46

Vector Clock Exercise• Assuming the only events are send and

receive:• What is the vector clock at events a-f?• Which events are concurrent?

A

B

C

a

b e

c d

f


46/46

END

unit7 clock synchronization.pdf

Documents