Efficient Solutions to the Efficient Solutions to the Replicated Log and Dictionary Replicated Log and Dictionary

ProblemsProblems

By Gene T J Wuu and Arthur J BernsteinBy Gene T J Wuu and Arthur J Bernstein

Sunita Gupta, COEN 317, Spring Sunita Gupta, COEN 317, Spring 0505

OutlineOutline

• IntroductionIntroduction• The model of the environmentThe model of the environment• The Log Problem and its ApplicationThe Log Problem and its Application

– The Log ProblemThe Log Problem– The Dictionary ProblemThe Dictionary Problem

• Efficient SolutionsEfficient Solutions– A New Solution to the Log ProblemA New Solution to the Log Problem– A Efficient Solution to the Dictionary ProblemA Efficient Solution to the Dictionary Problem

• Comparison with other workComparison with other work• Some ApplicationsSome Applications

IntroductionIntroduction

• In this paper, authors propose In this paper, authors propose efficient algorithms to maintain a efficient algorithms to maintain a replicated dictionary in an unreliable replicated dictionary in an unreliable distributed system using log.distributed system using log.

• Replicated log is used to achieve Replicated log is used to achieve mutual consistency of replicated data mutual consistency of replicated data in an unreliable network.in an unreliable network.

Introduction (cont’)Introduction (cont’)

• Data replicationData replication In many application, data objects are In many application, data objects are

highly shared & reliability and fast access highly shared & reliability and fast access are very important. are very important.

• Logs Logs Useful in achieving distributed Useful in achieving distributed synchronization and in gathering synchronization and in gathering information about the state of the information about the state of the networknetwork

The model of the environmentThe model of the environment

• An n node connected Network with nodes N1, An n node connected Network with nodes N1, N2,.., Nn.N2,.., Nn.

• A node id – an integer in [n] denote set {1,2,..n}A node id – an integer in [n] denote set {1,2,..n}• Two kinds of operations – the communication Two kinds of operations – the communication

operation send and receive, and non operation send and receive, and non communication operationscommunication operations

• Events are locally atomic ( i.e. if a node crashes Events are locally atomic ( i.e. if a node crashes during the execution of an operation, there will be during the execution of an operation, there will be not effect on local data)not effect on local data)

• Events are distinguished by the time and place at Events are distinguished by the time and place at which they occur.which they occur.

The model of the environmentThe model of the environment(Cont’)(Cont’)• There is local clock at each nodeThere is local clock at each node• time(e) – time of event etime(e) – time of event e• Node(e) – node at which event e occurredNode(e) – node at which event e occurred• Op(e) – invoked operations and its Op(e) – invoked operations and its

parametersparameters• <E, → > a partial ordering relation on E<E, → > a partial ordering relation on E• Partial ordering relation – events occurring Partial ordering relation – events occurring

at the same node are totally ordered and at the same node are totally ordered and e1→ e2 – e1 send event and e2 – e1→ e2 – e1 send event and e2 – corresponding receive eventcorresponding receive event

The model of the environmentThe model of the environment(Cont’)(Cont’)• The unreliable behaviors covered by The unreliable behaviors covered by

this model – lost messages, broken this model – lost messages, broken communication links, network communication links, network partition, and failed nodespartition, and failed nodes

• Message in transmit not allowed to Message in transmit not allowed to change in arbitrary ways.change in arbitrary ways.

The Log Problem and its The Log Problem and its applicationapplication

• Each Node maintains its own view of Each Node maintains its own view of the log the log

• a distribution algorithm is employed a distribution algorithm is employed to keep the view up to date.to keep the view up to date.

The Log ProblemThe Log Problem

• Problem of finding an algorithm to Problem of finding an algorithm to maintain the log such that, given an maintain the log such that, given an execution < E, → >, for every event execution < E, → >, for every event e,e,

f → e iff fR є L(e)f → e iff fR є L(e)

The Log Problem (Cont’)The Log Problem (Cont’)

f → e iff fR є L(e) --- P1f → e iff fR є L(e) --- P1

• Event f happened before event e iff Event f happened before event e iff event recordevent record describing event f is describing event f is propagated to local copy of log of propagated to local copy of log of Node(e) immediately after event e Node(e) immediately after event e completes. completes.

• event record – contains operation type, event record – contains operation type, time type and Node Id.time type and Node Id.

The Log Problem (Cont’)The Log Problem (Cont’)

• To achieve P1, node exchange messages To achieve P1, node exchange messages containing appropriate portion of the log.containing appropriate portion of the log.

E.g. E.g. • Consider Node N1 and N2Consider Node N1 and N2• N1 only sends records to those events N1 only sends records to those events

which have occurred at N1 since it last which have occurred at N1 since it last sent a message to N2 (updated one)sent a message to N2 (updated one)

• Cannot achieve P1 – message delivery is Cannot achieve P1 – message delivery is not guaranteed.not guaranteed.

The Log Problem (Cont’)The Log Problem (Cont’)

• Trivial solution:Trivial solution:– At each non-communication event, e, At each non-communication event, e,

occurring at Ni, Ni inserts eR into Li, and occurring at Ni, Ni inserts eR into Li, and at each send event Ni includes Li in the at each send event Ni includes Li in the message. message.

The Dictionary ProblemThe Dictionary Problem

• Dictionary – an abstraction of data objects Dictionary – an abstraction of data objects (file directory), a data base dictionary, a (file directory), a data base dictionary, a recourse management tablerecourse management table

• Two non-communication operations on Two non-communication operations on dictionary x –dictionary x –– delete(x) delete(x) – insert(x) insert(x) – For Uniquness of each entry – tag each item to For Uniquness of each entry – tag each item to

be inserted with time stamp.be inserted with time stamp.

The Dictionary Problem The Dictionary Problem (Cont’)(Cont’)

• Problem of finding an algorithm for Problem of finding an algorithm for maintaining the dictionary such that, maintaining the dictionary such that, given an execution < E, →>, for given an execution < E, →>, for every event e,every event e,

x є V(e) iff Cx → e and there does x є V(e) iff Cx → e and there does not exist an x-delete event g, such not exist an x-delete event g, such that g → e.that g → e.

The Dictionary Problem The Dictionary Problem (Cont’)(Cont’)

x є V(e) iff Cx → e and there does not x є V(e) iff Cx → e and there does not exist an x-delete event g, such that g → e exist an x-delete event g, such that g → e --- P2--- P2

• Dictionary entry x is a part of local copy of Dictionary entry x is a part of local copy of dictionary at Node(e) iff the unique event dictionary at Node(e) iff the unique event Cx which inserts x happened before event Cx which inserts x happened before event e and there is no x-delete event g before e and there is no x-delete event g before event e occurred.event e occurred.

The Dictionary Problem (Cont’)The Dictionary Problem (Cont’)

• Obvious solution:Obvious solution:

• At each event, e, such that node(e) = At each event, e, such that node(e) = i, Ni computed V(e) in the following i, Ni computed V(e) in the following way:way:

Efficient solutionsEfficient solutions

• Problem with previous solution:Problem with previous solution:– Excessive communication costExcessive communication cost– Excessive computational costExcessive computational cost– Excessive storage costExcessive storage cost

A New Solution to the Log A New Solution to the Log ProblemProblem

• 2-Dimension Time Table(2DTT) 2-Dimension Time Table(2DTT) – Used by each node Used by each node – Each Node Ni keeps a two-dimensional Each Node Ni keeps a two-dimensional

time-table Ti , which corresponds to Ni's time-table Ti , which corresponds to Ni's most recent knowledge of the vector most recent knowledge of the vector clocks.clocks.

– Can tell how up to date other nodes are Can tell how up to date other nodes are about events occurring in the networkabout events occurring in the network

A New Solution to the Log A New Solution to the Log Problem (Cont’)Problem (Cont’)

• Node maintains following information:Node maintains following information:– A service called clocki – each reference to A service called clocki – each reference to

clocki returns an integer number greater clocki returns an integer number greater than that returned by the last reference.than that returned by the last reference.

– 2-Dimension time table Ti – Each time-2-Dimension time table Ti – Each time-table ensures the following time- table table ensures the following time- table propertyproperty•Ti[k, u] = t, Ni node knows that Nk node has Ti[k, u] = t, Ni node knows that Nk node has

received the records of all events which have received the records of all events which have occurred at Nu up to t.occurred at Nu up to t.

A New Solution to the Log A New Solution to the Log Problem (Cont’)Problem (Cont’)• To reduce communication it is desirable for a Node To reduce communication it is desirable for a Node

to know which node have received the record of a to know which node have received the record of a particular event.particular event.To this end a predicate HasRec, is defined as To this end a predicate HasRec, is defined as follows :follows :

HasRec(Ti;eR;k) = Ti[k;eR.node] >= eR.timeHasRec(Ti;eR;k) = Ti[k;eR.node] >= eR.time..

where if hasrec function is true at Ni then Nk has where if hasrec function is true at Ni then Nk has learned about event e.learned about event e.

• The protocol ensures that whenever a site is aware The protocol ensures that whenever a site is aware of an event (insert or delete in the dictionary), it is of an event (insert or delete in the dictionary), it is aware of all causally proceeding events. aware of all causally proceeding events.

An efficient Solution to the An efficient Solution to the Dictionary ProblemDictionary Problem

• Developed to overcome excessive Developed to overcome excessive computational costs and excessive computational costs and excessive storage coststorage cost

• Each node separately maintains its Each node separately maintains its view of the dictionary as well as view of the dictionary as well as partial log which records some partial log which records some events which have happened in the events which have happened in the network. network.

• Only partial log is kept at each node.Only partial log is kept at each node.

An efficient Solution to the An efficient Solution to the Dictionary Problem (Cont’)Dictionary Problem (Cont’)

• If the communication link between two If the communication link between two node breaks or message is lost, node can node breaks or message is lost, node can still learn indirectly, of a new event using still learn indirectly, of a new event using information passed through other nodes.information passed through other nodes.

• We assume that the state of node is We assume that the state of node is maintained in stable storage and data is maintained in stable storage and data is not lost when a crash occurs.not lost when a crash occurs.

Comparison with Other Comparison with Other WorkWork1. Fishers and Michael – Solution requires a 1. Fishers and Michael – Solution requires a

node to send its entire copy of the node to send its entire copy of the dictionary in each message. Solution is dictionary in each message. Solution is expensive. expensive.

2. 1- dimension time table – Each node 2. 1- dimension time table – Each node maintains synchronization set and node maintains synchronization set and node sends event record to other node even sends event record to other node even though node learnt about the event. Thus though node learnt about the event. Thus SS grows unboundedly. SS grows unboundedly.

Comparison with Other Comparison with Other WorkWork• 2-DTT – Deficient of approach - it is sent as a 2-DTT – Deficient of approach - it is sent as a

part of message => has size O(n^2) part of message => has size O(n^2) excessive storage and communication excessive storage and communication overhead.overhead.

• Modified algorithm:Modified algorithm:– Each node stores the complete 2DTT but sends Each node stores the complete 2DTT but sends

only its own row.only its own row.– Each node stores only its own row and a row for Each node stores only its own row and a row for

each of its neighbors; it sends only those rows each of its neighbors; it sends only those rows which correspond to neighbors of the target node.which correspond to neighbors of the target node.

– Each node stores only its neighbors rows and Each node stores only its neighbors rows and neighbors columns; it sends only those rows to neighbors columns; it sends only those rows to node which correspond to neighbors of that node.node which correspond to neighbors of that node.

Some applications uses Log Some applications uses Log solutionsolution

• Replicated numeric data with add-to Replicated numeric data with add-to and subtract-from operations. and subtract-from operations.

• Detection of failure – Log is used to Detection of failure – Log is used to collect records of communication collect records of communication events occuring in the network.events occuring in the network.

ReferencesReferences

• Partial Database Replication using Partial Database Replication using Epidemic Communication by J Epidemic Communication by J Holliday, D Agrawal, A Abbadi Holliday, D Agrawal, A Abbadi

• Class NotesClass Notes