optimistic virtual synchrony

Optimistic Virtual Synchrony

Jeremy Sussman - IBM T.J.WatsonIdit Keidar – MIT LCS

Keith Marzullo – UCSD CS Dept.

2Optimistic Virtual Synchrony

Overview of Talk

• Group Communication Services (GCS)– Group Membership and Reliable Group Multicast

• and why some properties force processes to block

• Optimistic Virtual Synchrony (OVS)– Concept– Evaluation

• Related Work• Conclusions


Group Communication Systems (1)

• Group Membership– Processes organized

into groups– Particular

memberships stamped as views

• In partitionable system, views can be concurrent

• Views provide a form of Concurrent Common Knowledge about system

p1time

p2 p3

V1 {p1, p2, p3}

V2 {p1, p2}

V5 {p1, p2, p3}

V3 {p3}


• Reliable Multicast– Messages sent to group– Same View Delivery

• If p1 delivers m while in v1 and p2 delivers m,then p2 delivers m in v1

– Sending View Delivery• If p1 sends a message m

while in v1,then m is delivered in v1

Group Communication Systems (2)

p1time

p2 p3

V1 {p1, p2, p3}

V2 {p1, p2}


• Assume that message delivery takes

• Let processes send a message every t <

• When can a new view be installed…– without violating

Sending View Delivery?– without dropping

messages?

Why Properties Imply Blockingp1tim

e

p2 p3

V1 {p1, p2, p3}


Observations

• Many problems are caused by stale views– Processes block when a view is stale– A sending process cannot know if the view

will be stale before a message is delivered– For many applications, a message

delivered in a stale view is useless• Many applications do not require exact

semantics of Sending View Delivery– State transfer not required for splitting– Leader election not required for joins


OVS: The Idea

• Inform processes of stale views– Give an “educated guess” of subsequent

view(called the optimistic view)

• Allow processes to send optimistic messages that will be delivered in subsequent view– But deliver or drop message based on

some predicate that the application provides


OVS Implementation

• Sender side:– All messages sent in a view that has

become stale are sent optimistically• Enhanced by a MessageCondition predicate

• Receiver side:– Store in a queue all optimistic messages

received before a view change– On new view, deliver all optimistic

messages for which the MessageCondition is true• If more optimistic messages are received, treat

them as if they were at the end of the queue


Message Conditions

• Separation of mechanism and policy– Provided by the application– Expressed as a predicate over the previous

view, subsequent view, and optimistic view

• Examples:– Leader election

• ( leader in subsequent_view)

– Need for state transfer• (previous_view subset subsequent_view)


Evaluation of OVS

• Implementation of OVS on top of Transis– Comparison to a blocking system and to

one that does not provide Sending View Delivery

– Measurement of overhead of OVS

• Examination of applications that can benefit from the OVS semantics– (see proceedings)


Message Lifecycle

Pre-delivery processing

4%

Pre-send processing

8%

W ire time88%

Pre-send

Pre-delivery

Pre-delivery

Message Transmission Time

Pre-send ~ 90 microsecondsWire-time ~1000 microsecondsPre-delivery ~ 40 microseconds-------------------------------------Total ~1130 microseconds

p1time

server p2server


OVS Overhead

0 20 40 60 80 100 120 140

Time in Microseconds

Regular messages

Regular messages

Optimistic messages

Optimistic messages

Optimistic messageswith retransmission

Optimistic messageswith retransmission

Sen

der

side

Rec

eive

r si

de

Processing Time, pre-delivery


OVS Performance Benefits

1000

10000

100000

Sending View Delivery

No Sending View Delivery

Optimistic

Optimistic with retransmission

Average Time to Deliver Messages After View Change

Tim

e in

M

icro

seco

nds

Messages delivered


Related Work

• Optimistic Atomic Broadcast [Pedone, Schiper]+ Uses optimism for total order+ Complementary to our approach

• Non-blocking light-weight groups [Amir et al; Dolev, Malki]+ Scales well+ Provides fast view delivery– Does not provide Sending View Delivery Property– Often allows messages to span more than one view

(problematic for state transfer, other applications)

• Weak Virtual Synchrony [Friedman, van Renesse]+ Eliminates blocking+ Optimized for membership translation– Does not provide same level of policy/mechanism split– May require extra views to be delivered by system


Conclusions

• Optimistic Virtually Synchrony– Uses a very simple form of optimism

• Receiving processes never need to rollback from optimistic messages

• Sending process informed of dropped messages

– Provides applications with useful properties• Policy/Mechanism split on delivery semantics

– From Sending View Delivery toall messages being delivered in subsequent view

– Has low overhead

optimistic virtual synchrony

Documents