Appia vs Cactus Comparing Protocol Composition Frameworks

Sergio Mena, Xavier Cuvellier, Christophe Grégoire, André Schiper

Presented by:

André Contente

Introduction

• First Frameworks (ISIS, Transis, Phoenix) – Monolithic – Tightly coupled – Hidden dependencies – Not Reusable

• Next Frameworks (Consul, Horus, JavaGroups, Ensemble) – Modular – Reusability, extensability, configurability – More complex – 2 families:

• X-Kernel and sucessors(Coyote, Cactus) • Horus and sucessors(Ensemble, Appia, JavaGroups)

Composition model: Appia

• Pair Layer-Session – Session

• Private state • Protocol code

– Layer • Accepted events • Provided events • Required events

• Quality of Service – Correctness check

• Channels

Interaction model: Appia

• Event-driven communication – Events triggered with:

• Channel

• Source session

• Direction

• Events can’t cross channels – Except “Sendable Events”

• Open event model – New event classes may be defined

Concurrency model: Appia

• Single Threaded

– No worries about synchronisation

• Event occurrences are queued

– One event is popped every step

Interface with the environment: Appia

• Insert events into channels

– Packets has arrived from the network

– Application has sent a message

• “TimerEvent”

– Sessions can set up timers

Composition model: Cactus

• Coarse grain

– Composite protocols

– Hierarchical graph

– Multiple protocols in the same level

• Fine grain

– Micro protocols

– No hierarchy

– Event triggering and data sharing

Composition model: Cactus Composite Protocols

• Session

– State

– Micro protocols

• Acyclic oriented graph

• Only linked protocols can interact

• Message based communication between sessions

– Demux function, if there are multiple sessions

Composition model: Cactus Micro Protocols

• Event driven communication – Events are only seen in one session

• Structure – Private State

– Event Handlers

• Micro protocols handlers can modify shared state

• Handler biding at start-up – Each micro protocol binds its handlers to events

Concurrency model: Cactus

• When an event arrives, all handlers bounded for that event will execute

• Trigger events – Raising (asynchronous, thread spawn or reused) – Invoking (synchronous, function call semantics)

• Raising events need synchronization to access shared data – Not enforced by Cactus

• Coordinating Raising – Register(mp,e) – Signal(mp,e)

Interface with the environment: Cactus

• Micro protocols can use OS resources at will – Timers, sockets, etc...

• Lowest level composite protocols can open

sockets

• Top level protocols are used using their interface, like another composite protocol

• Raising events after delay

Service Implemented

• To compare both protocols, a Fault-Tolerant Atomic Broadcast was implemented

• Modules

– Reliable channel

– Failure detector

– Consensus

– Atomic Broadcast

Service Implemented

• Common protocol code for both frameworks

• Event routing using framework facilities – Cactus: One Composite protocol, Modules as micro protocols

– Appia: One session per module

• Concurrency model – Appia: Single thread

– Cactus: Mutual exclusion was implemented

Similarities

• Internal structure of protocol modules – Session structure – Event Handling

• Managing message • Modifying internal state • Trigger new events

• Message abstraction – If a module pushes a header, only he can strip it

• Lack of flow control – Appia: may lead to overflow – Cactus: prevented by mutex

Differences

• Appia's channels vs Cactus cooperative composition – Appia: events flow in one direction

• Workaround: EchoEvent

– Cactus: no restrictions

• Correctness check – Appia: QoS check at start-up

– Cactus: Doesn't check

• Multiplexing occurrences – Two modules may accept the same event class

Differences

• Shared data – Impossible in Appia

• Fifo guarantee – Guaranteed by Appia's single thread model – Cactus doesn't, but mutex ensures it

• Concurrency model – Synchronization needs to be implement in Cactus to prevent

racing conditions

• Interface with the environment – Cactus: both network and application adopt a composite

protocol interface – Appia: asynchronous events, from application to the channel

Performance comparision

• Performance measured using “IBM SockPerf benchmark suite”

– SockPerf Proxy implemented

Performance comparison: Throughput

• Throughput with respect to message size

Performance comparison: Request-Reply

• Round-Trip time of a message

Performance comparison: Execution profiling

• Profiler on the client-side

Proposals for better frameworks

• Non-hierarchical composition model is better • Interaction model

– Point-to-Point events – Connectors

• Concurrency model – allow different models – Default sync – Fifo event handling – Transparency

• Interface with the environment – Application/Network insert events in the framework – Frameworks should provide various interfaces – Efficient serialisation

Conclusions

• Optimal framework would use features from both: – Composition and Interaction close to Cactus

– Several concurrency models, including Appia's

– Extended version of Appia's interface

• It would also include: – Point-to-Point events

– Connectors