creating scalable message driven solutions akkadotnet

CREATING SCALABLE MESSAGE-DRIVEN

SOLUTIONS (WITH AKKA.NET)

David Hoerster

ABOUT ME 5-Time .NET (Visual C#) MVP (April 2011) Sr. Solutions Architect at Confluence One of the organizers for Pittsburgh TechFest (http://pghtechfest.com) Organizer of Pittsburgh Reactive Dev Group (http://meetup.com/reactive)

Past President of Pittsburgh .NET Users Group Twitter - @DavidHoerster Blog – http://blog.agileways.com Email – david@agileways.com

AGENDA Messages as Part of Application Design Being Reactive What the Actor Model Provides Hello Akka.NET Working with Akka.NET Cross-Process Actor Systems (Akka.Remote)

CQRS Command Query Responsibility Segregation Coined by Greg Young

Evolution of CQS (Command Query Separation) Coined by Bertrand Meyer

Both revolve around the separation of writes (command) and reads (query)

CQS is more at the unit level CQRS is at the bounded context level, or so

MESSAGES The core of CQRS Messages Communication from the service layer to the domain via commands Command handlers could also update the underlying repository

As actions happen, events are raised Event handlers could issue other commands Event handlers could update the underlying repository

MESSAGES Regardless of how it’s implemented, communication between application parts is via messages

Not only helps communicate intent of the action “PublishWidget”, “ExpireWorkItem”, “UpdateDefinition”, “DefinitionUpdated”

But allows for remote handling of messages Queue, REST payload, etc.

MESSAGES ARE KEY TO BEING REACTIVE

SO? So how does this affect my system design?

Let’s consider some procedural pitfalls

WORD COUNTER EXAMPLE Simple program to count the occurrences of words in a document

Print out the top 25 words and their counts at the end

Can be accomplished easily via procedural code

WORD COUNTER EXAMPLE

WORD COUNTER EXAMPLE What if you wanted to spread out the counting load?

What if you wanted to scale out to other nodes/processes/machines?

Fault tolerance, concurrent actions, etc.?

BUILDING CONCURRENT APPS In word counter, may need to spawn some threads

Lock some critical sections, etc.

The system becomes responsible for knowing everything going on Very Proactive (not Reactive!) system

SIDE NOTE: REACTIVE VS. PROACTIVE Isn’t proactive a good thing?

A system taking upon itself to check state is proactive It’s doing too much

A reactive system reacts to events that occur It’s doing just the right amount

SIDE NOTE: REACTIVE VS. PROACTIVE Isn’t proactive a good thing?

A system taking upon itself to check state is proactive It’s doing too much

A reactive system reacts to events that occur It’s doing just the right amount

System

EvtEvt

System

EvtEvt

Proactive

Reactive

REAL WORLD SCENARIO Loosely structured data

Multiple files to combine

Operations to perform during the projection process

REAL WORLD SCENARIO

File File File

Join

Op

Join/End

REAL WORLD SCENARIO Number of “steps”

May need some sub-steps

Can be broken down into individual pieces Actors

Communicate via messages for loose coupling

REAL WORLD SCENARIO Actually looks like this:

Multiple sources

Several transforms

Several joins

Configurable

ACTOR MODEL Actor Model is a system made of small units of concurrent computation Formulated in the early 70’s

Each unit is an actor

Communicates to each other via messages

Actors can have children, which they can supervise

WHAT IS AN ACTOR? Class that encapsulates state and behavior State can be persisted (Akka.Persistence) Behavior can be switched (Become/Unbecome)

Has a mailbox to receive messages Ordered delivery by default Queue

Can create child actors Has a supervisory strategy for child actors How to handle misbehaving children

Are always thread safe Have a lifecycle Started, stopped, restarted

AKKA.NET Akka.NET is an open source framework Actor Model for .NET

Port of Akka (for JVM/Scala)

http://getakka.net

NuGet package – search for Akka.Net (not the first result usually!)

STARTING WITH AKKA.NET Akka.NET is a hierarchical system

Root of the system is ActorSystem

Expensive to instantiate – create and hold!

ActorSystem can then be used to create Actors

CREATING AN ACTOR An Actor has a unique address Can be accessed/communicated to like a URI

Call ActorOf off ActorSystem, provide it a name URI (actor path) is created hierarchically

Actor Path = akka://myName/user/announcer

SENDING A MESSAGE Messages are basis of actor communication Should be IMMUTABLE!!!

“Tell” an actor a message Async call

Immutable!!

RECEIVING A MESSAGE Create your Actor

Derive from ReceiveActor

In constructor, register your message subscriptions

RECEIVING A MESSAGE Each actor has a mailbox

Messages are received in the actor’s mailbox (queue)

Actor is notified that there’s a message It receives the message Takes it out of the mailbox Next one is queued up

Ordered delivery by default Other mailboxes (like Priority) that are out-of-order

Actor

Mai

lbox

Msg

HELLO AKKA.NET Demo

CREATING CHILDREN Best to call ActorOf to get an IActorRef

Create a static Props creator

Actor path is relative to parent’s actor path

akka://myDemo/user/countChocula/{childName}

When creating a child, good idea to name it So you can reference it later!

WORD COUNTER WITH ACTORS Program sends StartCount message to CountSupervisor CountSupervisor, for each line, sends line to one of 5 LineCounterActors (ReadLineForCounting message)

LineCounterActor counts the word in that line Tells the Sender (CountSupervisor) the results (MappedList message) CountSupervisor aggregates results of MappedList messages When all lines complete, CountSupervisor sends each LineActor a Complete message

LineCounterActor cleans up and sends CountSupervisor Complete When all LineCounterActors are Complete, CountSupervisor prints out top 25 words

MAP-REDUCE DEMO

ROUTING Map Reduce example used Round Robin There are others OOTB Broadcast Random Consistent Hashing (used by Cluster) Smallest Mailbox More

Pools and Groups Pools are more anonymous Groups are pre-created

FINDING CHILDREN Within an actor, call Context.Child with the name of the child actor

If not found, ActorRefs.Nobody is returned

If found, you have an IActorRef to work with Using ActorSelection doesn’t guarantee actor is initialized

MORE COMPLEX ACTOR DEMO

baseball

gameCoordinator

gameInfo-x

gameInfo-y

playerSupervisor

batter-abatter-bbatter-c

c-01 c-11 c-32

INTEGRATION Integration with other systems is simple

Best to encapsulate in its own actor

Contain any errors there Prevent affecting parts of the system further upstream

ASYNC Actors have messages delivered to them in order via a Mailbox As message is received, handed to Actor to process Messages in mailbox queue up As a result, async processing of messages isn’t possible Receive<> (async msg => { … }) doesn’t work

If you need to perform async tasks within Receive PipeTo(Self) usually does the trick Re-queues message back on Mailbox When delivered, actor resumes

Actor

Mai

lbox

Msg

PipeTo

ACTORS ACROSS PROCESSES Akka.Remote module

Configure via HOCON Human Optimized Configuration Object Notation

Allows an actor system to distribute messages to actors across processes

Location transparency!!

ACTORS ACROSS PROCESSES

baseball

gameCoordinator

gameInfo-x

gameInfo-y

playerSupervisor

batter-abatter-bbatter-c

eventSupervisor

batter-a batter-b

c-01 c-11 c-32

atBatWriter Remotely call actor across process

ACTORS ACROSS PROCESSES Actor Path modified akka.tcp://localhost:50000@{system}/user/{supervisor}/{name}

Protocol and location added to actor path

Protocol defaults to tcp, but you can provide your own

Actor path above is remote path

AKKA.REMOTE DEMO

ELASTICITY WITH ACTORS Akka.Cluster

Multiple nodes of an actor system acting as one

Allows for failure

Can also expand out for performance

SAVING ACTOR STATE Akka.Persistence

Persist each message (or snapshots) to storage

Like event sourcing

Rehydrate actor state after failure or for whatever reason

ASKING AN ACTOR Wait for a response from an actor

Somewhat of an anti-pattern for Actor Model

If used, use sparingly

HANDLING ERRORS An exception during processing with a classic app could crash the whole system

An exception within a child actor is isolated

IS AKKA.NET…? Pub/Sub? No, but it could be

Message Bus, like RabbitMQ or MassTransit Uses messages to communicate Message bus concerned with building a distributed system topology

Storm, ETL/ELT, etc.? Kind of Can build system like that with Akka.NET

RESOURCES http://getakka.net

Petabridge bloghttps://petabridge.com/blog/

Gitter https://gitter.im/akkadotnet/akka.nethttps://gitter.im/petabridge/akka-bootcamp