an introduction to celluloid

8/13/2019 An Introduction to Celluloid

1/30

An Introduction to Celluloid

A few years ago, there used to be a very easy way to optimize code. If you found out that

your processor-heavy code was running a bit slower than you wanted it to, the simple

solution was just to wait for the next hardware iteration, which would magically amp up theclock rate on the !"s and your app would suddenly run faster.

"nfortunately, those times are now past. #ecause of a little something called $transition

energy% inside the logic gates &the incredibly tiny electronically controlled $switches% that

run processors', it has become close to impossible to push the clock rate any further at a

reasonable price. #ut, a smart solution was (uickly uncovered. Instead of trying to push one

processor to run at break neck speed, we instead spread out the workload over multiple

processors. )hat an excellent idea*

+or people that write software, understanding concurrency and using it well suddenly

becomes incredibly important. In order to scale up with the hardware, your code " useconcurrency* his seemed simple enough to everyone at first/ it couldn0t be that hard right1

)ell, it was. he mechanism that is &arguably' most popular for concurrency is threads,

which are immensely complicated. A few small mistakes here and there can wreak havoc. If

you0re really interested in this sort of stuff, I0d recommend youreadupalittle.

If you were brave enough to click a a few of those links, it shouldn0t take long to realize that

thread management isn0t always a walk in the park. onsider, as an example, deadlocking.

(dead) Locking

uppose you have two people that have a single copy of a book that they both want to read.

Assuming that they won0t both be able to read the same book at the same time, so one person

will have to wait until the other is done reading it. 2ow, consider a situation where each

person thinks that the other person isn0t done reading the book and, as such, decides not to

start reading it. 2ow, neither person reads the book.

his situation is known as a deadlock.

If your program has two $threads% &which are analogous to the two people we discussed' and

they both write to a file called $steve%. #oth of them cannot write to the file at the same time,

so the file is $locked% while one thread is writing to it. A similar situation can be present forshared variables.

3f course, this situation works just great, but the problem occurs when there is some error

that causes the file to be locked for both threads. If you0ve written any amount of thread

driven code, you know this happens a lot. )hen it does, it is a complete pain to track down

and fix.

Race Conditions

In deadlocking, the two threads wait around for each other forever. 4owever, deadlocking

has a cousin that is just as bad, if not worse.

5
http://en.wikipedia.org/wiki/POSIX_Threadshttp://en.wikipedia.org/wiki/POSIX_Threadshttp://en.wikipedia.org/wiki/POSIX_Threadshttp://en.wikipedia.org/wiki/Deadlockhttp://en.wikipedia.org/wiki/Deadlockhttp://en.wikipedia.org/wiki/Green_threadshttp://en.wikipedia.org/wiki/Race_conditionhttp://en.wikipedia.org/wiki/Race_conditionhttp://en.wikipedia.org/wiki/POSIX_Threadshttp://en.wikipedia.org/wiki/Deadlockhttp://en.wikipedia.org/wiki/Green_threadshttp://en.wikipedia.org/wiki/Race_condition


2/30

6ace conditions occur when two threads try to access and write to a variable at the same time.

o, both threads read the variable, then each one races to see who can write to the variable

first7last.

his causes all kinds of problems, because it might cause one thread0s changes to the variable

to be completely hammered by the other.

Solution?

he two problems mentioned above are just the tip of the iceberg 8 there0s all kinds of other

issues to be dealt with when using threads.

Around the 59:;http?77celluloid.io7@

makes this stuff happen and that0s what we0ll be using*

Celluloid

elluloid brings the actor model to 6uby, making writing concurrent applications incredibly

easy.

+irst of all, elluloid comes with built in deadlock protection, because all of the messaging

between actors between is handled in a such a way that deadlocking is darn near impossible,

as long as you0re not doing something crazy or messing with native &i.e. ' code.
http://www.sitepoint.com/introduction-to-event-machinehttp://www.sitepoint.com/introduction-to-event-machine


3/30

If you0re familiar with =rlang &it is okay if you0re not', elluloid borrows one of its most

important ideas? fault tolerance. elluloid automatically restarts and handles crashed actors,

so you don0t have to worry about every last thing that could go wrong.

here0s all kinds of other features &linking, futures, etc.' that make threading a breeze, but

there a few things to keep in mind.

The GIL

he $regular% or vanilla ruby that we0re all used to is backed by either 6I or BA6C, which

are different types of interpreters7vitual machines for 6uby.

2ow, there is, debatably, a problem with this interpreter. he thing is, all threads inside

6I7BA6C aren0t reallyconcurrent 8 everything is run under a single thread. his is called

the Dlobal Interpreter Eock. 6uby isn0t the only language with an interpreter that has this 8 so

does !ython and don0t even get me started on threading with !4!.

)hen a new thread is created, the result is the computation that is performed isn0t actually

performed at the same time as other stuff is being done. An illusion is created that makes the

user think that this is what is happening.

+ortunately, there is a solution. Fust use a different interpreter* ake your pick?

&F6uby'>jruby.org@

&6ubini.us'>http?77rubini.us7@

2ote that if you do choose to go with one of the above interpreters, make sure you areoperating in 5.9 mode in order to be compaitible with elluloid.

Diving In

Eet0s get started with elluloid by writing a small actor 8 let0s make it read a file we tell it to,

and then display the results when they are wanted.

require 'celluloid'

class FilePutter include Celluloid

def initialize(filename) @filename = filename enddef load_file

@file_contents = File.read @filename end

def print p @file_contents end

end

G


4/30

fp = FilePutter.new "/ar/lo!/ernel.lo!"fp.load_filefp.print

6unning that &with your choice of interpreter', you should get a dump of the kernel log &of

course, assuming you0re using a !3IH system, )indows users can replace that line with

any file of their choice' followed by a message from elluloid telling you that two actors

have been terminated.

Alright, so, what just happened1

)e defined the +ile!utter actor and created an instance of it, which elluloid automatically

pushes into its own thread* here0s no difference in how we are calling the methods/ it is just

like we would for a regular class, and it dumps the contents of a file.

+irst, calling load_fileloads the file, then we proceed on to printing the contents. 2ot toocomplicated.

#ut, one thread isn0t all that interesting/ how about five1 =asy enough?

require 'celluloid'


def initialize(filename) @filename = filename

enddef load_file

@file_contents = File.read @filename end

def print p @file_contents end

end

files = #"/ar/lo!/ernel.lo!"$ "/ar/lo!/s%stem.lo!"$ "/ar/lo!/ppp.lo!"$"/ar/lo!/secure.lo!"&

files.eac do file fp = FilePutter.new file fp.load_file fp.printend

And, just like that, we0ve created five threads which each read the files in the $files% array.

#ut, all of the methods we0ve called so far have been called syncronously, meaning that we

have to wait for them to end before proceeding. )hat if we just pushed them to the side andmoved on1


5/30

his is where elluloid really shines?

require 'celluloid'


def initialize(filename) @filename = filename enddef load_file_and_print

@file_contents = File.read @filename p @file_contents end

end

files = #"/ar/lo!/ernel.lo!"$ "/ar/lo!/s%stem.lo!"$ "/ar/lo!/ppp.lo!"$

"/ar/lo!/secure.lo!"&

files.eac do file fp = FilePutter.new file fp.as%nc.load_file_and_printend

his is where it gets interesting. +irst of all, we combined the loading of the file and printing

the contents into one method, namely load_file_and_print. hen, notice inside the loop

over the files array, we don0t call load_file_and_print, instead, we call

async.load_file_and_print .

Wraing It !

)hen we call the given method with .async., elluloid runs that call asynchronously,

allowing our program to move right along without waiting for the file to load or the printing

to occur.

he asynchronous message delivering isn0t perfect. he message might not be delivered, the

actor might not respond, etc.

#ut, how do you figure out when this happens1

Also, threads are never really completely independent of each other 8 how do you have them

talking to each other1

his, and, several other important actor model features and niceties are coming in part II, so,

stay tuned*

If you felt this article went a bit too slow, you0ll be satisfied with !art II ?'

Pleaseask (uestions if you have any in the comments section.

J


6/30

his is the second article in the three-part series. If you missed the first one, you can find it

here

elluloid has a ton more awesome tools to make concurrent programming incredibly easy in

6uby.

Eet0s take a look at them.

Futures

here are times when we don0t just want to discard the return value of a method we0ve called

on an actor/ instead, we might want to use it somewhere else. +or that, elluloid provides

futures. he best way to learn about them is to see them in action.

)e0ll write a small script that computes the 4A5 checksum of an array of files, then outputs

them to the console.

)ithout further ado, here it is?

require 'celluloid'require 'di!est/sa'

class *+,Putterinclude Celluloid

def initialize(filename) @filename = filename

end

def output(cecsum_future) puts "-@filename 0 -cecsum_future.alue" enddef cecsum

@file_contents = File.read(@filename)

1i!est22*+,.e3di!est @file_contents endend

files = #"/ar/lo!/ernel.lo!"$ "/ar/lo!/s%stem.lo!"$ "/ar/lo!/ppp.lo!"$"/ar/lo!/secure.lo!"&

files.eac do file sa = *+,Putter.new file cecsum_future = sa.future 2cecsum sa.output cecsum_futureend

+irst of all, consider the checksum method. It is (uite straightforward, we use the

Kigest??4A5 to compute the checksum of the contents of a file that the actor is given.

L
http://www.sitepoint.com/an-introduction-to-celluloid-part-i/http://www.sitepoint.com/an-introduction-to-celluloid-part-i/


7/30

Eook at the files.each loop. his is where it gets interesting.

+irst, we create the actor and assign it a file. hen, instead of just calling the checksum

method, we call it using a future. #y doing this, a elluloid??+uture object is immediately

returned, instead of blocking.

hen, we take this future object and pass it on to the output method inside the actor.

Inside the output method, the value of the checksum is needed* o, it is attained from the

future object0s value method, which blocks until a value is available. hat solves the

problem*

Bou might be thinking, $hey, this does pretty much the same thing as the last example*%

4owever, in the last example, in order to do the file related operations asynchronously, we

dumped everything into a single method. )ith futures, we are able to cleanly seperate our

code.

Also, there are use cases where it is only possible to use futures. +or example, if one iswriting a library, the result of the checksum function must be a future since the user of the

library should be able to add in their own code.

Making An" #lock Concurrent

here is a verycool use for futures, namely, they allow us to push block of code to another

thread incredibly easily.

heck it out?

require 'celluloid'

def some_metod(future) -do sometin! craz% al = future.alue -do sometin! wit alend

future = Celluloid22Future.new do -incredi4l% comple3 computationend

some_metod(future)

)e use elluloid??+uture to push a block into its own thread. elluloid manages everything

about that thread, whose return value we can use later on &using the future0s return value, of

course'. o, this little part of elluloid can be plugged into literally anyapplication and once

mastered, can be incredibly useful.

"se it wisely*

M


8/30

Catching $rrors % Suervisors

o see how error handling works in elluloid, we0re going to build a simple tool that gets the

4E of various websites.

4ere it is, with the stuff we0ve learned so far?

require 'celluloid'require 'net/ttp'

class 5arupPutterinclude Celluloid

def initialize(url) @url = url end

def output(marup_future) puts "-@url" puts "-marup_future.alue" puts "0000000000000000000000000000" enddef !et_marup

6et22+77P.!et(89:.parse(@url)) endend

we4sites = #"ttp2//!oo!le.com/"$ "ttp2//%aoo.com/"$"ttp2//ru4%source.com/"$ "ttp2//tum4lr.com/"&

we4sites.eac do we4site mp = 5arupPutter.new we4site marup_future = mp.future 2!et_marup mp.output marup_futureend

If everything goes well, the markup is putsd.

#ut, what if things start going wrong1 )e0re not really doing much about that.

+or that purpose, elluloid provides a mechanism known as a supervisor. 4ere it is in action?

require 'celluloid'require 'net/ttp'

class 5arupPutterinclude Celluloid

def initialize(url) @url = url end

def output(marup_future) puts "-@url"

:


9/30

puts "-marup_future.alue" puts "0000000000000000000000000000" enddef !et_marup

6et22+77P.!et(89:.parse(@url))

endend

we4sites = #"ttp2//!oo!le.com/"$ "ttp2//%aoo.com/"$"ttp2//ru4%source.com/"$ "ttp2//tum4lr.com/"&

we4sites.eac do we4site superisor = 5arupPutter.superise_as 2mp$ we4site mp = Celluloid22,ctor#2mp&

marup_future = mp.future 2!et_marup mp.output marup_future

end

here0s several new concepts here, so pay close attention.

+irst of all, the 5arupPutterclass is left untouched. In other words, the implementation of

the business logic is left unchanged*

2ow, we call the superisemethod on the 5arupPutterclass. his does three things, first,

it creates &and puts into motion' an actor that is an instance of 5arupPutter. econdly, it

returns a supervisor object, which can do some interesting things. +inally, it takes its first

argument &which is $mp%', and puts an entry of that name in the registry.

he elluloid registry is a bit like a phonebook 8 the actors that are in there can be accessed

by name. o, on the next line, we use the elluloid registry to look up 2mp.

he code after that is (uite straightforward 8 simply using a future to output the markup.

)ith two lines of code added, elluloid automatically takes care of restarting and keeping

track of actors when they crash*

In case one of the actors hits some kind of exception &e.g. the website does not respond to the

re(uest and the re(uest times out', the actor is immediately restarted by the elluloid core. If

you0ve written this kind of threading code the old fashioned way, you know that this is a very

finicky and difficult process, but it is handled entirely by elluloid for us*

Co&&unication #et'een Actors

In nearly all applications, actors will not be working in isolated environments 8 they will be

communicating with other actors.

Fust to explain how communication between actors works in elluloid, we0ll write three

actors to print out $4ello, world*% when run correctly. heck it out?

9


10/30

require 'celluloid'

class +ello*pace,ctor include Celluloid def sa%_ms! print "+ello$ "

Celluloid22,ctor#2world&.sa%_ms! endend

class ;orld,ctor include Celluloid def sa%_ms! print "world


11/30

ooling

If you have read up a bit about how web servers operate, you know how important thread

pools are. !ools in elluloid are awesome/ they are completely transparent. I think they are

probably my favorite feature of elluloid &with so much cool stuff, its hard to choose*'.

)e0ll write a simple example to demonstrate how amazing they are?

require 'celluloid'require 'matn'

class Prime;orer include Celluloid

def prime(num4er) if num4er.prime> puts num4er

end endend

pool = Prime;orer.pool

(?..).to_a.map do i pool.prime< iend

sleep

+irst, we define the Prime;orerclass. he $)orker% in the name signifies that it is to beused with a pool 8 threads that are part of thread pools are usually called workers.

he function of the primemethod in Prime;oreris to print a number if it is prime &this

uses the Nmathn0 module introduced in 5.9 8 you can write your own prime number checker if

you like'.

he interesting part is when we introduce the pool by calling the pool method on

Prime;orer.

he $pool% object has all the methods of Prime;orer, but, it actually creates as many

instances of Prime;oreras the processor has cores. herefore, if you have a (uad core

processor, that would create four actors. )hen methods are called on $pool%, elluloid

decides which actor out of the pool to invoke.

+ollowing that, we have a map over a large range, in which we call prime &remember, it is

called asynchronously because of the bang' on pool. his automatically distributes the

workload over your processors*

)ow. It took maybe four or five lines of code extra to acheive complete concurrency. hat0s

amazing.

55


12/30

At the end of the program, there is a sleep call. here is a good reason for this. ince we are

calling primeasynchronously, the main thread &which is the 6uby thread' exits when it is

done telling all the actors $hey, remember to print out this prime%. 4owever, the actors aren0t

done actually printing the primes by the time the main thread exits, so the output never

reaches the terminal.

#ut, the sleep command keeps the main thread alive for long enough so that all the output

comes out correctly. Also notice that since we are calling primeasynchronously, there is no

gurantee of the orderof the primes that are outputted.

Wraing It !

I hope you enjoyed the article, and that you0re as excited about elluloid as I am.

o far, we0ve discussed how to use the various parts of elluloid are to be used seperately

with small examples.

In Part 3, we0ll cover how all of this ties together, create some more complex programs, and

cover more features, such as Einking.

Ko ask any (uestions you have in the comments section below ?'

The Big Project

o, assuming you0ve read the previous parts of this series, you should have a somewhat

working knowledge of the variouspartsof elluloid.

#ut, so far, we haven0t really covered how all of these parts come together to form a

complete application. )e0re going to do that here*

o, let0s dive in*

What is it?

)hat will be our objective1 Its actually (uite simple? we0ll build an 4! server.

ore precisely, we0ll build a very incomplete, yet easily extendable, 4! server written in

6uby with the elluoid library.

Kon0t expect it to be something like Apache or hin 8 but, we0ll learn a ton in the process

about elluloid and 4!, which &arguably' is the important networking protocol for

developers to understand.

A Little Aout *TT

3f course, if we want to build an 4! server, we need to be in the know about what 4!

is and how it functions.

5


13/30

In case you don0t know, 4! is a protocol that is used to transfer 4E from server to

client &this isn0t alwaysthe case, but it is the most common', and the most common, for the

client to tell the server to do certain things.

he protocol is based on re(uests. o, the client can issue a $D=% re(uest to the server in

order to get the 4E for a certain page. Also, the client can issue a $!3% re(uest toprovide some data to the server.

An important point about 4! is that the protocol itself isstateless. his means that each

re(uest has no knowledge of any previouse re(uests.

)e will only implement the D= re(uest/ this is for two reasons. +irstly, it is a core function

of the 4! server. econd, it is very easy to implement.

#ut, we will write our code so that it is modular enough for other methods to be added (uite

easily.

Starting +ut

+irst of all, let0s build a (uick and dirty prototype.

6uby has built in support for socket communication, which is all attained with the simple

$re(uire Nsocket0% statement. "sing that, and all the magic from elluloid, here is our

prototype?

require 'socet'require 'celluloid'

class +77P*erer include Celluloid

def initialize(port) @port = port end

def start @serer = 7CP*erer.new(@port) loop client = @serer.accept eaders = "+77P/. ? ABrn1ate2 7ue$ 1ec ? 2D2E

57rn*erer2 9u4%rnContent07%pe2 te3t/tmlG carset=iso0DDEH0rnrn" client.puts eaders client.puts "ItmlJI/tmlJ" client.close endend

s = +77P*erer.new Ks.start

)e have an +77P*ererclass, which revolves under its $start% method. his method simplystarts a server, using the 7CP*ererclass from the socket module'.

5G


14/30

hen, we do @serer.accept. his is very important, because this is a blocking call.

eaning that work won0t move any further until a client has come in to be served.

hen, we simply write the 4! headers and 4E &regardless of what type of re(uest

we0ve received.'

3f course, there0s an aparent problem. his isn0t concurrent. ince all of the calls we0re using

block, we0re just doing each client synchronously. hat0s no good*

Taking it As"nc

he solution to this dilemma comes in the form of writing another actor.

4ere0s the code?

require 'socet'

require 'celluloid'

class ,nswer,ctor include Celluloid

def initialize(client) @client = client end

def start @client.puts

eaders = "+77P/. ? ABrn1ate2 7ue$ 1ec ? 2D2E57rn*erer2 9u4%rnContent07%pe2 te3t/tmlG carset=iso0DDEH0rnrn"

@client.puts eaders @client.puts "ItmlJI/tmlJ"

loop -ind of Lust an! around and 4loc te actor

endend



def start @serer = 7CP*erer.new(@port) loop aa = ,nswer,ctor.new @serer.accept puts "waddup" aa.start< end

end

5


15/30


hat0s pretty big to process at once, but we0ll simply take it step by step.

,nswer,ctoris just another actor 8 it creates a new thread when an instance of it is created.

he real work is happening in the startmethod.

4ere, we get a hold of the client socket &with which we can talk to the client' and write the

header and 4E to it and then just wait around.

In the +77P*ererclass &which is also an actor*', we call aa.start


16/30

def start @serer = 7CP*erer.new(@port) client = nil pool = ,nswer;orer.pool(size2 E)

loop client = @serer.accept pool.start< client end

end


If you look at the modified code in +77P*erer.start, it is (uite clear. )e simply create a

pool of J; actors &and, therefore, J; threads' which we can then assign work. All of that with

one line of code. hat0s awesome*

)hy J;1 )ell, just a random number I picked. here0s been some good workbehind

selecting optimal sizes of thread pools, if you0re interested.

In case an exception occurs for an actor within the pool, it will be restarted automatically and

ready to use the next time it is needed*

Ans'ering Re-uests

o far, we0ve only spit back just a pair of html tags 8 we haven0t actually listened to what the

user is re(uesting. Eet0s work that in.

Coila?


class Muer% attr_accessor 2t%pe$ 2url$ 2oter

def initialize (quer%_strin!) @t%pe$ @url$ @oter = quer%_strin!.split " "end

end

class ,nswer;orer include Celluloid

def process_!et ... end

def start(client) @client = client

5L
http://tmullen.ist.psu.edu/pubs/threadpool.pdfhttp://tmullen.ist.psu.edu/pubs/threadpool.pdfhttp://tmullen.ist.psu.edu/pubs/threadpool.pdf


17/30

client.putseaders = "+77P/. ? ABrn1ate2 7ue$ 1ec ? 2D2E

57rn*erer2 9u4%rnContent07%pe2 te3t/tmlG carset=iso0DDEH0rnrn"

client.puts eaders

loop quer% = Muer%.new client.readline process_!et if quer%.t%pe == "N7"

endend


def initialize(port)

@port = port end

def start @serer = 7CP*erer.new(@port) client = nil pool = ,nswer;orer.pool(size2 E)

loop client = @serer.accept pool.start< client

endend

s = +77P*erer.new OOOs.start

Again, there are a number of changes. )e0ve added the Muer%class, which has a type, url

and other attributes to it, representing an 4! re(uest.

As an example, a D= re(uest looks like $D= 7index.html 4!75.5O, where 7index.html is

the "6E being re(uested and 4!75.5 is the protocol being used &as opposed to 4!75.;'.

hen, in ,nswer;orer, we call a process_!etmethod if the (uery type is D=.

#ut, we haven0t defined what exactly process_!etshould do 8 let0s make a simple way to

view files inside a certain folder.

Ans'ering the G$Ts

#eware, this is a horrible implementation of process_!et, but, it is meant to be very simple

and straightforward since it really has nothing to do with the scope of this article, which is to

learn more about elluloid.

5M


18/30

4ere it is?


class Muer%

attr_accessor 2t%pe$ 2url$ 2oter

def initialize (quer%_strin!) @t%pe$ @url$ @oter = quer%_strin!.split " " end

end

class ,nswer;orer include Celluloid

def process_!et(quer%) dir_pat = "serer"

filepat = dir_pat quer%.url

if File.e3ists> [email protected] (File.open(filepat).read)

else @client.puts "Can't find file" end

end

def process_req(quer%) process_!et quer% if quer%.t%pe == "N7" end

def start(client) @client = client

eaders = "+77P/. ? ABrn1ate2 7ue$ 1ec ? 2D2E57rn*erer2 9u4%rnContent07%pe2 te3t/tmlG carset=iso0DDEH0rnrn"

client.puts eaders

loop quer% = Muer%.new client.readline process_req quer%

endend



def start

@serer = 7CP*erer.new(@port) client = nil pool = ,nswer;orer.pool(size2 E)

5:


19/30

loop client = @serer.accept pool.start< client end

end


It simply takes the $server% folder &in the same folder as the 4! server0s source code

itself', and returns the contents of a given file.

And, we0ve also sorted out the way we handle what type of re(uest has been sent by the

client, to make the server easy to extend.

2ote that, because of the simple way this has been done, directly plugging in 5M.;.;.5?G;;;

into your browser will not work 8 you will have to use 5M.;.;.5?G;;;7index.html or some

other filename*

Wrapping it Up

)e0ve built a &very' rudimentary 4! server using elluloid.

In doing so, we0ve seen how actors can work together in simple situations.

3f course, you will write applications larger than this one using elluloid &I sure hope so*'

and you probably will use more advanced features. 6emember, however, that the basics are

very important. I hope seeing some of the concepts come together made the learning easier.

If you liked the article, do weet it out to your followers. hanks for reading*

59


20/30

A gentle introduction to actor-based concurrency

his issue was a collaboration with Alberto +ernPndez apel, a 6uby developer from pain.

Although it has been through many revisions since we started, AlbertoQs ideas, code, and

explanations provided an excellent starting point that lead us to publish this article.

onventional wisdom says that concurrent programming is hard, especially in 6uby. his

basic assumption is what lead many 6ubyists to take an interest in languages like =rlang and

cala -- their baked in support for theactor modelis meant to make concurrent systems much

easier for everyday programmers to implement and understand.

#ut do you really need to look outside of 6uby to find concurrency primitives that can make

your work easier1 he answer to that (uestion probably depends on the levels of concurrency

and availability that you re(uire, but things have definitely been shaping up in recent years.

In particular, the elluloidframework has brought us a convenient and clean way to

implement actor-based concurrent systems in 6uby.

In order to appreciate what elluloid can do for you, you first need to understand what the

actor model is, and what benefits it offers over the traditional approach of directly using

threads and locks for concurrent programming. In this article, weQll try to shed some light on

those points by solving a classic concurrency puzzle in three ways? "sing 6ubyQs built-in

primitives &threads and mutex locks', using the elluloid framework, and using a minimal

implementation of the actor model that weQll build from scratch.

#y the end of this article, you certainly wonQt be a concurrency expert if you arenQt already,

but youQll have a nice head start on some basic concepts that will help you decide how to

tackle concurrent programming within your own projects. EetQs begin*

The Dining hilosohers role&

he Kining !hilosophersproblem was formulated by =dsger Kjisktra in 59LJ to illustrate the

kind of issues we can find when multiple processes compete to gain access to exclusive

resources.

In this problem, five philosophers meet to have dinner. hey sit at a round table and each one

has a bowl of rice in front of them. here are also five chopsticks, one between each

philosopher. he philosophers spent their time thinking about The Meaning of Life.

)henever they get hungry, they try to eat. #ut a philosopher needs a chopstick in each hand

in order to grab the rice. If any other philosopher has already taken one of those chopsticks,

the hungry philosopher will wait until that chopstick is available.

his problem is interesting because if it is not properly solved it can easily lead to deadlock

issues. )eQll take a look at those issues soon, but first letQs convert this problem domain into a

few basic 6uby objects.

Modeling the table and its chopsticks

All three of the solutions weQll discuss in this article rely on a Copsticclass and a 7a4leclass. he definitions of both classes are shown below?

;
https://github.com/afcapelhttp://en.wikipedia.org/wiki/Actor_modelhttp://en.wikipedia.org/wiki/Actor_modelhttp://celluloid.io/http://celluloid.io/http://en.wikipedia.org/wiki/Dining_philosophershttps://github.com/afcapelhttp://en.wikipedia.org/wiki/Actor_modelhttp://celluloid.io/http://en.wikipedia.org/wiki/Dining_philosophers


21/30

class Copstic def initialize @mute3 = 5ute3.new end

def tae

@mute3.loc end

def drop @mute3.unloc

rescue 7readNrror puts "7r%in! to drop a copstic not acquired" end

def in_use> @mute3.loced> end

end

class 7a4le def initialize(num_seats) @copstics = num_seats.times.map Copstic.new end

def left_copstic_at(position) inde3 = (position 0 ) Q @copstics.size @copstics#inde3& end

def ri!t_copstic_at(position)

inde3 = position Q @copstics.size @copstics#inde3& end

def copstics_in_use @copstics.select f f.in_use> .size endend

he Copsticclass is just a thin wrapper around a regular 6uby mutex that will ensure that

two philosophers can not grab the same chopstick at the same time. he 7a4leclass deals

with the geometry of the problem/ it knows where each seat is at the table, which chopstick is

to the left or to the right of that seat, and how many chopsticks are currently in use.

2ow that youQve seen the basic domain objects that model this problem, weQll look at different

ways of implementing the behavior of the philosophers. )eQll start with what doesn'twork.

A solution that leads to deadlocks

he Pilosoperclass shown below would seem to be the most straightforward solution to

this problem, but has a fatal flaw that prevents it from being thread safe. an you spot it1

class Pilosoper

def initialize(name) @name = name end

5


22/30

def dine(ta4le$ position) @left_copstic = ta4le.left_copstic_at(position) @ri!t_copstic = ta4le.ri!t_copstic_at(position)

loop do

tin eat end end

def tin puts "-@name is tinin!" end

def eat tae_copstics

puts "-@name is eatin!."

drop_copstics end

def tae_copstics @left_copstic.tae @ri!t_copstic.tae end

def drop_copstics @left_copstic.drop @ri!t_copstic.drop end

end

If youQre still scratching your head, consider what happens when each philosopher object is

given its own thread, and all the philosophers attempt to eat at the same time.

In this naive implementation, it is possible to reach a state in which every philosopher picks

up their left-hand chopstick, leaving no chopsticks on the table. In that scenario, every

philosopher would simply wait forever for their right-hand chopstick to become available --

resulting in a deadlock. Bou can reproduce the problem by running the following code?

names = Qw+eraclitus ,ristotle Npictetus *copenauer Popper

pilosopers = names.map name Pilosoper.new(name) ta4le = 7a4le.new(pilosopers.size)

treads = pilosopers.map.wit_inde3 do pilosoper$ i 7read.new pilosoper.dine(ta4le$ i) end

treads.eac(R2Loin)sleep

6uby is smart enough to inform you of what went wrong, so you should end up seeing a

backtrace that looks something like this?

,ristotle is tinin!


23/30

Popper is eatin!.Popper is tinin!Npictetus is eatin!.Npictetus is tinin!+eraclitus is eatin!.+eraclitus is tinin!

*copenauer is eatin!.*copenauer is tinin!

dinin!_pilosopers_uncoordinated.r42OH2in SLoin'2 deadloc detected(fatal) from dinin!_pilosopers_uncoordinated.r42OH2in Seac' from dinin!_pilosopers_uncoordinated.r42OH2in SImainJ

In many situations, the most simple solution tends to be the best one, but this is obviously not

one of those cases. ince weQve learned the hard way that the philosophers cannot be safely

left to their own devices, weQll need to do more to make sure their behaviors remain

coordinated.

A coordinated mutex-based solution

3ne easy solution to this issue is introduce a ;aiterobject into the mix. In this model, the

philosopher must ask the waiter before eating. If the number of chopsticks in use is four or

more, the waiter will make the philosopher wait until someone finishes eating. his will

ensure that at least one philosopher will be able to eat at any time, avoiding the deadlock

condition.

hereQs still a catch, though. +rom the moment the waiter checks the number of chopstick in

use until the next philosopher starts to eat we have a critical region in our program? If we let

two concurrent threads execute that code at the same time there is still a chance of a

deadlock. +or example, suppose the waiter checks the number of chopsticks used and see it is

G. At that moment, the scheduler yields control to another philosopher who is just picking the

chopstick. )hen the execution flow comes back to the original thread, it will allow the

original philosopher to eat, even if there may be more than four chopsticks already in use.

o avoid this situation we need to protect the critical region with a mutex, as shown below?

class ;aiter def initialize(capacit%) @capacit% = capacit%

@mute3 = 5ute3.new end

def sere(ta4le$ pilosoper) @mute3.s%ncronize do sleep(rand) wile ta4le.copstics_in_use J= @capacit%

pilosoper.tae_copstics end

pilosoper.eat endend

Introducing the ;aiterobject re(uires us to make some minor changes to our Pilosoperobject, but they are fairly straightforward?

G


24/30

class Pilosoper

- ... all omitted code same as 4efore

def dine(ta4le$ position$ waiter) @left_copstic = ta4le.left_copstic_at(position)

@ri!t_copstic = ta4le.ri!t_copstic_at(position)

loop do tin

- instead of callin! eat() directl%$ mae a request to te waiterwaiter.sere(ta4le$ self)

end end

def eat - remoed tae_copstics call$ as tat's now andled 4% te waiter

puts "-@name is eatin!."

drop_copstics endend

he runner code also needs minor tweaks, but is mostly similar to what you saw earlier?


pilosopers = names.map name Pilosoper.new(name)

ta4le = 7a4le.new(pilosopers.size)

waiter = ;aiter.new(pilosopers.size 0 )

treads = pilosopers.map.wit_inde3 do pilosoper$ i 7read.new pilosoper.dine(ta4le$ i$ waiter) end

treads.eac(R2Loin)sleep

his approach is reasonable and solves the deadlock issue, but using mutexes to synchronize

code re(uires some low level thinking. =ven in this simple problem, there were several

gotchas to consider. As programs get more complicated, it becomes really difficult to keep

track of critical regions while ensuring that the code behaves properly when accessing them.

he actor model is meant to provide a more systematic and natural way of sharing data

between threads. )eQll now take a look at an actor-based solution to this problem so that we

can see how it compares to this mutex-based approach.

An actor.ased solution using Celluloid

)eQll now rework our Pilosoperand ;aiterclasses to make use of elluloid. uch of

the code will remain the same, but some important details will change. he full class

definitions are shown below to preserve context, but the changed portions are marked withcomments.


25/30

)eQll spend the rest of the article explaining the inner workings of this code, so donQt worry

about understanding every last detail. Instead, just try to get a basic idea of whatQs going on

here?

class Pilosoper

include Celluloid

def initialize(name) @name = name end

- *witcin! to te actor model requires us !et rid of our - more procedural eent loop in faor of a messa!e0oriented - approac usin! recursion. 7e call to tin() eentuall% - leads to a call to eat()$ wic in turn calls 4ac to tin()$ - completin! te loop.

def dine(ta4le$ position$ waiter)

@waiter = waiter

@left_copstic = ta4le.left_copstic_at(position) @ri!t_copstic = ta4le.ri!t_copstic_at(position)

tin end

def tin puts "-@name is tinin!." sleep(rand)

- ,s%ncronousl% notifies te waiter o4Lect tat - te pilosopor is read% to eat

@waiter.as%nc.request_to_eat(,ctor.current) end

def eat tae_copstics

puts "-@name is eatin!." sleep(rand)

drop_copstics

- ,s%ncronousl% notifies te waiter - tat te pilosoper as finised eatin!

@waiter.as%nc.done_eatin!(,ctor.current)

tin end

def tae_copstics @left_copstic.tae @ri!t_copstic.tae end

def drop_copstics

@left_copstic.drop @ri!t_copstic.drop

J


26/30

end

- 7is code is necessar% in order for Celluloid to sut down cleanl% def finalize drop_copstics end

end

class ;aiter include Celluloid

def initialize @eatin! = #& end

- 4ecause s%ncronized data access is ensured - 4% te actor model$ tis code is muc more - simple tan its mute304ased counterpart. +oweer$

- tis approac requires two metods - (one to start and one to stop te eatin! process)$ - were te preious approac used a sin!le sere() metod.

def request_to_eat(pilosoper) return if @eatin!.include>(pilosoper)

@eatin! II pilosoper pilosoper.as%nc.eat end

def done_eatin!(pilosoper) @eatin!.delete(pilosoper)

endend

he runner code is similar to before, with only some very minor changes?


pilosopers = names.map name Pilosoper.new(name)

waiter = ;aiter.new - no lon!er needs a "capacit%" ar!umentta4le = 7a4le.new(pilosopers.size)

pilosopers.eac_wit_inde3 do pilosoper$ i

- 6o lon!er manuall% create a tread$ rel% on as%nc() to do tat for us. pilosoper.as%nc.dine(ta4le$ i$ waiter)end

sleep

he runtime behavior of this solution is similar to that of our mutex-based solution. 4owever,

the following differences in implementation are worth noting?

=ach class that mixes in Celluloidbecomes an actor with its own thread of

execution.

L


27/30

he elluloid library intercepts any method call run through the as%ncproxy object

and stores it in the actorQs mailbox. he actorQs thread will se(uentially execute those

stored methods, one after another.

his behavior makes it so that we donQt need to manage threads and mutex

synchronization explicitly. he elluloid library handles that under the hood in anobject-oriented manner.

If we encapsulate all data inside actor objects, only the actorQs thread will be able to

access and modify its own data. hat prevents the possibility of two threads writing to

a critical region at the same time, which eliminates the risk of deadlocks and data

corruption.

hese features are very useful for simplifying the way we think about concurrent

programming, but youQre probably wondering how much magic is involved in implementing

them. EetQs build our own minimal drop-in replacement for elluloid to find out*

Rolling our o'n actor &odel

elluloid provides much more functionality than what we can discuss in this article, but

building a barebones implementation of the actor model is within our reach. In fact, the

following :; lines of code are enough to serve as a replacement for our use of elluloid in the

previous example?

require 'tread'

module ,ctor - 7o use tis$ %ou'd include ,ctor instead of Celluloid

module Class5etods def new(Tar!s$ R4loc) Pro3%.new(super) end end

class II self def included(lass) lass.e3tend(Class5etods) end

def current 7read.current#2actor&

end end

class Pro3% def initialize(tar!et) @tar!et = tar!et @mail4o3 = Mueue.new @mute3 = 5ute3.new @runnin! = true

@as%nc_pro3% = ,s%ncPro3%.new(self)

@tread = 7read.new do

7read.current#2actor& = self process_in4o3

M


28/30

end end

def as%nc @as%nc_pro3% end

def send_later(met$ Tar!s) @mail4o3 II #met$ ar!s& end

def terminate @runnin! = false end

def metod_missin!(met$ Tar!s) process_messa!e(met$ Tar!s) end

priate

def process_in4o3 wile @runnin! met$ ar!s = @mail4o3.pop process_messa!e(met$ Tar!s) end

rescue N3ception =J e3 puts "Nrror wile runnin! actor2 -e3" end

def process_messa!e(met$ Tar!s)

@mute3.s%ncronize do @tar!et.pu4lic_send(met$ Tar!s) end end end

class ,s%ncPro3% def initialize(actor) @actor = actor end

def metod_missin!(met$ Tar!s) @actor.send_later(met$ Tar!s) end endend

his code mostly builds upon concepts that have already been covered in this article, so it

shouldnQt be too hard to follow with a bit of effort. hat said, combining meta-programming

techni(ues and concurrency can lead to code that makes your eyes glaze over, so we should

also make an attempt to discuss how this module works at the high level. EetQs do that now*

Any class that includes the ,ctormodule will be converted into an actor and will be able to

receive asynchronous calls. )e accomplish this by overriding the constructor of the target

class so that we can return a proxy object every time an object of that class is instantiated. )e

also store the proxy object in a thread level variable. his is necessary because when sending

:


29/30

messages between actors, if we refer to self in method calls we will exposed the inner target

object, instead of the proxy. his same gotcha is also present in elluloid.

"sing this mixin, whenever we attempt to create an instance of a Pilosoperobject, we

will actually receive an instance of ,ctor22Pro3%. he Pilosoperclass is left mostly

untouched, and so the actor-like behavior is handled entirely by the proxy object. "poninstantiation, that proxy creates a mailbox to store the incoming asynchronous messages and

a thread to process those messages. he inbox is a thread-safe (ueue that ensures that

incoming message are processed se(uentially even if they arrive at the same time. )henever

the inbox is empty, the actorQs thread will be blocked until a new message needs to be

processed.

his is roughly how things work in elluloid as well, although its implementation is much

more complex due to the many additional features it offers. till, if you understand this code,

youQre well on your way to having a working knowledge of what the actor model is all about.

Actors are helpful, but are not a golden hammer

=ven this minimal implementation of the actor model gets the low-level concurrency

primitives out of our ordinary class definitions, and into a centralized place where it can be

handled in a consistent and reliable way. elluloid goes a lot farther than we did here by

providing excellent fault tolerance mechanisms, the ability to recover from failures, and lots

of other interesting stuff. 4owever, these benefits do come with their own share of costs and

potential pitfalls.

o what can go wrong when using actors in 6uby1 )eQve already hinted at the potential

issues that can arise due to the issue of self schizophreniain proxy objects. !erhaps morecomplicated is the issue of mutable state? while using actors guarantees that the state within

an object will be accessed se(uentially, it does not provide the same guarantee for the

messages that are being passed around between objects. In languages like =rlang, messages

consist of immutable parameters, so consistency is enforced at the language level. In 6uby,

we donQt have that constraint, so we either need to solve this problem by convention, or by

freezing the objects we pass around as arguments -- which is (uite restrictive*

)ithout attempting to enumerate all the other things that could go wrong, the point here is

simply that there is no such thing as a golden hammer when it comes to concurrent

programming. 4opefully this article has given you a basic sense of both the benefits and

drawbacks of applying the actor model in 6uby, along with enough background knowledge toapply some of these ideas in your own projects. If it has done so, please do share your story.

Source code from this article

All of the code from this article is in !racticing 6ubyQs example repository, but the links

below highlight the main points of interest?

A solution that leads to deadlocks

A coordinated mutex-based solution

An actor-based solution using elluloid

An actor-based solution using a hand-rolled actor library

inimal implementation of the actor model

9
https://github.com/celluloid/celluloid/wiki/Gotchashttp://en.wikipedia.org/wiki/Schizophrenia_(object-oriented_programming)http://en.wikipedia.org/wiki/Schizophrenia_(object-oriented_programming)https://github.com/elm-city-craftworks/practicing-ruby-examples/tree/master/v6/003https://github.com/elm-city-craftworks/practicing-ruby-examples/blob/master/v6/003/mutex_uncoordinated/dining_philosophers.rbhttps://github.com/elm-city-craftworks/practicing-ruby-examples/blob/master/v6/003/mutex_coordinated/dining_philosophers.rbhttps://github.com/elm-city-craftworks/practicing-ruby-examples/blob/master/v6/003/celluloid/dining_philosophers.rbhttps://github.com/elm-city-craftworks/practicing-ruby-examples/blob/master/v6/003/actors_from_scratch/dining_philosophers.rbhttps://github.com/elm-city-craftworks/practicing-ruby-examples/blob/master/v6/003/lib/actors.rbhttps://github.com/celluloid/celluloid/wiki/Gotchashttp://en.wikipedia.org/wiki/Schizophrenia_(object-oriented_programming)https://github.com/elm-city-craftworks/practicing-ruby-examples/tree/master/v6/003https://github.com/elm-city-craftworks/practicing-ruby-examples/blob/master/v6/003/mutex_uncoordinated/dining_philosophers.rbhttps://github.com/elm-city-craftworks/practicing-ruby-examples/blob/master/v6/003/mutex_coordinated/dining_philosophers.rbhttps://github.com/elm-city-craftworks/practicing-ruby-examples/blob/master/v6/003/celluloid/dining_philosophers.rbhttps://github.com/elm-city-craftworks/practicing-ruby-examples/blob/master/v6/003/actors_from_scratch/dining_philosophers.rbhttps://github.com/elm-city-craftworks/practicing-ruby-examples/blob/master/v6/003/lib/actors.rb


30/30

hopsticks class definition

able class definition
https://github.com/elm-city-craftworks/practicing-ruby-examples/blob/master/v6/003/lib/chopstick.rbhttps://github.com/elm-city-craftworks/practicing-ruby-examples/blob/master/v6/003/lib/table.rbhttps://github.com/elm-city-craftworks/practicing-ruby-examples/blob/master/v6/003/lib/chopstick.rbhttps://github.com/elm-city-craftworks/practicing-ruby-examples/blob/master/v6/003/lib/table.rb

an introduction to celluloid

Documents