YAPC::Asia 2009 Tokyo

2009/9/10

Gosuke Miyashita

Speaker

Gosuke Miyashita （ mizzy ） Technical manager at paperboy&co. Have most kids in Japan Perl world?

3 kids.Even with Yappo.4th kid will be born in November.

Agenda Asynchronous processing with event

driven programming Asynchronous processing with

Danga::Socket Overview of Perlbal asynchronous

processing How to write asynchronous Perlbal

plugins Summary

Event driven programming A programming paradigm in which the

flow of the program is determined by events

Counter paradigm of flow-based programming

Events

I/OOn I/O read ready or write ready

TimersOn time passed

SignalOn getting a signal

Child processOn child process exit

Main loop

Also called “event loop” On each loop, check that events are

occurred or not, and process the events. After process the events, return to the

loop and repeat the loop till next events occur

This is the flow of asynchronous processing

Eaxmple of main loop

Process timers

Wait I/O events

Process I/O events

Post main loop process

Events supported by Danga::Socket

I/O Timers

Process I/O events

Danga::Socket supports 3 I/O event notification facilitykqueueepollpoll

Select adequate facility automatically

Add I/O watcher with Danga::Socketuse base ‘Danga::Socket’;

sub new { # pass file descripter $self->SUPER::new($fd); }

sub event_read { # process when $fd is read ready}

sub event_write { # procdess when $fd is write ready}

Add I/O wathcer (pattern 2)

Danga::Socket->AddOtherFds( $fd => sub { # process when $fd is read ready

},);

Add I/O watcher (pattern 3)

Danga::Socket::Callback->new( handle => $fd, on_read_ready => sub { # process on read ready }, on_write_ready => sub { # process on write ready },);

Add timer watcher with Danga::Socket

Danga::Socket->AddTimer( 10, sub { # process after 10 seconds

},}

Start main loop with Danga::Socket

Danga::Socket->EventLoop();

Main loop(again)

Process Timers

Wait I/O events

Process I/O events

Post main loop process

Perlbal mechanism(as reverse proxy)

BackendHTTP

ClientProxy

TCPListenerClient

Server

ClientProxy

Client

BackendHTTP

Danga::Socket based objects

Perlbal with a plugin (on start_proxy_request hook)

BackendHTTP

ClientProxy

TCPListenerClient

Server

Plugin::Hoge

Target of this session

Hooks

Perlbal has many plugin hooks Explain with start_proxy_request hook in

this session

In case of a synchronous plugin

ClientProxy

TCPListenerClient

Server

Plugin::SyncClient

In case of an asynchronous plugin

ClientProxy

TCPListenerClient

Server

Plugin::AsyncClient

ClientProxy

How to write asynchronous plugins? Write the main process based on

Danga::Socketprocess within the main loop of

Danga::SocketCreate Danga::Socket based classOr use Danga::Socket::CallbackOr use Danga::Socket::AddTimerAlso other libraries need to be non-blocking

○ You can use Gearman::Client::Async with blocking processes

How to write asynchronous plugins? When finish asynchronous process, go

to next phaseCall back functionAlso need to fix Perlbal itself

Main process of a pluginpackage My::Drizzle;

use base ‘Danga::Socket’;use Net::Drizzle ':constants';

sub new { $self->SUPER::new($fh); $self->watch_read(1);}

sub event_read { # Check db request status and call back

when finished}

Attention You can also use

Danga::Socket::Callback You cannot use AddOtherFds()

File descriptors added by Add AddOtherFds() are only evaluated at the beginning of the main loop

So if already in the main loop, AddOtherFds() are meaningless.

You can call epoll_ctl, EV_SET and so on directly, but no portability

plugin registerpackage Perlbal::Plugin::AsyncDb;

sub register { my ( $class, $svc ) = @_;

$svc->register_hook( 'Async' => 'start_proxy_request', \&request_db, );

return 1;}

Call asynchronous processsub request_db { my Perlbal::ClientProxy $client =

shift;

My::Drizzle->new( callback => sub { # call back }, );

return 1; # very important!}

In case of “return 0”

BackendHTTP

ClientProxy

TCPListenerClient

Server

Plugin::Async

Go to next step even if Plugin::Async is not

finished

BackendHTTP

“return 1” & “call back”

ClientProxy

TCPListenerClient

Plugin::Async

Stop the process and go to main loop call back

Server

return 1

Call back function If plugin process finished, ClientProxy must

go to the next process(call BackendHTTP) Stopped at the following code of

handle_request() :

return if $svc->run_hook( 'start_proxy_request', $self ); ClientProxy must start processing from the

next of this code

Call back function

my Perlbal::ClientProxy $client = shift;

My::Drizzle->new( callback => sub { $client->{async_complete} = 1;

$client->handle_request; },);

Patch of ClientProxy::handle_request- return if $svc->run_hook(- 'start_proxy_request', $self- );+ unless ( $self->{async_complete} ) {

+ return if $svc->run_hook(+ 'start_proxy_request', $self

+ );+ }+ $self->{async_complete} = 0;

Points of asynchronous Perlbal plugins Process within the Danga::Socket main

loop return 1 when plugin called Restart ClientProxy process by a call

back function when plugin finished Also need to fix Perlbal itself Be careful with order of plugins

Following plugins on a same hook are ignored when after a plugin returns 1