scaling symfony2 apps with rabbitmq - symfony uk meetup

Scaling Symfony2 apps

with RabbitMQ

Kacper Gunia @cakper Software Engineer @SensioLabsUK

Symfony Certified Developer

PHPers Silesia @PHPersPL

Agenda

❖What is RabbitMQ!❖How to communicate with it from PHP!❖How to integrate it with Symfony!❖Diving into details!❖Demo

Why would I need Messaging?

Use Cases

❖ Offloading / Background jobs!❖ Integration !❖ Scaling!❖ Queueing!❖ Scheduling!❖ Events

Message-oriented Middleware

“(…) allows distributed applications !to communicate and exchange data !by sending and receiving messages”

http://docs.oracle.com/cd/E19316-01/820-6424/aeraq/index.html

What is inside a Rabbit?

http://madamtruffle.deviantart.com/art/What-­‐is-­‐inside-­‐a-­‐rabbit-­‐81036248

What is inside the RabbitMQ?

Message Broker

AMQP

Erlang

Interoperability

❖ PHP!❖ Clojure!❖ Erlang!❖ Java!❖ Perl!❖ Python!❖ Ruby

❖ C#!❖ JavaScript!❖ C/C++!❖ Go!❖ Lisp!❖ Haskell!❖ …

How does it work?

Broker

Bindings

Producer Exchange

Queue

Queue

Consumer

Consumer

Consumer

Consumer

Producer

Producer

Exchange

Exchange

Direct Exchange

Exchange Queue

Fanout Exchange

Exchange

Queue

Queue

Topic Exchange

#.error

#.warning, log.*

*.mobile.*

log.error

log.sql.warning

log.mobile.error

“.” - word separator

“#” - prefix/suffix wildcard

“*” - word wildcard

Bindings

Bindings

Exchange

Queue

Queue

Queue

Queue

Consumer

Consumer

How to feed Rabbit from PHP?

PHP libraries and tools

❖ php-amqplib!❖ PECL AMQP !❖ amqphp!❖ VorpalBunny!❖ Thumper!❖ CAMQP

Declare queue$connection  =  new  AMQPConnection                    ('localhost',  5672,  'guest',  'guest');  $channel  =  $connection-­‐>channel();  !

$channel-­‐>queue_declare                    ('hello',  false,  false,  false,  false);  !

/**  Your  stuff  */  !

$channel-­‐>close();  $connection-­‐>close();

Producer

$msg  =  new  AMQPMessage('Hello  World!');  $channel-­‐>basic_publish($msg,  '',  'hello');  !

echo  "  [x]  Sent  'Hello  World!'\n";

Consumer$callback  =  function($msg)  {          echo  '  [x]  Received  ',  $msg-­‐>body,  "\n";  };  !

$channel-­‐>basic_consume    ('hello',  '',  false,  true,  false,  false,  $callback);  !

while(count($channel-­‐>callbacks))  {            $channel-­‐>wait();    }

RabbitMQ & Symfonyhttps://secure.flickr.com/photos/8725928@N02/9657136424

Step 1

Install RabbitMQ Bundle

composer  require  “oldsound/rabbitmq-­‐bundle  1.5.*”

Step 2

Enable bundle in Kernel

public  function  registerBundles()  {        $bundles  =[            (…)              new  OldSound\RabbitMqBundle\OldSoundRabbitMqBundle()        ];  }

Step 3

Say thanks to @old_sound :)

Step 4

Get rid off manual configuration

old_sound_rabbit_mq:          connections:                  default:                          host:          'localhost'                          port:          5672                          user:          'guest'                          password:  'guest'                          vhost:        '/'                          lazy:          false

Get rid off manual configuration

producers:          hello_world:                  connection:              default                  exchange_options:  {name:  'hello',  type:  direct}  consumers:          hello_world:                  connection:              default                  exchange_options:  {name:  'hello',  type:  direct}                  queue_options:        {name:  'hello'}                  callback:                  hello_world_service

Get rid off manual configuration

producers:          hello_world:                  connection:              default                  exchange_options:  {name:  'hello',  type:  direct}  consumers:          hello_world:                  connection:              default                  exchange_options:  {name:  'hello',  type:  direct}                  queue_options:        {name:  'hello'}                  callback:                  hello_world_service

Step 5

Produce some data

public  function  indexAction($name)  {          $this          -­‐>get('old_sound_rabbit_mq.hello_world_producer')          -­‐>publish($name);  }

Step 6

Implement consumer

class  HelloWorldConsumer  implements  ConsumerInterface  {          public  function  execute(AMQPMessage  $msg)          {                  echo  "Hello  $msg-­‐>body!".PHP_EOL;          }  }

Step 7

Configure Service Container

services:          hello_world_service:                  class:  Cakper\HelloWorldConsumer

Step 8

Run the Consumer

./app/console  rabbitmq:consumer  hello_world

Nailed it!

Diving intohttps://secure.flickr.com/photos/toms/159393358

Producers

Custom producer class

producers:          hello_world:                  connection:              default                  exchange_options:  {name:  'hello',  type:  direct}                  class:                        Cakper\HelloProducer

Custom producer class

class  HelloProducer  extends  Producer  {          public  function  publish($msgBody,  …)          {                  $msgBody  =  serialize($msgBody);  !

               parent::publish($msgBody,  …);          }  }  

Set content type

function  __construct()  {          $this-­‐>setContentType('application/json');  }  !

public  function  publish($msgBody,  …)  {          parent::publish(json_encode($msgBody),  …);  }

Consumers

Re-queue message

public  function  execute(AMQPMessage  $msg)  {          if  ('cakper'  ===  $msg-­‐>body)  {                  return  false;          }  !

       echo  "Hello  $msg-­‐>body!".PHP_EOL;  }

Idle timeouts

consumers:          hello_world:                  connection:              default                  exchange_options:  {name:  'hello',  type:  direct}                  queue_options:        {name:  'hello'}                  callback:                  hello_world_service                  idle_timeout:          180

Limit number of messages

./app/console  rabbitmq:consumer  hello_world  -­‐m  10

Quality of Service

qos_options:          prefetch_size:      0          prefetch_count:    0..65535          global:                    false/true

Exchanges

Exchange options

exchange_options:          name:                                  ~          type:                                  direct/fanout/topic          durable:                            true/false

Queues

Queue options

queue_options:          name:                                  ~          durable:                            true/false          arguments:                  'x-­‐message-­‐ttl':  ['I',  20000]          routing_keys:              -­‐  'logs.sql.#'              -­‐  '*.error'  

Purge queue

./app/console  rabbitmq:purge  -­‐-­‐no-­‐confirmation  hello

Connection

Make it lazy!

old_sound_rabbit_mq:          connections:                  default:                          host:          'localhost'                          port:          5672                          user:          'guest'                          password:  'guest'                          vhost:        '/'                          lazy:          true

Setup

Setup Fabric

./app/console  rabbitmq:setup-­‐fabric  !

producers:          upload_picture:              auto_setup_fabric:  false  consumers:          upload_picture:              auto_setup_fabric:  false

Multiple consumers

Multiple consumersmultiple_consumers:          hello:                  connection:              default                  exchange_options:  {name:  'hello',  type:  direct}                  queues:                          hello-­‐vip:                                  name:          hello_vip                                  callback:  hello_vip_world_service                                  routing_keys:                                          -­‐  vip                          hello-­‐regular:                                  name:          hello_regular                                  callback:  hello_regular_world_service

Anonymous Consumers

Anonymous Consumer

producers:          hello:                  connection:              default                  exchange_options:  {name:  'hello',  type:  topic}

Anonymous Consumer

anon_consumers:          hello:                  connection:              default                  exchange_options:  {name:  'hello',  type:  topic}                  callback:                  hello_world_service

Anonymous Consumer

./app/console_dev  rabbitmq:anon-­‐consumer  -­‐r  '#.vip'  hello

Management console

Demo time

Summary

❖ RabbitMQ is fast!❖ and reliable!❖ also language agnostic!❖ and easy to install and use!❖ gives you flexibility!❖ supports high-availability, clustering

Kacper Gunia Software Engineer

Symfony Certified Developer

PHPers Silesia

Thanks!