scaling symfony2 apps with rabbitmq - symfony uk meetup
DESCRIPTION
Slides from my talk at Symfony UK Meetup. London, 20 Aug 2014. http://twitter.com/cakper Video: https://www.youtube.com/watch?v=cha92Og9M5ATRANSCRIPT
Scaling Symfony2 apps
with RabbitMQ
Kacper Gunia @cakper Software Engineer @SensioLabsUK
Symfony Certified Developer
PHPers Silesia @PHPersPL
Agenda
❖What is RabbitMQ!❖How to communicate with it from PHP!❖How to integrate it with Symfony!❖Diving into details!❖Demo
Why would I need Messaging?
Use Cases
❖ Offloading / Background jobs!❖ Integration !❖ Scaling!❖ Queueing!❖ Scheduling!❖ Events
Message-oriented Middleware
“(…) allows distributed applications !to communicate and exchange data !by sending and receiving messages”
http://docs.oracle.com/cd/E19316-01/820-6424/aeraq/index.html
What is inside a Rabbit?
http://madamtruffle.deviantart.com/art/What-‐is-‐inside-‐a-‐rabbit-‐81036248
What is inside the RabbitMQ?
Message Broker
AMQP
Erlang
Interoperability
❖ PHP!❖ Clojure!❖ Erlang!❖ Java!❖ Perl!❖ Python!❖ Ruby
❖ C#!❖ JavaScript!❖ C/C++!❖ Go!❖ Lisp!❖ Haskell!❖ …
How does it work?
Broker
Bindings
Producer Exchange
Queue
Queue
Consumer
Consumer
Consumer
Consumer
Producer
Producer
Exchange
Exchange
Direct Exchange
Exchange Queue
Fanout Exchange
Exchange
Queue
Queue
Topic Exchange
#.error
#.warning, log.*
*.mobile.*
log.error
log.sql.warning
log.mobile.error
“.” - word separator
“#” - prefix/suffix wildcard
“*” - word wildcard
Bindings
Bindings
Exchange
Queue
Queue
Queue
Queue
Consumer
Consumer
How to feed Rabbit from PHP?
PHP libraries and tools
❖ php-amqplib!❖ PECL AMQP !❖ amqphp!❖ VorpalBunny!❖ Thumper!❖ CAMQP
Declare queue$connection = new AMQPConnection ('localhost', 5672, 'guest', 'guest'); $channel = $connection-‐>channel(); !
$channel-‐>queue_declare ('hello', false, false, false, false); !
/** Your stuff */ !
$channel-‐>close(); $connection-‐>close();
Producer
$msg = new AMQPMessage('Hello World!'); $channel-‐>basic_publish($msg, '', 'hello'); !
echo " [x] Sent 'Hello World!'\n";
Consumer$callback = function($msg) { echo ' [x] Received ', $msg-‐>body, "\n"; }; !
$channel-‐>basic_consume ('hello', '', false, true, false, false, $callback); !
while(count($channel-‐>callbacks)) { $channel-‐>wait(); }
RabbitMQ & Symfonyhttps://secure.flickr.com/photos/8725928@N02/9657136424
Step 1
Install RabbitMQ Bundle
composer require “oldsound/rabbitmq-‐bundle 1.5.*”
Step 2
Enable bundle in Kernel
public function registerBundles() { $bundles =[ (…) new OldSound\RabbitMqBundle\OldSoundRabbitMqBundle() ]; }
Step 3
Say thanks to @old_sound :)
Step 4
Get rid off manual configuration
old_sound_rabbit_mq: connections: default: host: 'localhost' port: 5672 user: 'guest' password: 'guest' vhost: '/' lazy: false
Get rid off manual configuration
producers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} consumers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} queue_options: {name: 'hello'} callback: hello_world_service
Get rid off manual configuration
producers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} consumers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} queue_options: {name: 'hello'} callback: hello_world_service
Step 5
Produce some data
public function indexAction($name) { $this -‐>get('old_sound_rabbit_mq.hello_world_producer') -‐>publish($name); }
Step 6
Implement consumer
class HelloWorldConsumer implements ConsumerInterface { public function execute(AMQPMessage $msg) { echo "Hello $msg-‐>body!".PHP_EOL; } }
Step 7
Configure Service Container
services: hello_world_service: class: Cakper\HelloWorldConsumer
Step 8
Run the Consumer
./app/console rabbitmq:consumer hello_world
Nailed it!
Diving intohttps://secure.flickr.com/photos/toms/159393358
Producers
Custom producer class
producers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} class: Cakper\HelloProducer
Custom producer class
class HelloProducer extends Producer { public function publish($msgBody, …) { $msgBody = serialize($msgBody); !
parent::publish($msgBody, …); } }
Set content type
function __construct() { $this-‐>setContentType('application/json'); } !
public function publish($msgBody, …) { parent::publish(json_encode($msgBody), …); }
Consumers
Re-queue message
public function execute(AMQPMessage $msg) { if ('cakper' === $msg-‐>body) { return false; } !
echo "Hello $msg-‐>body!".PHP_EOL; }
Idle timeouts
consumers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} queue_options: {name: 'hello'} callback: hello_world_service idle_timeout: 180
Limit number of messages
./app/console rabbitmq:consumer hello_world -‐m 10
Quality of Service
qos_options: prefetch_size: 0 prefetch_count: 0..65535 global: false/true
Exchanges
Exchange options
exchange_options: name: ~ type: direct/fanout/topic durable: true/false
Queues
Queue options
queue_options: name: ~ durable: true/false arguments: 'x-‐message-‐ttl': ['I', 20000] routing_keys: -‐ 'logs.sql.#' -‐ '*.error'
Purge queue
./app/console rabbitmq:purge -‐-‐no-‐confirmation hello
Connection
Make it lazy!
old_sound_rabbit_mq: connections: default: host: 'localhost' port: 5672 user: 'guest' password: 'guest' vhost: '/' lazy: true
Setup
Setup Fabric
./app/console rabbitmq:setup-‐fabric !
producers: upload_picture: auto_setup_fabric: false consumers: upload_picture: auto_setup_fabric: false
Multiple consumers
Multiple consumersmultiple_consumers: hello: connection: default exchange_options: {name: 'hello', type: direct} queues: hello-‐vip: name: hello_vip callback: hello_vip_world_service routing_keys: -‐ vip hello-‐regular: name: hello_regular callback: hello_regular_world_service
Anonymous Consumers
Anonymous Consumer
producers: hello: connection: default exchange_options: {name: 'hello', type: topic}
Anonymous Consumer
anon_consumers: hello: connection: default exchange_options: {name: 'hello', type: topic} callback: hello_world_service
Anonymous Consumer
./app/console_dev rabbitmq:anon-‐consumer -‐r '#.vip' hello
Management console
Demo time
Summary
❖ RabbitMQ is fast!❖ and reliable!❖ also language agnostic!❖ and easy to install and use!❖ gives you flexibility!❖ supports high-availability, clustering
Kacper Gunia Software Engineer
Symfony Certified Developer
PHPers Silesia
Thanks!