doctrine in the real world

Doctrine in the Real WorldReal world examples

doctrine

Tuesday, February 8, 2011

My name isJonathan H. Wage


• PHP Developer for 10+ years

• Long time Symfony and Doctrine contributor

• Published Author

• Entrepreneur

• Currently living in Nashville, Tennessee


I work full-time for OpenSky

http://shopopensky.com




Previously employed by SensioLabs


What is OpenSky?


A new way to shop

• OpenSky connects you with innovators, trendsetters and tastemakers. You choose the ones you like and each week they invite you to their private online sales.


We Love OpenSource• PHP 5.3

• Apache2

• Symfony2

• Doctrine2

• jQuery

• mule, stomp, hornetq

• MongoDB

• nginx

• varnish


We don’t just use open source projects


We help build them


OpenSky has some of the top committers in Symfony2 and other

projects


Symfony2 OpenSky Committers

• 65 Kris Wallsmith

• 52 Jonathan H. Wage

• 36 Jeremy Mikola

• 36 Bulat Shakirzyanov

• 6 Justin Hileman


Doctrine MongoDB Committers





MongoDB ODM Committers




• 13 Steven Surowiec

• 2 Jeremy Mikola


Sorry to bore you


Moving on to the stuff you came here for


OpenSky uses Doctrine ORM and ODM


Why?


We are an eCommerce site


Actions involving commerce need

transactions


ORM and MySQL

• Order

• Order\Transaction

• Order\Shipment


ODM and MongoDB

• Product

• Seller

• Supplier

• User

• ... basically everything else that is not involving $$$ and transactions


Blending the Two


Defining our Product Document


/** @mongodb:Document(collection="products") */class Product{ /** @mongodb:Id */ private $id;

/** @mongodb:String */ private $title;

public function getId() { return $this->id; }

public function getTitle() { return $this->title; }

public function setTitle($title) { $this->title = $title; }}


Defining our Order Entity


/** * @orm:Entity * @orm:Table(name="orders") * @orm:HasLifecycleCallbacks */class Order{ /** * @orm:Id @orm:Column(type="integer") * @orm:GeneratedValue(strategy="AUTO") */ private $id;

/** * @orm:Column(type="string") */ private $productId;

/** * @var Documents\Product */ private $product; // ...}


Setting the Product

public function setProduct(Product $product){ $this->productId = $product->getId(); $this->product = $product;}


• $productId is mapped and persisted

• but $product which stores the Product instance is not a persistent entity property


Order has a reference to product?

• How?

• Order is an ORM entity stored in MySQL

• and Product is an ODM document stored in MongoDB


Loading Product ODM reference in Order

Entity


Lifecycle Events to the Rescue


EventManager

• Event system is controlled by the EventManager

• Central point of event listener system

• Listeners are registered on the manager

• Events are dispatched through the manager


$eventListener = new OrderPostLoadListener($dm);$eventManager = $em->getEventManager();$eventManager->addEventListener( array(\Doctrine\ORM\Events::postLoad), $eventListener);

Add EventListener


In Symfony2 DI

<?xml version="1.0" encoding="utf-8" ?><container xmlns="http://www.symfony-project.org/schema/dic/services" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.symfony-project.org/schema/dic/services http://www.symfony-project.org/schema/dic/services/services-1.0.xsd">

<parameters> <parameter key="order.post_load.listener.class">OrderPostLoadListener</parameter> </parameters>

<services> <service id="order.post_load.listener" class="%order.post_load.listener.class%" scope="container"> <argument type="service" id="doctrine.odm.mongodb.default_document_manager" /> <tag name="doctrine.orm.default_event_listener" event="postLoad" /> </service> </services></container>


http://www.symfony-project.org/schema/dic/services


http://www.w3.org/2001/XMLSchema-instance

http://www.w3.org/2001/XMLSchema-instance



http://www.symfony-project.org/schema/dic/services/services-1.0.xsd




OrderPostLoadListeneruse Doctrine\ODM\MongoDB\DocumentManager;use Doctrine\ORM\Event\LifecycleEventArgs;

class OrderPostLoadListener{ public function __construct(DocumentManager $dm) { $this->dm = $dm; }

public function postLoad(LifecycleEventArgs $eventArgs) { // get the order entity $order = $eventArgs->getEntity();

// get odm reference to order.product_id $productId = $order->getProductId(); $product = $this->dm->getReference('MyBundle:Document\Product', $productId);

// set the product on the order $em = $eventArgs->getEntityManager(); $productReflProp = $em->getClassMetadata('MyBundle:Entity\Order') ->reflClass->getProperty('product'); $productReflProp->setAccessible(true); $productReflProp->setValue($order, $product); }}


All Together Now// Create a new product and order$product = new Product();$product->setTitle('Test Product');$dm->persist($product);$dm->flush();

$order = new Order();$order->setProduct($product);$em->persist($order);$em->flush();

// Find the order later$order = $em->find('Order', $order->getId());

// Instance of an uninitialized product proxy$product = $order->getProduct();

// Initializes proxy and queries the monogodb databaseecho "Order Title: " . $product->getTitle();print_r($order);


Seamless

• Documents and Entities play together like best friends

• Because Doctrine persistence remains transparent from your domain this is possible


Order Object( [id:Entities\Order:private] => 53 [productId:Entities\Order:private] => 4c74a1868ead0ed7a9000000 [product:Entities\Order:private] => Proxies\DocumentProductProxy Object ( [__isInitialized__] => 1 [id:Documents\Product:private] => 4c74a1868ead0ed7a9000000 [title:Documents\Product:private] => Test Product )

)

print_r($order)


Example from Blog

• This example was first written on my personal blog http://jwage.com

• You can read the blog post here http://jwage.com/2010/08/25/blending-the-doctrine-orm-and-mongodb-odm/


http://jwage.com

http://jwage.com

http://jwage.com/2010/08/25/blending-the-doctrine-orm-and-mongodb-odm/






MongoDB ODM SoftDelete Functionality


I like my deletes soft, not hard


Why?


Deleting data is dangerous business


Flickr accidentally deleted a pro members

account and 4000 pictures


They were able to restore it later but it

took some time


Instead of deleting, simply set a deletedAt field


Install SoftDelete Extension for Doctrine

MongoDB ODM

$ git clone git://github.com/doctrine/mongodb-odm-softdelete src/vendor/doctrine-mongodb-odm-softdelete

http://github.com/doctrine/mongodb-odm-softdelete


https://github.com/doctrine/mongodb-odm-softdelete-bundle


Autoload Extension

$loader = new UniversalClassLoader();$loader->registerNamespaces(array( // ... 'Doctrine\\ODM\\MongoDB\\SoftDelete' => __DIR__.'/vendor/doctrine-mongodb-odm-softdelete/lib',));$loader->register();


use Doctrine\ODM\MongoDB\SoftDelete\Configuration;

use Doctrine\ODM\MongoDB\SoftDelete\UnitOfWork;use Doctrine\ODM\MongoDB\SoftDelete\SoftDeleteManager;use Doctrine\Common\EventManager;

// $dm is a DocumentManager instance we should already have

$config = new Configuration();$uow = new UnitOfWork($dm, $config);$evm = new EventManager();$sdm = new SoftDeleteManager($dm, $config, $uow, $evm);

Raw PHP Configuration


Symfony2 Integrationhttp://github.com/doctrine/mongodb-odm-softdelete-bundle

$ git clone git://github.com/doctrine/mongodb-odm-softdelete-bundle.git src/vendor/doctrine-mongodb-odm-softdelete-bundle




Autoload the Bundle

$loader = new UniversalClassLoader();$loader->registerNamespaces(array( // ... 'Doctrine\\ODM\\MongoDB\\Symfony\\SoftDeleteBundle' => __DIR__.'/vendor/doctrine-mongodb-odm-softdelete-bundle',));$loader->register();


Register the Bundle

public function registerBundles(){ $bundles = array( // ...

// register doctrine symfony bundles new Doctrine\ODM\MongoDB\Symfony\SoftDeleteBundle\SoftDeleteBundle() );

// ...

return $bundles;}


Enable the Bundle

// app/config/config.yml

doctrine_mongodb_softdelete.config: ~


SoftDeleteManager

$sdm = $container->get('doctrine.odm.mongodb.soft_delete.manager');


SoftDeleteable

interface SoftDeleteable{ function getDeletedAt(); function isDeleted();}

ODM Documents must implement this interface


User implements SoftDeletable

/** @mongodb:Document */class User implements SoftDeleteable{ /** @mongodb:Date @mongodb:Index */ private $deletedAt;

public function getDeletedAt() { return $this->deletedAt; }

public function isDeleted() { return $this->deletedAt !== null ? true : false; }}


SoftDelete a User$user = new User('jwage');// ...$dm->persist($user);$dm->flush();

// later we can soft delete the user jwage$user = $dm->getRepository('User')->findOneByUsername('jwage');$sdm->delete($user);$sdm->flush();


Query Executed

db.users.update( { _id : { $in : [new ObjectId('1234567891011123456')] } }, { $set : { deletedAt: new Date() } })


Restore a User

// now again later we can restore that same user$user = $dm->getRepository('User')->findOneByUsername('jwage');$sdm->restore($user);$sdm->flush();


Query Executed

db.users.update( { _id : { $in : [new ObjectId('1234567891011123456')] } }, { $unset : { deletedAt: true } })


Limit cursors to only show non deleted users

$qb = $dm->createQueryBuilder('User') ->field('deletedAt')->exists(false);$query = $qb->getQuery();$users = $query->execute();


Get only deleted users

$qb = $dm->createQueryBuilder('User') ->field('deletedAt')->exists(true);$query = $qb->getQuery();$users = $query->execute();


Restore several deleted users

$qb = $dm->createQueryBuilder('User') ->field('deletedAt')->exists(true) ->field('createdAt')->gt(new DateTime('-24 hours'));$query = $qb->getQuery();$users = $query->execute();

foreach ($users as $user) { $sdm->restore($user);}$sdm->flush();


Soft Delete Events

- preDelete- postDelete- preRestore- postRestore

class TestEventSubscriber implements \Doctrine\Common\EventSubscriber{ public function preSoftDelete(LifecycleEventArgs $args) { $document = $args->getDocument(); $sdm = $args->getSoftDeleteManager(); }

public function getSubscribedEvents() { return array(Events::preSoftDelete); }}

$eventSubscriber = new TestEventSubscriber();$evm->addEventSubscriber($eventSubscriber);


Symfony2 and supervisor

http://supervisord.org/




What is supervisor?


Supervisor is a client/server system that allows its users to monitor and control a number of processes on UNIX-like operating systems.

http://supervisord.org




Daemonize a Symfony2 Console Command

with supervisor


Scenario

• You want to send an e-mail when new users register in your system.

• But, sending an e-mail directly from your action introduces a failure point to your stack.

• ....What do you do?


Tailable Cursor

• Use a tailable mongodb cursor

• Tail a NewUser document collection

• Insert NewUser documents from your actions

• The daemon will instantly process the NewUser after it is inserted and dispatch the e-mail


Define NewUsernamespace MyCompany\Bundle\MyBundle\Document;

/** * @mongodb:Document(collection={ * "name"="new_users", * "capped"="true", * "size"="100000", * "max"="1000" * }, repositoryClass="MyCompany\Bundle\MyBundle\Document\NewUserRepository") */class NewUser{ /** @mongodb:Id */ private $id;

/** @mongodb:ReferenceOne(targetDocument="User") */ private $user;

/** @mongodb:Boolean @mongodb:Index */ private $isProcessed = false;

// ...}


Create Collection• The NewUser collection must be capped in

order to tail it so we need to create it.

• Luckily, Doctrine has a console command for it.

• It will read the mapping information we configured and create the collection

$ php app/console doctrine:mongodb:schema:create --class="MyBundle:NewUser" --collection


Insert NewUser upon Registration

public function register(){ // ...

$user = new User(); $form = new RegisterForm('register', $user, $validator);

$form->bind($request, $user); if ($form->isValid()) { $newUser = new NewUser($user); $dm->persist($newUser); $dm->persist($user); $dm->flush();

// ... } // ...}


The Daemon Console Command

• You can find the console command code to use to tail a cursor here:

• https://gist.github.com/812942


https://gist.github.com/812942

https://gist.github.com/812942

Executing Console Command

$ php app/console doctrine:mongodb:tail-cursor MyBundle:NewUser findUnProcessed new_user.processor

• The command requires 3 arguments:

• document - the name of the document to tail

• finder - the repository finder method used to get the cursor

• processor - the id of the service used to process the new users


findUnProcessed()

• We need the findUnProcessed() method to return the unprocessed cursor to tail

class NewUserRepository extends DocumentRepository{ public function findUnProcessed() { return $this->createQueryBuilder() ->field('isProcessed')->equals(false) ->getQuery() ->execute(); }}


NewUserProcessor

use Swift_Message;use Symfony\Component\Console\Output\OutputInterface;

class NewUserProcessor{ private $mailer;

public function __construct($mailer) { $this->mailer = $mailer; }

public function process(OutputInterface $output, $document) { }}

We need a service id new_user.processor with a process(OutputInterface $output, $document) method


Send the e-mailpublic function process(OutputInterface $output, $document){ $user = $document->getUser();

$message = Swift_Message::newInstance() ->setSubject('New Registration') ->setFrom('[email protected]') ->setTo($user->getEmail()) ->setBody('New user registration') ; $this->mailer->send($message);

$document->setIsProcessed(true);}


mailto:[email protected]

mailto:[email protected]

Daemonization

• Now, how do we really daemonize the console command and keep it running 24 hours a day, 7 days a week?

• The answer is supervisor, it will allow us to configure a console command for it to manage the process id of and always keep an instance of it running.


Install supervisorhttp://supervisord.org/installing.html

$ easy_install supervisor


http://supervisord.org/installing.html

http://supervisord.org/installing.html

Configure a Profile• We need to configure a profile for supervisor to

know how to run the console command

[program:tail-new-user]numprocs=1

startretries=100directory=/

stdout_logfile=/path/to/symfonyproject/app/tail-new-user-supervisord.logautostart=true

autorestart=trueuser=root

command=/usr/local/bin/php /path/to/symfonyproject/app/console doctrine:mongodb:tail-cursor MyBundle:NewUser

findUnprocessed new_user.processor

[program:tail-new-user]numprocs=1

startretries=100directory=/

stdout_logfile=/path/to/symfonyproject/app/tail-new-user-supervisord.logautostart=true

autorestart=trueuser=root

command=/usr/local/bin/php /path/to/symfonyproject/app/console doctrine:mongodb:tail-cursor MyBundle:NewUser

findUnprocessed new_user.processor[program:tail-new-user]numprocs=1startretries=100directory=/stdout_logfile=/path/to/symfonyproject/app/logs/tail-new-user-supervisord.logautostart=trueautorestart=trueuser=rootcommand=/usr/local/bin/php /path/to/symfonyproject/app/console doctrine:mongodb:tail-cursor MyBundle:NewUser findUnprocessed new_user.processor

$ vi /etc/supervisor/conf.d/tail-new-user.conf


Start supervisord

• Start an instance of supervisord

• It will run as a daemon in the background

• The tail-new-user.conf will always be running

$ supervisord


Where do I use supervisor?

• sociallynotable.com

• Indexes tweets with links to Amazon.com products

• Maintains statistics on each product and lets you shop the popular products each day


sn:watch-twitter console command


I can manually start it


But, what if it crashes or stops unexpectedly?


This is exactly what supervisor is for


Setup a configuration profile for supervisorand it will ensure the console command is

always running


[program:watch-twitter]numprocs=1startretries=100000000000directory=/stdout_logfile=/var/www/vhosts/sociallynotable.com/socially-notable/sociallynotable/logs/watch-twitter-supervisord.logautostart=trueautorestart=trueuser=rootcommand=/usr/local/bin/php /var/www/vhosts/sociallynotable.com/socially-notable/sociallynotable/console sn:watch-twitter


Now when I start supervisor the twitter watcher will always

remain running. Even if I kill the pid myself, it will start back up.


Thanks!I hope this presentation was useful to you!


Questions?- http://jwage.com- http://shopopensky.com- http://sociallynotable.com- http://twitter.com/jwage- http://facebook.com/jwage- http://about.me/jwage


http://jwage.com

http://jwage.com



http://sociallynotable.com

http://sociallynotable.com

http://twitter.com/jwage

http://twitter.com/jwage

http://facebook.com/jwage

http://facebook.com/jwage

http://about.me/jwage

http://about.me/jwage