using document databases with typo3 flow
TRANSCRIPT
Inspiring people to
shareInspiring people to
share
Using Document Databases with
TYPO3 Flow
Karsten Dambekalns
TYPO3 Neos and Flow developer
35 years old
lives in Lübeck, Germany
1 wife, 3 sons
1 espresso machine
likes canoeing & climbing
Persistence Basicsin Flow
Flow uses Doctrine 2 ORM
MySQL PostgreSQL SQLite Oracle
Native and PDO DB drivers
Doctrine DBAL
Doctrine ORM
TYPO3 Flow
Flow provides a centralized PersistenceManager
Native and PDO DB drivers
Doctrine DBAL
Doctrine ORM
Doctrine\PersistenceManager
Repository Doctrine\Repository
Repositories encapsulate persistence concerns
Doctrine\PersistenceManager
Repository Doctrine\Repository
Your Application Code
Models do not know about persistence
Models do not know (a lot)
about persistence(internals)
Properties
/** * @Flow\Entity */class Organisation {
/** * The name * @var string * @Flow\Validate(type="NotEmpty") * @Flow\Validate(type="StringLength", options={"maximum"=40}) * @ORM\Column(length=40) */ protected $name;
Relations
/** * @Flow\Entity */class Organisation {
/** * @var \Doctrine\Common\Collections\Collection <\Acme\Demo\Domain\Model\Alias> * @ORM\OneToMany(mappedBy="organisation",cascade={"persist"}) * @ORM\OrderBy({"name" = "ASC"}) * @Flow\Lazy */ protected $aliases;
Document Databases
Provide some powerful advantages
• schema-less storage
• usually scale well
• really good for non-relational data
No more Object-Relational Mapping, now you have Object-Document Mapping
Queries, but not as you know them
•Most document databases do not allow queries on arbitary properties
Which Database to Choose
Many different document databases exist
Which one is right for your project depends on many factors
•MongoDB: For most things that you would do with MySQL or PostgreSQL, but having predefined columns really holds you back.
• CouchDB: For accumulating, occasionally changing data, on which pre-defined queries are to be run. Places where versioning is important.
A nice overview with much more can be found onhttp://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis
CouchDB
Is one of the more widespread products
Stores documents in JSON
Provides a REST interface
Provides master-master replication and MVCC
Futon on CouchDB
CouchDB REST API
Create a database
curl -X PUT http://127.0.0.1:5984/demo
{"ok":true}
Create documents
curl -H 'Content-Type: application/json' \ -X POST http://127.0.0.1:5984/demo \ -d '{"company": "Example, Inc."}'
{"ok":true,"id":"8843faaf0b831d364278331bc3001bd8","rev":"1-33b9fbce46930280dab37d672bbc8bb9"}
CouchDB Basics
Fetch a document
curl -X GET http://127.0.0.1:5984/demo/8843faaf0b831d364278331bc3001bd8
{"_id":"8843faaf0b831d364278331bc3001bd8","_rev":"1-33b9fbce46930280dab37d672bbc8bb9","company":"Example, Inc."}
CouchDB Basics
Create a database using Futon or with
curl -X PUT http://127.0.0.1:5984/demo
Create documents using Futon or with
curl -H 'Content-Type: application/json' \ -X POST http://127.0.0.1:5984/demo \ -d '{"company": "Example, Inc."}'
Fetch documents using Futon
If you do manual work on the shell, try HTTPie instead of cURL
Read guide.couchdb.org and docs.couchdb.org
CouchDB Basics
Read guide.couchdb.org and docs.couchdb.org
Learn about views and map/reduce
Hint: A better cURL
If you do manual work on the shell, try HTTPie instead of cURL
TYPO3.CouchDB
Developed for Rossmann by networkteam
Fully replaces the ORM persistence
Model annotations stay the same
Provides basic QOM-to-View mapping
Note: Not working with TYPO3 Flow 2.0 right now
Installation
cd Packages/Applicationgit clone git://git.typo3.org/FLOW3/Packages/TYPO3.CouchDB.git
No composer package, so just clone it
Configuration
TYPO3: FLOW3: persistence: # Options for the CouchDB backend backendOptions: # database: 'database_name' dataSourceName: 'http://127.0.0.1:5984' username: ~ password: ~ enableCouchdbLucene: no driver: ~ path: ~
Usage – Repository
class TestEntityRepository extends Repository {
/** * @param string $name * @return \TYPO3\FLOW3\Persistence\QueryResultInterface */ public function findByNameLike($name) { $query = $this->testIndex->createQuery(); $query->matching( $query->like('name', $name) ); return $query->execute(); }}
Usage – LuceneIndex
class MyIndex extends \TYPO3\CouchDB\Domain\Index\LuceneIndex {
/** * Configure the index * * @return void */ public function configure() { $this->forEntity('Acme\Demo\Domain\Model\SomeEntity') ->indexProperty('name') ->indexProperty('relatedValueObject.color'); }
}
Usage – Views & Design Docs
Views can be defined by
• implementing TYPO3\CouchDB\ViewInterface
•configuring TYPO3\CouchDB\QueryView
Design documents can be defined by extending TYPO3\CouchDB\DesignDocument
Now for somebad news
Only CouchDB is supported
Kristina AlexandersonCC BY-NC-SA 2.0
One Backend at a Time
Doctrine DBAL
Doctrine ORM
Doctrine\PersistenceManager
Repository Doctrine\Repository
One Backend at a Time
Doctrine DBAL
Doctrine ORM
Doctrine\PersistenceManager
Repository Doctrine\Repository
You can work around this!
In your repositories you can basically do what you want
• Open a direct database connection
• Read CSV files
• Connect to a Solr server
• Instantiate another Doctrine EntityManager
Do something more advanced, let Radmiraal.CouchDB inspire you
As long as you encapsulate nicely!
And now somegood news
Radmiraal.CouchDB
Radmiraal.CouchDB
Developed for some projects having similar needs
Started out as an extension to TYPO3.CouchDB
Since then refactored & now based on Doctrine 2 CouchDB ODM
Can be used alongside the usual ORM
Radmiraal.CouchDB
Radmiraal.CouchDB
"repositories": [ { "type": "git", "url": "https://github.com/radmiraal/Radmiraal.CouchDB.git" }],
"require": { "radmiraal/couchdb": "dev-doctrine",}
Installation
TYPO3: Flow: persistence: backendOptions: … as usual
Radmiraal: CouchDB: persistence: backendOptions: databaseName: 'mycouchdb' username: 'mycouchuser' password: 'mycouchpassw0rd'
Configuration
Usage – Models
use Doctrine\ODM\CouchDB\Mapping\Annotations as ODM;
/** * @ODM\Document(indexed=true) */class DataSet {
/** * @var string * @ODM\Id(type="string") */ protected $id;
/** * @var \Acme\Demo\Domain\Model\Organisation * @ODM\Field(type="string") * @ODM\Index */ protected $organisation;
/** * @var \DateTime * @ODM\Field(type="datetime") * @ODM\Index */ protected $creationDateTime;
Usage – Models
/** * @var \Acme\Demo\Domain\Model\Organisation * @ODM\Field(type="string") * @ODM\Index */ protected $organisation;
/** * @var \DateTime * @ODM\Field(type="datetime") * @ODM\Index */ protected $creationDateTime;
Usage – Repository
class OrganisationBoundaryRepository extends \Radmiraal\CouchDB\Persistence\AbstractRepository {
public function findAll() { return $this->createQuery( 'organisationboundaries', 'by_organisation_id') ->execute(); }}
Usage – Repository
class DataSetRepository extends \Radmiraal\CouchDB\Persistence\AbstractRepository {
public function findByQuux(Quux $quux) { return $this->createQuery('datasets', 'most_recent', 'native') ->setStartKey(array($this->getQueryMatchValue($quux))) ->setGroup(TRUE) ->setGroupLevel(1) ->setLimit(1) ->execute()}
Usage – Views & Design Docs
And now someeven better news
Multiple backends in parallel
Doctrine DBAL
Doctrine ORM
PersistenceManager
Doctrine\Repository
MongoDB\Repository
Doctrine MongoDB
MongoDB ODM
Doctrine\Repository
The Future
TYPO3 Flow will support multiple persistence backends in parallel:
•of the same type: 2 MySQL and 1 PostgreSQL databases
•of different types: MySQL and MongoDB
The Future
Configuration extended but backwards compatible
Consistent use independent of backend used:Annotations, result interfaces & queries stay consistent
References across persistence backends? Eventually, but not right away…
You can help, too…
The work on a proper ODM integration so far has been done and made possible by (applause!):
•Beech IT
•BKWI
•Rens Admiraal
•myself
More work is needed, so if you think this is the way to go… contact me!
questionsyour
now!
Thank You!
These slides can be found at:http://speakerdeck.com/kdambekalnshttp://slideshare.net/kfish
Give me feedback: [email protected] | [email protected]
Book me: [email protected]
Follow me on twitter: @kdambekalns
Support me using
My Amazon wishlist: http://bit.ly/kdambekalns-wishlist
