c* summit eu 2013: delivering christmas gifts in france since 2012
DESCRIPTION
Speaker: Matthieu Nantern, Software Engineer at Xebia Video: http://youtu.be/ci9lrViHE9E Every year more and more people buy their Christmas gifts online and that gifts are delivered by the postal service of France "La Poste". At the end of the infrastructure a (not so) little MySQL was struggling for survival against that overwhelming load. Then, in 2011, MySQL hit its limit... Come hear the true story of La Poste switching its parcel management, PHP-based application, from MySQL to Cassandra in 3 weeks. You'll be taught about the details of the project constraints, how to use Cassandra from PHP, the migration plan, how to manage resilience testing, deploy your Cassandra with Puppet, and all the wonderful knowledge we accumulate through this project.TRANSCRIPT
Cassandra Delivering Christmas gifts in France since
2012
www.xebia.fr | http://blog.xebia.fr | #CassandraEU Telephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Puppet
Me, myself and I
�2
SoftwareCassandr
aMatthie
u Nantern
@mNantern
www.xebia.fr | http://blog.xebia.fr | #CassandraEU Telephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Context
Delivering parcels at La Poste
Why ?
Why do we need another system ?
Project’s constraints and
alternatives
How ?
How we build our solution. Using C*
with PHP and managing TTL
Go Live!
Provisioning servers.
Performance and resilience testing
41 2 3
OverviewFrom Mysql to Cassandra
�3
www.xebia.fr | http://blog.xebia.fr | #CassandraEU Telephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Mail services
Parcels distribution
More than 250 000 employees
70 000 postmen delivering parcels
La Poste
�4
www.xebia.fr | http://blog.xebia.fr | #CassandraEU Telephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Scan Print
Scan Distribute
La PostePostman’s tour
�5
(1) A postman scans every parcels
(2) He prints his list of parcels
(3) He distributes the parcels
(4) He scans every remaining parcels
Why ?
From MySQL to C*
www.xebia.fr | http://blog.xebia.fr | #CassandraEU Telephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Why do we need another system ?
�7
www.xebia.fr | http://blog.xebia.fr | #CassandraEU Telephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Max 13ms per call
15 days of retention for data
Easy to deploy and operate
Easy to scale for years to come
Project’s constraints
�8
How ?
From MySQL to C*
www.xebia.fr | http://blog.xebia.fr | #CassandraEU Telephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Data Modeling
�10
Id Data key1 key2 key3
1 <Xml> k11 k21 k31
2 <Xml> k12 k22 k32
3 <Xml> k13 k23 k33
Id Data Search1 Search2
1 <Xml> k11k31 k21k11
2 <Xml> k12k32 k22k12
3 <Xml> k13k33 k23k13
www.xebia.fr | http://blog.xebia.fr | #CassandraEU Telephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Using Cassandra with PHP
�11
$pool = new ConnectionPool("Keyspace1", array("cass:9160"));
$column_family = new ColumnFamily($pool, 'ColumnFamily1');
For dummies
$column_family->insert('row_key', array('name1' => 'val1', 'name2' => 'val2'));$column_family->get('row_key');
$row1 = array('name1' => 'val1', 'name2' => 'val2'); $row2 = array('foo' => 'bar'); $column_family->batch_insert(array('row1' => $row1, 'row2' => $row2);
www.xebia.fr | http://blog.xebia.fr | #CassandraEU Telephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Using Cassandra with PHP
�12
Level 2
$index_exp=CassandraUtil::create_index_expression($indexName, $indexValue,$operator); !
!
$index_clause = CassandraUtil::create_index_clause($indexArray); !
!
$rows = $columnFamily->get_indexed_slices($index_clause);
www.xebia.fr | http://blog.xebia.fr | #CassandraEU Telephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
TTL in C*
�13
$cf->insert('row_key', array('col1' => 'col_val1','col2' =>'col_val2'),null,TTL);
$columnsTTL = array('col1'=>3, 'col2'=>4); !
$cf->insert('row_key', array('col1' => 'val1','col2' =>'val2'),null, $columnsTTL);
Go Live!
From MySQL to C*
www.xebia.fr | http://blog.xebia.fr | #CassandraEU Telephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Provisioning C* with Puppet
•12 servers with Ubuntu 10.04
!
�15
package { 'cassandra': ensure => installed }
initial_token: <%= ((2**127) * (current_cdb_server.to_i-1) / cdb_servers_number.to_i ) %>
www.xebia.fr | http://blog.xebia.fr | #CassandraEU Telephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Provisioning C* with Puppet
�16
package { 'cassandra-data-model': ensure => latest }
Managing the data model
Current Migration Id
2
migration_1.txt migration_2.txt migration_3.txt migration_4.txt
www.xebia.fr | http://blog.xebia.fr | #CassandraEU Telephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Testing our cluster
�17
Mysql performance
www.xebia.fr | http://blog.xebia.fr | #CassandraEU Telephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Testing our cluster
�18
Cassandra performance
www.xebia.fr | http://blog.xebia.fr | #CassandraEU Telephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Testing our cluster
�19
Resilience
www.xebia.fr | http://blog.xebia.fr | #CassandraEU Telephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Migration plan
�20
t=0
WriteRead
t+15d
Read Write
Final words
From MySQL to C*
YOUFor watching
THANK
Merci!
www.xebia.fr | http://blog.xebia.fr | #CassandraEU Telephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Thanks !
�23
Thank you for the support !
Thank you for the opportunity to work at La Poste !
Thank you for trusting me on Cassandra (and everything else)!