handlersocket - a nosql plugin for mysql
DESCRIPTION
Presented on osdc.tw, 2011TRANSCRIPT
![Page 1: HandlerSocket - A NoSQL plugin for MySQL](https://reader036.vdocuments.site/reader036/viewer/2022081413/54853291b4af9f910d8b4d11/html5/thumbnails/1.jpg)
HANDLERSOCKET - A NOSQL PLUGIN FOR MYSQL
Jui-Nan Lin @OSDC.TW 2011
![Page 2: HandlerSocket - A NoSQL plugin for MySQL](https://reader036.vdocuments.site/reader036/viewer/2022081413/54853291b4af9f910d8b4d11/html5/thumbnails/2.jpg)
About this talk & me
URL of this Slide:
http://0rz.tw/jqQqY QRCode: About me:
Jui-Nan Lin (jnlin) Developer @
www.pixnet.net Twitter: @jnlin Github: jnlin
2
![Page 3: HandlerSocket - A NoSQL plugin for MySQL](https://reader036.vdocuments.site/reader036/viewer/2022081413/54853291b4af9f910d8b4d11/html5/thumbnails/3.jpg)
What is HandlerSocket?
“A NoSQL interface for MySQL” Only supports queries which use index Developed by DeNA (a Japanese social
gaming provider) https://github.com/ahiguti/HandlerSocket
-Plugin-for-MySQL
3
![Page 4: HandlerSocket - A NoSQL plugin for MySQL](https://reader036.vdocuments.site/reader036/viewer/2022081413/54853291b4af9f910d8b4d11/html5/thumbnails/4.jpg)
Architecture4
libmysql libhsclient
Application (Ex. PHP, Ruby, Perl …)
Listener for libmysql
SQL Layer
Handler Interface
InnoDB MyISAM Other storage engines
HandlerSocket Plugin
application
mysqld
Reference: http://www.slideshare.net/akirahiguchi/handlersocket-plugin-for-mysql-4664154
![Page 5: HandlerSocket - A NoSQL plugin for MySQL](https://reader036.vdocuments.site/reader036/viewer/2022081413/54853291b4af9f910d8b4d11/html5/thumbnails/5.jpg)
NoSQL for Web Application
Most operations in web application are simple CRUD.
RDBMS is too powerful for web application.
Typical solution: Memcached + MySQL Cassandra Apache CouchDB Amazon SimpleDB
5
![Page 6: HandlerSocket - A NoSQL plugin for MySQL](https://reader036.vdocuments.site/reader036/viewer/2022081413/54853291b4af9f910d8b4d11/html5/thumbnails/6.jpg)
NoSQL for Web Application
Pros: Scalable Fast
Cons: Unstable
Twitter drops Cassandra in 2010: http://engineering.twitter.com/2010/07/cassandra-at-twitter-today.html
6
![Page 7: HandlerSocket - A NoSQL plugin for MySQL](https://reader036.vdocuments.site/reader036/viewer/2022081413/54853291b4af9f910d8b4d11/html5/thumbnails/7.jpg)
HandlerSocket for MySQL
MySQL is very stable HandlerSocket provides a NoSQL
interface for MySQL HandlerSocket is very fast. Faster than
MySQL.
7
![Page 8: HandlerSocket - A NoSQL plugin for MySQL](https://reader036.vdocuments.site/reader036/viewer/2022081413/54853291b4af9f910d8b4d11/html5/thumbnails/8.jpg)
How fast is it
750,000 qps on a commodity server (tested by original developers) MySQL 5.1 + E5540 2.53Ghz + 32 GB RAM All data fits in memory All Clients are written in C 100,000 qps with Original MySQL
About 7.5x faster than original MySQL!
8
![Page 9: HandlerSocket - A NoSQL plugin for MySQL](https://reader036.vdocuments.site/reader036/viewer/2022081413/54853291b4af9f910d8b4d11/html5/thumbnails/9.jpg)
How fast is it
1m PK SELECT queries each test on 1 table contains 1m rows. Client is written in PHP. SELECT * from table1 WHERE pk = ?
9
Concurrent Connections
1 10 50 100
HandlerSocket
Time (sec)
256 28 15 13
QPS 3906 35714 66667 76923
MySQL
Time (sec)
372 83 87 85
QPS 2688 12048 11494 11765
![Page 10: HandlerSocket - A NoSQL plugin for MySQL](https://reader036.vdocuments.site/reader036/viewer/2022081413/54853291b4af9f910d8b4d11/html5/thumbnails/10.jpg)
Why HandlerSocket is fast10
from http://yoshinorimatsunobu.blogspot.com/2010/10/using-mysql-as-nosql-story-for.html
![Page 11: HandlerSocket - A NoSQL plugin for MySQL](https://reader036.vdocuments.site/reader036/viewer/2022081413/54853291b4af9f910d8b4d11/html5/thumbnails/11.jpg)
Why HandlerSocket is fast
HandlerSocket do not need to do: Parsing SQL Statements Opening, locking table Making SQL execution plans
Index usage Unlocking, closing tables
11
![Page 12: HandlerSocket - A NoSQL plugin for MySQL](https://reader036.vdocuments.site/reader036/viewer/2022081413/54853291b4af9f910d8b4d11/html5/thumbnails/12.jpg)
Why HandlerSocket is fast
HandlerSocket does do: Plain text-based protocol Keep table open for reuse Event based (use epoll)
Many Connection per MySQL thread Accept only index queries
Support range query, LIMIT, multi_get operations
12
![Page 13: HandlerSocket - A NoSQL plugin for MySQL](https://reader036.vdocuments.site/reader036/viewer/2022081413/54853291b4af9f910d8b4d11/html5/thumbnails/13.jpg)
Language Bindings
PHP: php-handlersocket Java: hs4j Python: python-handler-socket Ruby: ruby-handlersocket Perl: Net::HanderSocket
13
![Page 14: HandlerSocket - A NoSQL plugin for MySQL](https://reader036.vdocuments.site/reader036/viewer/2022081413/54853291b4af9f910d8b4d11/html5/thumbnails/14.jpg)
Programming Example
PHP Read Operation Port used for write operation is different
14
![Page 15: HandlerSocket - A NoSQL plugin for MySQL](https://reader036.vdocuments.site/reader036/viewer/2022081413/54853291b4af9f910d8b4d11/html5/thumbnails/15.jpg)
Configuration (my.cnf)
handlersocket_port = 9998 Port for reading
handlersocket_port_wr = 9999 Port for writing
handlersocket_threads = 16 No. of reader threads Recommended Value: # of CPUs
handlersocket_threads_wr = 1 No. of writer threads
open_files_limit = 65535
15
![Page 16: HandlerSocket - A NoSQL plugin for MySQL](https://reader036.vdocuments.site/reader036/viewer/2022081413/54853291b4af9f910d8b4d11/html5/thumbnails/16.jpg)
Pros and Cons
Pros Stable High Performance Support a lot of query types No duplicate cache (compared with MySQL
+ memcached) Invalidation Problem
Flexible Support SQL interface. We can still use SQL
statements! (ex. report generating) Write binlog and support MySQL replication.
Included in Percona Server
16
![Page 17: HandlerSocket - A NoSQL plugin for MySQL](https://reader036.vdocuments.site/reader036/viewer/2022081413/54853291b4af9f910d8b4d11/html5/thumbnails/17.jpg)
Pros and Cons
Cons No Security
Without username and password Do not update MySQL Query Cache
Fixed in recent commits No “INSERT … ON DUPLICATED KEY
UPDATE …” Feature Do not support auto increment column
while inserting Fixed in recent commits
17
![Page 18: HandlerSocket - A NoSQL plugin for MySQL](https://reader036.vdocuments.site/reader036/viewer/2022081413/54853291b4af9f910d8b4d11/html5/thumbnails/18.jpg)
Q&A
Thank you!
18