mysql nosql apis
DESCRIPTION
TRANSCRIPT
NoSQL APIsMorgan TockerMySQL Community Manager
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!2
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!3
MySQL 5.5
MySQL Enterprise Monitor 2.3 & 3.0MySQL Enterprise Backup Security Scalability HA Audit
MySQL 5.6MySQL Workbench 6.1
MySQL Utilities
MySQL Applier for Hadoop
MySQL Workbench 5.2 & 6.0MySQL Enterprise Oracle Certifications
MySQL@Oracle: 4 Years of MySQL Innovation
MySQL Cluster ManagerWindows installer & Tools
MySQL Cluster 7.2MySQL Cluster 7.1
MySQL Migration Wizard MySQL 5.7
MySQL Cluster 7.4
MySQL Fabric
MySQL Cluster 7.3
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!4
Discussion Topics
▪ What is memcached? ▪ History of HandlerSocket ▪ NoSQL Interface in MySQL 5.6 ▪ Future in MySQL 5.7
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!5
What is memcached?
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!6
What do we know about memcached?
▪ Simple in-memory caching system. ▪ APIs available for most programming languages. ▪ Stores data in memory. ▪ Excellent read performance. ▪ Great write performance. ▪ Supports distribution across multiple servers. ▪ No security.
Good Introduction: https://code.google.com/p/memcached/wiki/NewProgrammingFAQ
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!7
Common Configuration
▪ As a look-aside cache:
▪ Also possible to rebuild cache from cron/inject new cached data.
value = get_from_memcache(key); if (value === null) { value = get_value_from_database(key); set_memcache(key, value) }
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!8
Scaling
▪ Hashing is performed by the clients. ▪ Mod-based hashing. ▪ Consistent hashing.
▪ No cross talk by memcached servers.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!9
History of HandlerSocket
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!10
The beginning
▪ Yoshinori Matsunobu working at DeNA. ▪ Profiled MySQL and discovered a lot of time spent in:
▪ Parsing SQL statements ▪ Opening, locking tables ▪ Making SQL execution plans ▪ Unlocking, closing tables
http://yoshinorimatsunobu.blogspot.ca/2010/10/using-mysql-as-nosql-story-for.html
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!11
The Question
▪ Is SQL really good for fast PK lookups? ▪ MySQL could only perform 100,000+ queries per second ▪ Memcached 400,000+ gets per second.
▪ Is it possible to bypass some parts of MySQL and get higher throughput?
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!12
Implementation
Credits: http://yoshinorimatsunobu.blogspot.ca/2010/10/using-mysql-as-nosql-story-for.html
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!13
The Result
! approx qps server CPU util MySQL via SQL 105,000 %us 60% %sy 28% memcached 420,000 %us 8% %sy 88% MySQL via HandlerSocket 750,000 %us 45% %sy 53%
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!14
Outcome
▪ Benefits: ▪ No inconsistent data. ▪ Crash-safe storage ▪ SQL can still be used. ▪ All operational benefits of
MySQL (backup, etc.) !!
▪ Limitations: ▪ New API ▪ No Security ▪ No Benefit for HDD bound
workloads
▪ In the case of DeNA: ▪ Switched from using memcached as a row cache. ▪ Kept memcached as a page/larger object cache.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!15
NoSQL Interface in MySQL 5.6
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!16
MySQL 5.6
▪ Builds on the spirit of HandlerSocket. ▪ New implementation based on the standard memcached protocol.
▪ From a technical detail actually embeds memcached, but your data remains safe between restarts.
▪ i.e. not just in memory.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!17
Architecture
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!18
Implementation Details
▪ Plugin ships included in MySQL packages. ▪ Not enabled by default (security). ▪ Tables have to “opt-in” to be exposed via memcached.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!19
Setup (1/3)
# Add this to my.cnf file: !plugin-load = daemon_memcached=libmemcached.so !# Load included SQL file to initialize innodb_memcached # schema with permitted mappings. !mysql> source /usr/share/mysql/innodb_memcached_config.sql
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!20
Setup (2/3)
mysql> show plugins; +----------------------------+----------+--------------------+-----------------+---------+ | Name | Status | Type | Library | License | +----------------------------+----------+--------------------+-----------------+---------+ | binlog | ACTIVE | STORAGE ENGINE | NULL | GPL | | mysql_native_password | ACTIVE | AUTHENTICATION | NULL | GPL | | mysql_old_password | ACTIVE | AUTHENTICATION | NULL | GPL | | sha256_password | ACTIVE | AUTHENTICATION | NULL | GPL | | CSV | ACTIVE | STORAGE ENGINE | NULL | GPL | | MEMORY | ACTIVE | STORAGE ENGINE | NULL | GPL | | MyISAM | ACTIVE | STORAGE ENGINE | NULL | GPL | | MRG_MYISAM | ACTIVE | STORAGE ENGINE | NULL | GPL | | ARCHIVE | ACTIVE | STORAGE ENGINE | NULL | GPL | | BLACKHOLE | ACTIVE | STORAGE ENGINE | NULL | GPL | | FEDERATED | DISABLED | STORAGE ENGINE | NULL | GPL | | InnoDB | ACTIVE | STORAGE ENGINE | NULL | GPL | | INNODB_TRX | ACTIVE | INFORMATION SCHEMA | NULL | GPL | | INNODB_LOCKS | ACTIVE | INFORMATION SCHEMA | NULL | GPL | | INNODB_LOCK_WAITS | ACTIVE | INFORMATION SCHEMA | NULL | GPL | | .. | INNODB_SYS_DATAFILES | ACTIVE | INFORMATION SCHEMA | NULL | GPL | | PERFORMANCE_SCHEMA | ACTIVE | STORAGE ENGINE | NULL | GPL | | partition | ACTIVE | STORAGE ENGINE | NULL | GPL | | daemon_memcached | ACTIVE | DAEMON | libmemcached.so | GPL | +----------------------------+----------+--------------------+-----------------+---------+ 43 rows in set (0.01 sec)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!21
Setup (Part 3/3)
# Pretend this is my hypothetical table: !mysql> create database myapp; mysql> create table myapp.user_session_data (id_name varchar(255), value varchar(255), expiretime int, primary key (id_name)); !# Insert your desired mapping into the containers table: !mysql> insert into innodb_memcache.containers values ("sessions", "myapp", "user_session_data", "id_name", "value", 0, 0, "expiretime", "PRIMARY");
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!22
Setup - Complete
mysql> select * from containers\G .. *************************** 2. row *************************** name: sessions db_schema: myapp db_table: user_session_data key_columns: id_name value_columns: value flags: 0 cas_column: 0 expire_time_column: expiretime unique_idx_name_on_key: PRIMARY 2 rows in set (0.00 sec)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!23
Testing
shell> telnet 127.0.0.1 11211 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. set @@sessions.1|color 0 600 5 black STORED set @@sessions.1|size 0 600 5 large STORED quit
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!24
Testing (cont.)
mysql> select * from user_session_data\G *************************** 1. row *************************** id_name: 1|color value: black expiretime: 1398608066 *************************** 2. row *************************** id_name: 1|size value: large expiretime: 1398608106 2 rows in set (0.00 sec)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!25
Updating Multiple Columns At Once
▪ Requires Serialization using the “|” character:
DROP TABLE myapp.users; CREATE TABLE myapp.users ( `user_name` varchar(60) NOT NULL PRIMARY KEY, `first_name` varchar(60), `last_name` varchar(60), expire_time_column INT ) ENGINE=InnoDB; !!insert into innodb_memcache.containers (name,db_schema,db_table,key_columns,value_columns,unique_idx_name_on_key) VALUES (‘users’,'myapp','users','user_name', 'first_name,last_name','PRIMARY');
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!26
Testing Multiple Columns:
set @@users.fred 0 0 10 Fred|Smith STORED !mysql> select * from users; +-----------+------------+-----------+--------------------+ | user_name | first_name | last_name | expire_time_column | +-----------+------------+-----------+--------------------+ | fred | Fred | Smith | 0 | +-----------+------------+-----------+--------------------+ 1 row in set (0.00 sec)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!27
Benchmarks
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!28
Future in MySQL 5.7
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!29
What has changed?
▪ Improvements inside of InnoDB to scale better. ▪ Several contentions related to MVCC code reduced.
▪ memcached plugin tuned to keep several reads within a single internal transaction.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!30
Benchmarks
48 Core Machine / Read Only Memcached http://dimitrik.free.fr/blog/archives/2013/11/mysql-performance-over-1m-qps-with-innodb-memcached-plugin-in-mysql-57.html
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!31
Further Reading
▪ Article by Matt Lord and Andrew Morgan:http://www.drdobbs.com/database/nosql-with-mysql/240167115 !
▪ MySQL Manual page:https://dev.mysql.com/doc/refman/5.6/en/innodb-memcached-setup.html
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12!32
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12!33