mariadb: the new m in lamp?
DESCRIPTION
TRANSCRIPT
![Page 1: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/1.jpg)
MariaDB: The New M in LAMP?
Colin Charles, Monty Program [email protected] / [email protected] http://montyprogram.com/ | http://mariadb.org/ http://bytebot.net/blog/ | @bytebot on Twitter
OpenWest, Orem, Utah, USA3 May 2013
![Page 2: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/2.jpg)
whoami
• MariaDB guy at Monty Program Ab
• SkySQL merges with Monty Program Ab
• MariaDB Server is at MariaDB Foundation
• Formerly MySQL AB/Sun Microsystems
• Past lives include Fedora Project (FESCO), OpenOffice.org
![Page 3: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/3.jpg)
Agenda
• 39 months: major server releases (5.1, 5.2, 5.3, 5.5, Galera Cluster) and 10.0 series
• Delving into history of previous releases
• MariaDB 10.0
• Client libraries, Galera Cluster
• Roadmap
![Page 4: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/4.jpg)
What is MariaDB?
• Community developed branch of MySQL
• Feature enhanced
• Fully compatible & feature complete with MySQL
![Page 5: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/5.jpg)
Ownership
• MySQL (database) owned by MySQL AB (company) -> Sun -> Oracle
• Monty Program is a major sponsor of MariaDB
• MariaDB governed by MariaDB Foundation
• maria-captains contains community like Sphinxsearch, LinkedIn, SkySQL, Taobao, Facebook, Percona, Codership, & more
![Page 6: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/6.jpg)
Aims of MariaDB
• Compatible, drop-in replacement to MySQL
• data on disk & on the wire the same
• same file names, sockets, port
• Stable (bug-free) releases with no regressions
• GPLv2
![Page 7: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/7.jpg)
Why MariaDB 10.0?
• The 5.5 merge took about a year (!)
• In MariaDB 5.5, we have over 1.5 million lines of extra code ~61MB diff
• MySQL 5.6 refactored with huge losses in commit history
• We’re not a patch set against MySQL
• MariaDB clearly does not depend on MySQL for future development
![Page 8: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/8.jpg)
MariaDB 10.0 in a nutshell
• Built on MariaDB 5.5
• Backported features from MySQL 5.6
• Multiple new features
![Page 9: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/9.jpg)
First, let’s start with some history
• MariaDB 5.1 (MySQL 5.1 base)
• Table elimination, new storage engines, code cleanup, more test cases, pool of threads
• MariaDB 5.2 (MariaDB 5.1 base)
• Virtual columns, extended user statistics, segmented MyISAM keycache
![Page 10: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/10.jpg)
History II• MariaDB 5.3 (MariaDB 5.2 base)
• Biggest changes to optimizer (faster subqueries, joins, etc.)
• Microsecond precision
• Faster HANDLER (HANDLER READ 50% faster w/530,000 qps), dynamic columns, HandlerSocket
• Better replication (group commit, checksum for binlog events, consistent snapshot between engines, etc.)
• Progress reporting for ALTER TABLE/LOAD DATA INFILE
![Page 11: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/11.jpg)
MariaDB 5.5 (MariaDB 5.3 + MySQL 5.5)
• Opensource, more efficient threadpool
• Non-blocking client library
• New LIMIT ROWS EXAMINED option
• Extended keys for XtraDB/InnoDB
• New SphinxSE
• Lots of security fixes, new status variables, etc.
![Page 12: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/12.jpg)
MariaDB 10.0
Date Version Status
12 Nov 2012
10.0.0 Alpha
6 Feb 2013
10.0.1 Alpha
24 Apr 2013
10.0.2 Alpha
10.0.3 Beta
![Page 13: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/13.jpg)
Backported features• InnoDB including TRANSACTION READ ONLY
• PERFORMANCE_SCHEMA
• Online ALTER TABLE (in-progress)
• Optimizer
• ORDER BY...LIMIT optimization (shows only few rows of a result set)
• Re-implemented:
• Error messages (w/system error string)
• CURRENT_TIMESTAMP as DEFAULT for DATETIME columns
• Global Transaction ID (10.0.2 - MDEV-26)
• Parallel replication (in progress)
• New (optimizer)
• EXISTS-TO-IN optimization
![Page 14: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/14.jpg)
Only in MariaDB 10.0: Multi-source replication
• Work from Taobao
• Many users partition data across many masters... now you can replicate many masters to a single slave
• Great for analytical queries, complete backups, etc.
![Page 15: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/15.jpg)
Only in MariaDB 10.0
• SHOW EXPLAIN for <thread_id> gets the query plan for a running statement
• Faster ALTER TABLE with unique keys for Aria & MyISAM
• Per-thread memory usage (Taobao)
• INFORMATION_SCHEMA.PROCESSLIST has MEMORY_USAGE & EXAMINED_ROWS now
• SHOW STATUS has memory usage too
![Page 16: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/16.jpg)
Only in MariaDB 10.0: CassandraSE, LevelDB support• MariaDB as a “data platform”
• Integration with NoSQL/Big Data DB, Apache Cassandra cluster, seen as a storage engine to MariaDB
• Combine (join) data between Cassandra & MariaDB & Oracle (via CONNECT)
• Write to Cassandra from SQL (SELECT, INSERT, UPDATE, DELETE)
• LevelDB (in-progress)
![Page 17: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/17.jpg)
Only in MariaDB 10.0: Engine independent persistent statistics
• InnoDB has persistent statistics in MySQL 5.6; we have an engine-independent version
• These statistics aren’t limited by the SE API, and are used by query optimizer to choose best execution plan for each statement
• Statistics collected for non-indexed columns too (unlike InnoDB’s)
![Page 18: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/18.jpg)
Dynamic columns
• Store a different set of columns for every row in the table (kinda NoSQL-like)
• Database interoperability is now a major feature; CassandraSE makes uses of this
• Can now request a row in JSON format
• And yes, you can name columns finally ;)
![Page 19: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/19.jpg)
Segmented MyISAM keycache
• Solves major read bottlenecks for MyISAM
• MyISAM usage with many readers
![Page 20: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/20.jpg)
Batched Key Access (BKA) speedups
select max(l_extendedprice) from orders, lineitem where
o_orderdate between $DATE1 and $DATE2 and
l_orderkey=o_orderkey
![Page 21: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/21.jpg)
Subqueries materialise
• Semi-join optimization, materialization for non-correlated IN queries, subquery cache
• Goodbye rewriting as JOINs or separate queries
![Page 22: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/22.jpg)
Group commit in the binary log
Source: Mark Callaghan, August 2011, https://www.facebook.com/note.php?note_id=10150261692455933
![Page 23: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/23.jpg)
Threadpool 5.5 vs 5.1
![Page 24: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/24.jpg)
PAM Authentication
• Authentication using /etc/shadow
• Authentication using LDAP, SSH pass phrases, password expiration, username mapping, logging every login attempt, etc.
• INSTALL PLUGIN pam SONAME ‘auth_pam.so’;
• CREATE USER foo@host IDENTIFIED via pam
• Remember to configure PAM (/etc/pam.d or /etc/pam.conf)
![Page 25: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/25.jpg)
SphinxSE• CREATE TABLE t1 (..)
ENGINE=SPHINX CONNECTION=”sphinx://localhost:9312/test”;
• Engine connects to Sphinx searchd
• Let indexing, searching, sorting, filtering be performed by Sphinx
• instead of WHERE, ORDER BY, LIMIT
• Sphinx is optimized/fast for these tasks
• Most of the Sphinx API is exposed to engine
• JOIN search table with other MySQL tables
![Page 26: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/26.jpg)
GIS support!
• MySQL has OpenGIS SFS (Simple feature access, SQL access method)
• Now, SQL with full geometry types
• ST_ prefix
• http://kb.askmonty.org/en/gis-features-in-533
![Page 27: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/27.jpg)
MariaDB 10.0.2
• Support for atomic writes on FusionIO DirectFS
• Optimizer collects & can use histogram-based statistics for non-indexed columns
• Better table discovery, so FederatedX has assisted discovery, Sequence engine (creates ascending/descending sequences, useful in joins)
![Page 28: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/28.jpg)
Roadmap
• MariaDB is already a superset of features in MySQL
• Merge is in 2 steps
• 10.0.x
• 10.1.x
• Plan is to have all important features of MySQL 5.6 by the time MariaDB 10.1.x is released as stable
• For all practical purposes MariaDB 10.1.x will be a drop in replacement to MySQL 5.6
![Page 29: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/29.jpg)
LGPL Client Libraries
• LGPL client libraries for C & Java
• Works with MariaDB/MySQL/Percona Server
• Developed by Monty Program Ab & SkySQL Ab, released as LGPL 29 Nov 2012
• ODBC is in the works, sponsors are welcome!
![Page 30: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/30.jpg)
MariaDB Galera Cluster
• MariaDB Galera Cluster is made for today’s cloud based environments. It is fully read-write scalable, comes with synchronous replication, allows multi-master topologies, and guarantees no lag or lost transactions.
• Currently 5.5-based, we see this as important for 10.0 as well (merges will happen closer to GA)
![Page 31: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/31.jpg)
TokuDB
• It’s now opensource
• Improved insert & query speed, compression, replication performance and online schema flexibility
• Uses Fractal Tree Indexes instead of B-Tree
• Tests & builds of TokuDB on multiple platforms (think greater distribution)
![Page 32: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/32.jpg)
CONNECT Storage Engine
• Made by Olivier Bertrand
• Read, write & update files in different storage formats:
• .DBF (dBASE format)
• .CSV
• .INI
• XML
• ODBC
• Possible to join data from CassandraSE, XtraDB and an ODBC data source like Oracle
![Page 33: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/33.jpg)
Benchmarks
• “Lies, damned lies, and statistics” - Mark Twain
• http://blog.mariadb.org/sysbench-oltp-mysql-5-6-vs-mariadb-10-0/
• http://dimitrik.free.fr/blog/archives/2013/02/mysql-performance-mysql-56-vs-mysql-55-vs-mariadb-55.html
• Yes, we’ve gotten Oracle to notice MariaDB :-)
![Page 34: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/34.jpg)
Continued commitments
• Security
• Since about a year now, we’re the go-to people for security - good track record
• We don’t like regressions
• http://www.skysql.com/blogs/hartmut/nasty-innodb-regression-mysql-5525
• http://www.skysql.com/blogs/kolbe/heads-no-more-query-cache-partitioned-tables-mysql-5523
• We care about backward compatibility & introduce features carefully
• XtraDB innodb_adaptive_checkpoint=none|reflex|estimate|keep_average (no more reflex...)
![Page 35: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/35.jpg)
Community involvement
• Many features since MariaDB 5.2 have come from the community or are sponsored features (5.3, 5.5)
• Knowledgebase has 2,600+ articles in English
• Yearly downloads of half a million (more users from mirrors + distributions)
• Active mailing lists (stats on KB)
• Google Summer of Code 2013
• MariaDB User Groups
![Page 36: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/36.jpg)
MariaDB is gaining popularity
• Wikipedia (English, German), Wikidata running MariaDB 5.5
• Fedora, OpenSUSE shipping MariaDB as a default
• Slackware, Chakra Linux, ArchLinux have followed suit
• Many success stories at KB: Limelight Networks, Nimbuzz, Paybox, FictionPress, OLX, SlashGear, Web of Trust, SpamExperts, Cougarboard, etc.
![Page 37: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/37.jpg)
Deployments!
“MariaDB had these same bugs that we ran into with MySQL. However the big difference was that when we reported these bugs, they were quickly resolved within 48 hours!” -- Dreas van Donselaar, Chief Technology Officer, SpamExperts B.V. after migrating over 300 servers from MySQL 5.0 to MariaDB 5.1.
“Migrating from MySQL 5.1 to MariaDB 5.2 was as simple as removing MySQL RPMs and installing the MariaDB packages, then running mysql_upgrade.” - Panayot Belchev, proprietor, Host Bulgaria on providingMariaDB to over 7,000 of their web hosting customers.
“We made the switch on Saturday -- and we’re seeing benefits already -- our daily optimization time is down from 24 minutes to just 4 minutes” -- Ali
Watters, CEO, travelblog.org
happy users: pap.fr, wabtec, Paybox Services, OLX, Jelastic, Web of Trust, SaltOS, ERP5, etc.
“@nginxorg & @mariadb have helped me save
$12000/year in infrastructure cost. I love it!
Do more with less!” - Ewdison Then, CEO,
Slashgear
![Page 38: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/38.jpg)
MariaDB Foundation
• Foundation is driver of MariaDB project
• Custodian of code, guardian of community
• Foundation can never be controlled by single entity or person
• Designed to be self-sustaining
![Page 39: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/39.jpg)
MariaDB Foundation Goals
• Increase adoption of MariaDB
• Ensure sustainable high-quality efforts to build, test and distribute MariaDB
• Ensure that community patches are reviewed and adopted
• Guarantee a community voice
• Keep MariaDB compatible with MySQL
• Maintain mariadb.org
![Page 40: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/40.jpg)
MariaDB Foundation People
• More founders & sponsors are welcome
• Michael Widenius, CTO: [email protected]
• Andrew Katz, Legal Counsel: [email protected]
• Simon Phipps, CEO: [email protected]
• Jeremy Zawodny (Craigslist), Mike Milinkovich (Eclipse Foundation) on Board
![Page 41: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/41.jpg)
Coming up next
• Merge SPIDER storage engine
• Column level encryption via MyDiamo from Penta Security
• New optimizations for Fusion-IO
• You can shape the roadmap
![Page 42: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/42.jpg)
Well supported
• Everyone supports MariaDB from a support standpoint with the exception of Oracle
• SkySQL, Percona, etc.
• Jelastic has it as PaaS too
• We support all GA releases for 5 years for security, etc.
![Page 43: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/43.jpg)
User stats plugin
• Disabled by default, consider enabling it to show use! http://mariadb.org/feedback_plugin/
data from over 113 countries!
![Page 44: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/44.jpg)
Compatibility with MySQL
• No NDB Cluster
• XtraDB enabled as default up to 5.5, InnoDB in 10.0 (for now)
• Optimizer: no worse than current plan
![Page 45: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/45.jpg)
FAQ• Can I replicate from MySQL 5.6 to MariaDB 10.x?
• Yes
• Can I replicate from MariaDB 10.x to MariaDB 5.5?
• Yes
• Can I replicate from MariaDB 10.x to MySQL 5.5/5.6?
• No
![Page 46: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/46.jpg)
FAQ on tools
• SELECT VERSION() returns 10.0.1-MariaDB
• Version string in handshake packet will be 5.5.30-mysql-10.0.2-MariaDB (mysql#68187, MDEV-4088)
• Tools should start recognising MariaDB for additional feature-set (mytop, HeidiSQL, etc.)
![Page 47: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/47.jpg)
Conclusion
• We’ve spent a lot of time adding features, some in parallel, some ahead
• MariaDB is binary compatible with MySQL
• Open bugs system, test suite, discussion lists
• Opensource, feature rich, no commercial extensions
![Page 48: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/48.jpg)
Resources• bugs: mariadb.org/jira
• fb.com/MariaDB.dbms
• twitter: @mariadb
• google plus: +MariaDB
• #maria on irc.freenode.net
• http://kb.askmonty.org/
![Page 49: MariaDB: The New M in LAMP?](https://reader033.vdocuments.site/reader033/viewer/2022042714/54c6369c4a7959e43f8b4608/html5/thumbnails/49.jpg)
Q&[email protected] | [email protected] http://montyprogram.com/ | http://mariadb.org/ twitter: @bytebot | url: http://bytebot.net/blog/
planetmysql.orgplanetmariadb.org