15 protips for mysql users pfz
DESCRIPTION
TRANSCRIPT
1
15 Pro-tips for MySQL UsersPFCongress - 17 september 2011Utrecht - Netherlands
whoami
2
Joshua ThijssenSenior Software Engineer @ EnriseDevelopment in PHP, Python, Perl, C, Java....
Blogs:http://www.adayinthelifeof.nlhttp://www.enrise.com/blogEmail: [email protected]
Twitter: @jaytaph
3
What are we going to discuss?
‣ QUESTIONS? RAISE YOUR HAND OR YELL LOUD
‣ 15 MySQL Pro-tips
‣ No “theoretical tips”, all taken from the field.
‣ Starting simple - ending “complex”
4
Tip 0
‣ MySQL 5.0 != 5.5
0) Use the correct MySQL version
5
Tip 0: Use the correct MySQL version
‣ RHEL 5-7: 5.0.77
‣ RHEL 6-1: 5.1.52
‣ Debian (lenny) 5.0.51a
‣ Debian (squeeze): 5.1.49
‣ Debian (sid): 5.1.58
http://distrowatch.com/table.php?distribution=redhathttp://distrowatch.com/table.php?distribution=debian
6
Tip 1
‣ EXPLAIN IS YOUR BESTEST FRIEND
1) Know how to use explain (and profiler)
7
Tip 1: Know your EXPLAIN (1)
‣ I will not show you how to use EXPLAIN.
‣ Use EXPLAIN and EXPLAIN EXTENDED/ SHOW WARNINGS;
8
Tip 1: Know your EXPLAIN (2)
mysql> desc varchartest;+----------+--------------+------+-----+---------+-------+| Field | Type | Null | Key | Default | Extra |+----------+--------------+------+-----+---------+-------+| id | int(11) | NO | PRI | 0 | || name | varchar(255) | NO | MUL | NULL | || utf8name | varchar(255) | NO | MUL | NULL | |+----------+--------------+------+-----+---------+-------+3 rows in set (0.01 sec)
mysql> desc varchartest;+----------+--------------+------+-----+---------+-------+| Field | Type | Null | Key | Default | Extra |+----------+--------------+------+-----+---------+-------+| id | int(11) | NO | PRI | 0 | || name | varchar(255) | NO | MUL | NULL | || utf8name | varchar(255) | NO | MUL | NULL | |+----------+--------------+------+-----+---------+-------+3 rows in set (0.01 sec)
9
Tip 1: Know your EXPLAIN (3)
mysql> EXPLAIN EXTENDED -> SELECT * FROM varchartest WHERE name LIKE 'joshua';+----+-------------+-------------+-------+---------------+----------+---------+------+------+----------+-------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |+----+-------------+-------------+-------+---------------+----------+---------+------+------+----------+-------------+| 1 | SIMPLE | varchartest | range | idx_name | idx_name | 257 | NULL | 1 | 100.00 | Using where |+----+-------------+-------------+-------+---------------+----------+---------+------+------+----------+-------------+1 row in set, 1 warning (0.01 sec)
mysql> EXPLAIN EXTENDED -> SELECT * FROM varchartest WHERE name LIKE 'joshua';+----+-------------+-------------+-------+---------------+----------+---------+------+------+----------+-------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |+----+-------------+-------------+-------+---------------+----------+---------+------+------+----------+-------------+| 1 | SIMPLE | varchartest | range | idx_name | idx_name | 257 | NULL | 1 | 100.00 | Using where |+----+-------------+-------------+-------+---------------+----------+---------+------+------+----------+-------------+1 row in set, 1 warning (0.01 sec)
10
Tip 1: Know your EXPLAIN (4)
mysql> SHOW WARNINGS\g+-------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+| Level | Code | Message |+-------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+| Note | 1003 | select `pfz`.`varchartest`.`id` AS `id`,`pfz`.`varchartest`.`name` AS `name`,`pfz`.`varchartest`.`utf8name` AS `utf8name` from `pfz`.`varchartest` where (`pfz`.`varchartest`.`name` like 'joshua') |+-------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
mysql> SHOW WARNINGS\g+-------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+| Level | Code | Message |+-------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+| Note | 1003 | select `pfz`.`varchartest`.`id` AS `id`,`pfz`.`varchartest`.`name` AS `name`,`pfz`.`varchartest`.`utf8name` AS `utf8name` from `pfz`.`varchartest` where (`pfz`.`varchartest`.`name` like 'joshua') |+-------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
11
Tip 1: Know your PROFILER (1)
mysql> SET profiling=1;mysql> SELECT * FROM table;mysql> SHOW PROFILE CPU, BLOCK IO;+--------------------------------+----------+----------+------------+--------------+---------------+| Status | Duration | CPU_user | CPU_system | Block_ops_in | Block_ops_out |+--------------------------------+----------+----------+------------+--------------+---------------+| starting | 0.000930 | 0.000000 | 0.000000 | 0 | 8 || checking query cache for query | 0.000547 | 0.000000 | 0.000000 | 0 | 0 || checking permissions | 0.000045 | 0.000000 | 0.000000 | 0 | 0 || Opening tables | 0.000142 | 0.000000 | 0.000000 | 0 | 0 || System lock | 0.000030 | 0.000000 | 0.000000 | 0 | 0 || Table lock | 0.000045 | 0.000000 | 0.000000 | 0 | 0 || init | 0.000207 | 0.000000 | 0.000000 | 0 | 0 || optimizing | 0.000007 | 0.000000 | 0.000000 | 0 | 0 || statistics | 0.000068 | 0.000000 | 0.000000 | 0 | 0 || preparing | 0.001393 | 0.004000 | 0.000000 | 0 | 0 || Creating tmp table | 0.001658 | 0.000000 | 0.000000 | 0 | 8 || executing | 0.000005 | 0.000000 | 0.000000 | 0 | 0 || Copying to tmp table | 0.000834 | 0.000000 | 0.004001 | 0 | 8 || Sorting result | 0.000179 | 0.000000 | 0.000000 | 0 | 0 || Sending data | 0.000089 | 0.000000 | 0.000000 | 0 | 0 || end | 0.000022 | 0.000000 | 0.000000 | 0 | 0 || removing tmp table | 0.000089 | 0.000000 | 0.000000 | 0 | 0 || end | 0.000005 | 0.000000 | 0.000000 | 0 | 0 || query end | 0.000004 | 0.000000 | 0.000000 | 0 | 0 || freeing items | 0.000844 | 0.000000 | 0.000000 | 0 | 0 || logging slow query | 0.000023 | 0.000000 | 0.000000 | 0 | 0 || logging slow query | 0.000060 | 0.000000 | 0.000000 | 0 | 8 || cleaning up | 0.000007 | 0.000000 | 0.000000 | 0 | 0 |+--------------------------------+----------+----------+------------+--------------+---------------+
mysql> SET profiling=1;mysql> SELECT * FROM table;mysql> SHOW PROFILE CPU, BLOCK IO;+--------------------------------+----------+----------+------------+--------------+---------------+| Status | Duration | CPU_user | CPU_system | Block_ops_in | Block_ops_out |+--------------------------------+----------+----------+------------+--------------+---------------+| starting | 0.000930 | 0.000000 | 0.000000 | 0 | 8 || checking query cache for query | 0.000547 | 0.000000 | 0.000000 | 0 | 0 || checking permissions | 0.000045 | 0.000000 | 0.000000 | 0 | 0 || Opening tables | 0.000142 | 0.000000 | 0.000000 | 0 | 0 || System lock | 0.000030 | 0.000000 | 0.000000 | 0 | 0 || Table lock | 0.000045 | 0.000000 | 0.000000 | 0 | 0 || init | 0.000207 | 0.000000 | 0.000000 | 0 | 0 || optimizing | 0.000007 | 0.000000 | 0.000000 | 0 | 0 || statistics | 0.000068 | 0.000000 | 0.000000 | 0 | 0 || preparing | 0.001393 | 0.004000 | 0.000000 | 0 | 0 || Creating tmp table | 0.001658 | 0.000000 | 0.000000 | 0 | 8 || executing | 0.000005 | 0.000000 | 0.000000 | 0 | 0 || Copying to tmp table | 0.000834 | 0.000000 | 0.004001 | 0 | 8 || Sorting result | 0.000179 | 0.000000 | 0.000000 | 0 | 0 || Sending data | 0.000089 | 0.000000 | 0.000000 | 0 | 0 || end | 0.000022 | 0.000000 | 0.000000 | 0 | 0 || removing tmp table | 0.000089 | 0.000000 | 0.000000 | 0 | 0 || end | 0.000005 | 0.000000 | 0.000000 | 0 | 0 || query end | 0.000004 | 0.000000 | 0.000000 | 0 | 0 || freeing items | 0.000844 | 0.000000 | 0.000000 | 0 | 0 || logging slow query | 0.000023 | 0.000000 | 0.000000 | 0 | 0 || logging slow query | 0.000060 | 0.000000 | 0.000000 | 0 | 8 || cleaning up | 0.000007 | 0.000000 | 0.000000 | 0 | 0 |+--------------------------------+----------+----------+------------+--------------+---------------+
12
Tip 2
‣ THERE ARE ONLY A FEW “BASIC” ONES.
2) Know the most basic my.cnf settings
13
Tip 2: My.cnf settings (1)
Know the most important ones:key_buffer_size, innodb_buffer_pool_size, sort_buffer_size, max_connections
14
Tip 2: My.cnf settings (2)
‣ Some settings work on global level, some per connection!
‣ Know some quirks: (max_heap_table_size vs tmp_table_size, binlog-do-db, replicate-ignore-db etc)
15
Tip 2: My.cnf settings (3)
‣ http://www.omh.cc/mycnf/
‣ http://rackerhacker.com/mysqltuner/
‣ http://www.day32.com/MySQL/
‣ phpmyadmin
16
Tip 3
‣ RESTORING JUST ONE TABLE CAN BE PAINFUL OTHERWISE
3) Backup on table level
17
Tip 3: Backup on table level (1)
‣ COULD YOU RESTORE TABLE x? YES! YES I CAN!
‣ mysqldump can dump per database OR by table.
‣ Simple scripts to scan/dump tables.
‣ Easy restore for single table (or part of table)
18
Tip 4
‣ DON’T ASK WHAT YOU DON’T NEED
4) Don’t use “SELECT *” when you only need one or two fields.
19
Tip 4: Select * (1)
‣ DON’T ASK WHAT YOU DON’T NEED
‣ Much more data to be read from disk
‣ Much more data will be send over, thus slower (blobs/texts)
‣ Cannot use covering indices
20
Tip 4: Select * (2)
mysql> SHOW FULL COLUMNS FROM `covering`;+--------------+------------------+------------------+------+-----+---------+----------------+| Field | Type | Collation | Null | Key | Default | Extra |+--------------+------------------+------------------+------+-----+---------+----------------+| id | int(10) unsigned | NULL | NO | PRI | NULL | auto_increment || email | varchar(255) | ascii_general_ci | NO | MUL | NULL | | | want_mailing | tinyint(1) | NULL | NO | MUL | NULL | || extra_info | varchar(255) | ascii_general_ci | NO | MUL | NULL | |+--------------+------------------+------------------+------+-----+---------+----------------+3 rows in set (0.00 sec)mysql> SHOW INDEXES FROM `covering`;+----------+------------+------------+--------------+--------------+-----------+-------------+| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | +----------+------------+------------+--------------+--------------+-----------+-------------+| covering | 0 | PRIMARY | 1 | id | A | 3 || covering | 1 | idx_email | 1 | email | A | 3 || covering | 1 | idx_email | 2 | want_mailing | A | 3 || covering | 1 | idx_email2 | 1 | want_mailing | A | 1 || covering | 1 | idx_email2 | 2 | email | A | 3 |+----------+------------+------------+--------------+--------------+-----------+-------------+5 rows in set (0.01 sec)
mysql> SHOW FULL COLUMNS FROM `covering`;+--------------+------------------+------------------+------+-----+---------+----------------+| Field | Type | Collation | Null | Key | Default | Extra |+--------------+------------------+------------------+------+-----+---------+----------------+| id | int(10) unsigned | NULL | NO | PRI | NULL | auto_increment || email | varchar(255) | ascii_general_ci | NO | MUL | NULL | | | want_mailing | tinyint(1) | NULL | NO | MUL | NULL | || extra_info | varchar(255) | ascii_general_ci | NO | MUL | NULL | |+--------------+------------------+------------------+------+-----+---------+----------------+3 rows in set (0.00 sec)mysql> SHOW INDEXES FROM `covering`;+----------+------------+------------+--------------+--------------+-----------+-------------+| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | +----------+------------+------------+--------------+--------------+-----------+-------------+| covering | 0 | PRIMARY | 1 | id | A | 3 || covering | 1 | idx_email | 1 | email | A | 3 || covering | 1 | idx_email | 2 | want_mailing | A | 3 || covering | 1 | idx_email2 | 1 | want_mailing | A | 1 || covering | 1 | idx_email2 | 2 | email | A | 3 |+----------+------------+------------+--------------+--------------+-----------+-------------+5 rows in set (0.01 sec)
21
Tip 4: Select * (2)
mysql> EXPLAIN SELECT want_mailing FROM `covering` WHERE email LIKE '[email protected]';+----+-------------+----------+-------+---------------+-----------+---------+------+------+--------------------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |+----+-------------+----------+-------+---------------+-----------+---------+------+------+--------------------------+| 1 | SIMPLE | covering | range | idx_email | idx_email | 257 | NULL | 1 | Using where; Using index |+----+-------------+----------+-------+---------------+-----------+---------+------+------+--------------------------+1 row in set (0.00 sec)
mysql> EXPLAIN SELECT want_mailing FROM `covering` WHERE email LIKE '[email protected]';+----+-------------+----------+-------+---------------+-----------+---------+------+------+--------------------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |+----+-------------+----------+-------+---------------+-----------+---------+------+------+--------------------------+| 1 | SIMPLE | covering | range | idx_email | idx_email | 257 | NULL | 1 | Using where; Using index |+----+-------------+----------+-------+---------------+-----------+---------+------+------+--------------------------+1 row in set (0.00 sec)
22
Tip 5
‣ ENFORCE CONSISTENCY
5) Use triggers and stored procedures
23
Tip 5: Triggers and stored procedures (1)
‣ ENFORCE CONSISTENCY
‣ 6 triggers per table (insert, update, delete, before and after the mutation)
‣ 3rd party tools (phpmyadmin etc) can also use the database without loosing data consistency.
‣ Watch out with (phpmyadmin) table dumps!
24
Tip 6
‣ THERE ARE MUCH BETTER SOLUTIONS
6) Don’t use FULLTEXT searches
25
Tip 6: Don’t use FULLTEXT search (1)
‣ THERE ARE MUCH BETTER SOLUTIONS
‣ They only work for MyISAM tables.
‣ Not compatible with other DB’s.
‣ Slow (especially compared to Solr, Sphinx).
‣ No extra features (faceted search, spell checking etc).
26
Tip 7
‣ IT LOOKS LIKE YOU NEED MORE ADVANCED SEARCH?
7) Some wildcard searches (%item%) are bad for performance
27
Tip 7: Wildcard searches (1)
‣ THERE ARE MUCH BETTER SOLUTIONS
‣ MySQL cannot use indexing!
‣ Revert your data: search for ‘moc.esirne@%’ instead of ‘%@enrise.com’.
‣ Use a better solution (solr, sphinx). You probably want it (no, really)
28
Tip 8
‣ MAKE CACHING AND LOCKING HAPPY AGAIN
8) Shard your volatile and non-volatile data.
29
Tip 8: Sharding (1)
mysql> SHOW FULL COLUMNS FROM `pages`;+------------+------------------+-------------------+------+-----+-------------------+-------+| Field | Type | Collation | Null | Key | Default | Extra |+------------+------------------+-------------------+------+-----+-------------------+-------+| page_id | int(10) unsigned | NULL | NO | PRI | NULL | || created_dt | timestamp | NULL | NO | | CURRENT_TIMESTAMP | || creator_id | int(11) | NULL | NO | | NULL | || title | varchar(100) | latin1_swedish_ci | NO | | NULL | || contents | text | latin1_swedish_ci | NO | | NULL | || hit_count | int(11) | NULL | NO | | 0 | |+------------+------------------+-------------------+------+-----+-------------------+-------+6 rows in set (0.01 sec)
mysql> SHOW FULL COLUMNS FROM `pages`;+------------+------------------+-------------------+------+-----+-------------------+-------+| Field | Type | Collation | Null | Key | Default | Extra |+------------+------------------+-------------------+------+-----+-------------------+-------+| page_id | int(10) unsigned | NULL | NO | PRI | NULL | || created_dt | timestamp | NULL | NO | | CURRENT_TIMESTAMP | || creator_id | int(11) | NULL | NO | | NULL | || title | varchar(100) | latin1_swedish_ci | NO | | NULL | || contents | text | latin1_swedish_ci | NO | | NULL | || hit_count | int(11) | NULL | NO | | 0 | |+------------+------------------+-------------------+------+-----+-------------------+-------+6 rows in set (0.01 sec)
30
‣ Remember: an update on a table will invalidate ALL caches referring to that table.
‣ UPDATE pages SET hit_count = hit_count + 1;
‣ Thus: page table will NEVER be cached.
Tip 8: Sharding (2)
31
‣ Define hot data (volatile, changes often) and cold data (static, changes never or infrequently)
‣ move to different tables
‣ UPDATE page_stats SET hit_count = hit_count + 1;
‣ Query cache is happy again
Tip 8: Sharding (3)
32
‣ MySQL 5.1 and higher has got partitioning (but you should shard anyway).
Tip 8: Sharding (4)
33
Tip 9
‣ PK’S ARE ON EVERY INDEX
9) Don’t use a large primary key for InnoDB tables.
34
Tip 9: Large primary keys (1)
‣ InnoDB adds the primary key to EACH index.
‣ No primary key given? It uses an internal 6(!)-byte key.
35
Tip 10
‣ COUNT(*) => MYISAM = FAST‣ COUNT(*) => INNODB = SLOW
10) Don’t “SELECT COUNT(*) FROM TABLE” on InnoDB.
36
Tip 10: SELECT COUNT(*) (1)
‣ InnoDB implements MVCC (multi-version concurrency control).
‣ COUNT(*) must be counted and is not fetched from metadata.
37
‣ What do you want to COUNT(*)?
‣ Just for displaying purposes (there are X amount of pages): do you need the EXACT amount? (guess &| cache)
Tip 10: SELECT COUNT(*) (2)
38
Tip 11
‣ IT ISN’T THAT VARIABLE AS YOU MIGHT THINK
11) Don’t rely on the VARCHAR()
39
Tip 11: VARCHAR() (1)
‣ IT DOESN’T SOLVE YOUR PROBLEMS
40
‣ DON’T WORRY ABOUT THE UTF-8, I’LL BASH THAT LATER
Tip 11: VARCHAR() (3)
mysql> SHOW FULL COLUMNS FROM `varchartest`;+----------+--------------+-------------------+------+-----+---------+-------+| Field | Type | Collation | Null | Key | Default | Extra |+----------+--------------+-------------------+------+-----+---------+-------+| id | int(11) | NULL | NO | PRI | 0 | || name | varchar(255) | latin1_swedish_ci | NO | MUL | NULL | || utf8name | varchar(255) | utf8_general_ci | NO | MUL | NULL | |+----------+--------------+-------------------+------+-----+---------+-------+3 rows in set (0.01 sec)mysql> SHOW INDEXES FROM `varchartest`;+-------------+------------+--------------+--------------+-------------+-----------+-------------+| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality |+-------------+------------+--------------+--------------+-------------+-----------+-------------+| varchartest | 0 | PRIMARY | 1 | id | A | 3 || varchartest | 1 | idx_name | 1 | name | A | 3 || varchartest | 1 | idx_utf8name | 1 | utf8name | A | 3 |+-------------+------------+--------------+--------------+-------------+-----------+-------------+3 rows in set (0.00 sec)
mysql> SHOW FULL COLUMNS FROM `varchartest`;+----------+--------------+-------------------+------+-----+---------+-------+| Field | Type | Collation | Null | Key | Default | Extra |+----------+--------------+-------------------+------+-----+---------+-------+| id | int(11) | NULL | NO | PRI | 0 | || name | varchar(255) | latin1_swedish_ci | NO | MUL | NULL | || utf8name | varchar(255) | utf8_general_ci | NO | MUL | NULL | |+----------+--------------+-------------------+------+-----+---------+-------+3 rows in set (0.01 sec)mysql> SHOW INDEXES FROM `varchartest`;+-------------+------------+--------------+--------------+-------------+-----------+-------------+| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality |+-------------+------------+--------------+--------------+-------------+-----------+-------------+| varchartest | 0 | PRIMARY | 1 | id | A | 3 || varchartest | 1 | idx_name | 1 | name | A | 3 || varchartest | 1 | idx_utf8name | 1 | utf8name | A | 3 |+-------------+------------+--------------+--------------+-------------+-----------+-------------+3 rows in set (0.00 sec)
41
‣ DON’T WORRY ABOUT THE UTF-8, I’LL BASH THAT LATER
Tip 11: VARCHAR() (3)
mysql> EXPLAIN SELECT * FROM `varchartest` WHERE name LIKE 'jthijssen';+----+-------------+-------------+-------+---------------+----------+---------+------+------+-------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |+----+-------------+-------------+-------+---------------+----------+---------+------+------+-------------+| 1 | SIMPLE | varchartest | range | idx_name | idx_name | 257 | NULL | 1 | Using where |+----+-------------+-------------+-------+---------------+----------+---------+------+------+-------------+1 row in set (0.00 sec)
mysql> EXPLAIN SELECT * FROM `varchartest` WHERE name LIKE 'jthijssen';+----+-------------+-------------+-------+---------------+----------+---------+------+------+-------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |+----+-------------+-------------+-------+---------------+----------+---------+------+------+-------------+| 1 | SIMPLE | varchartest | range | idx_name | idx_name | 257 | NULL | 1 | Using where |+----+-------------+-------------+-------+---------------+----------+---------+------+------+-------------+1 row in set (0.00 sec)
42
Tip 12
‣ DON’T EXCHANGE ONE PROBLEM FOR ANOTHER
12) UTF-8 is not the enemy, but it certainly isn’t your friend.
43
Tip 12: UTF-8 (1)
‣ Dr Jeckyl and Mr Hyde
‣ Solves all your multi-language problems!
‣ But gives back performance issues.
44
Tip 12: UTF-8 (1)
mysql> SHOW FULL COLUMNS FROM `varchartest`;+----------+--------------+-------------------+------+-----+---------+-------+| Field | Type | Collation | Null | Key | Default | Extra |+----------+--------------+-------------------+------+-----+---------+-------+| id | int(11) | NULL | NO | PRI | 0 | || name | varchar(255) | latin1_swedish_ci | NO | MUL | NULL | || utf8name | varchar(255) | utf8_general_ci | NO | MUL | NULL | |+----------+--------------+-------------------+------+-----+---------+-------+3 rows in set (0.01 sec)mysql> SHOW INDEXES FROM `varchartest`;+-------------+------------+--------------+--------------+-------------+-----------+-------------+| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality |+-------------+------------+--------------+--------------+-------------+-----------+-------------+| varchartest | 0 | PRIMARY | 1 | id | A | 3 || varchartest | 1 | idx_name | 1 | name | A | 3 || varchartest | 1 | idx_utf8name | 1 | utf8name | A | 3 |+-------------+------------+--------------+--------------+-------------+-----------+-------------+3 rows in set (0.00 sec)
mysql> SHOW FULL COLUMNS FROM `varchartest`;+----------+--------------+-------------------+------+-----+---------+-------+| Field | Type | Collation | Null | Key | Default | Extra |+----------+--------------+-------------------+------+-----+---------+-------+| id | int(11) | NULL | NO | PRI | 0 | || name | varchar(255) | latin1_swedish_ci | NO | MUL | NULL | || utf8name | varchar(255) | utf8_general_ci | NO | MUL | NULL | |+----------+--------------+-------------------+------+-----+---------+-------+3 rows in set (0.01 sec)mysql> SHOW INDEXES FROM `varchartest`;+-------------+------------+--------------+--------------+-------------+-----------+-------------+| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality |+-------------+------------+--------------+--------------+-------------+-----------+-------------+| varchartest | 0 | PRIMARY | 1 | id | A | 3 || varchartest | 1 | idx_name | 1 | name | A | 3 || varchartest | 1 | idx_utf8name | 1 | utf8name | A | 3 |+-------------+------------+--------------+--------------+-------------+-----------+-------------+3 rows in set (0.00 sec)
45
Tip 12: UTF-8 (1)
mysql> EXPLAIN SELECT * FROM `varchartest` WHERE name LIKE 'jthijssen';+----+-------------+-------------+-------+---------------+----------+---------+------+------+-------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |+----+-------------+-------------+-------+---------------+----------+---------+------+------+-------------+| 1 | SIMPLE | varchartest | range | idx_name | idx_name | 257 | NULL | 1 | Using where |+----+-------------+-------------+-------+---------------+----------+---------+------+------+-------------+1 row in set (0.00 sec)
mysql> EXPLAIN SELECT * FROM `varchartest` WHERE name LIKE 'jthijssen';+----+-------------+-------------+-------+---------------+----------+---------+------+------+-------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |+----+-------------+-------------+-------+---------------+----------+---------+------+------+-------------+| 1 | SIMPLE | varchartest | range | idx_name | idx_name | 257 | NULL | 1 | Using where |+----+-------------+-------------+-------+---------------+----------+---------+------+------+-------------+1 row in set (0.00 sec)
mysql> EXPLAIN SELECT * FROM `varchartest` WHERE utf8name LIKE 'jthijssen';+----+-------------+-------------+-------+---------------+--------------+---------+------+------+-------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |+----+-------------+-------------+-------+---------------+--------------+---------+------+------+-------------+| 1 | SIMPLE | varchartest | range | idx_utf8name | idx_utf8name | 767 | NULL | 1 | Using where |+----+-------------+-------------+-------+---------------+--------------+---------+------+------+-------------+1 row in set (0.01 sec)
mysql> EXPLAIN SELECT * FROM `varchartest` WHERE utf8name LIKE 'jthijssen';+----+-------------+-------------+-------+---------------+--------------+---------+------+------+-------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |+----+-------------+-------------+-------+---------------+--------------+---------+------+------+-------------+| 1 | SIMPLE | varchartest | range | idx_utf8name | idx_utf8name | 767 | NULL | 1 | Using where |+----+-------------+-------------+-------+---------------+--------------+---------+------+------+-------------+1 row in set (0.01 sec)
46
ALL temporary buffers are allocated for worst-case scenario’s.
This means a varchar(255) in UTF-8 uses 255*3 + 2 = 767 bytes PER row, even if you have only 1 single char inside.
Tip 12: UTF-8 (1)
47
Tip 13
13) Know your cardinality & selectivity
48
Tip 13: Cardinality & Selectivity (1)
‣ Cardinality: the number of unique entries inside the index.
‣ Selectivity: percentage of unique entries.
‣ S(I) = cardinality / count * 100%
49
Tip 13: Cardinality & Selectivity (2)
50
‣ with 10 records: 5/10 * 100% = 50%
‣ with 1000 records: 75/1000 * 100 = 7.5%
‣ with 10.000 records: 200/10000 * 100% = 2%
Tip 13: Cardinality & Selectivity (3)
country_id (max +-200, but effectively +- 50, maybe less)
51
‣ A selectivity < 30% ? Full table scan!
‣ ANALYZE TABLE frequently.
Tip 13: Cardinality & Selectivity (4)
52
‣ Adding records changes your cardinality and thus selectivity.
‣ Develop against a “real” dataset.
Tip 13: Cardinality & Selectivity (5)
53
Tip 14
‣ NOW(), RAND(), UUID(), CONNECTION_ID() ETC..
14) Non-deterministic functions do not go well with query caching
54
Tip 14: Query caching (1)
mysql> show status like '%qcache%';+-------------------------+----------+| Variable_name | Value |+-------------------------+----------+| Qcache_free_blocks | 1 || Qcache_free_memory | 16768400 || Qcache_hits | 3860 || Qcache_inserts | 975 || Qcache_lowmem_prunes | 0 || Qcache_not_cached | 486 || Qcache_queries_in_cache | 0 || Qcache_total_blocks | 1 |+-------------------------+----------+8 rows in set (0.00 sec)
mysql> show status like '%qcache%';+-------------------------+----------+| Variable_name | Value |+-------------------------+----------+| Qcache_free_blocks | 1 || Qcache_free_memory | 16768400 || Qcache_hits | 3860 || Qcache_inserts | 975 || Qcache_lowmem_prunes | 0 || Qcache_not_cached | 486 || Qcache_queries_in_cache | 0 || Qcache_total_blocks | 1 |+-------------------------+----------+8 rows in set (0.00 sec)
mysql> select * from varchartest;+----+------+----------+| id | name | utf8name |+----+------+----------+| 1 | j | joshua || 2 | j | jeroen || 3 | d | david |+----+------+----------+3 rows in set (0.00 sec)mysql> show status like '%qcache%';+-------------------------+----------+| Variable_name | Value |+-------------------------+----------+| Qcache_free_blocks | 1 || Qcache_free_memory | 16766864 || Qcache_hits | 3860 || Qcache_inserts | 976 || Qcache_lowmem_prunes | 0 || Qcache_not_cached | 486 || Qcache_queries_in_cache | 1 || Qcache_total_blocks | 4 |+-------------------------+----------+8 rows in set (0.00 sec)
mysql> select * from varchartest;+----+------+----------+| id | name | utf8name |+----+------+----------+| 1 | j | joshua || 2 | j | jeroen || 3 | d | david |+----+------+----------+3 rows in set (0.00 sec)mysql> show status like '%qcache%';+-------------------------+----------+| Variable_name | Value |+-------------------------+----------+| Qcache_free_blocks | 1 || Qcache_free_memory | 16766864 || Qcache_hits | 3860 || Qcache_inserts | 976 || Qcache_lowmem_prunes | 0 || Qcache_not_cached | 486 || Qcache_queries_in_cache | 1 || Qcache_total_blocks | 4 |+-------------------------+----------+8 rows in set (0.00 sec)
55
Tip 14: Query caching (2)
mysql> select * from varchartest;+----+------+----------+| id | name | utf8name |+----+------+----------+| 1 | j | joshua || 2 | j | jeroen || 3 | d | david |+----+------+----------+3 rows in set (0.00 sec)mysql> show status like '%qcache%';+-------------------------+----------+| Variable_name | Value |+-------------------------+----------+| Qcache_free_blocks | 1 || Qcache_free_memory | 16766864 || Qcache_hits | 3861 || Qcache_inserts | 976 || Qcache_lowmem_prunes | 0 || Qcache_not_cached | 486 || Qcache_queries_in_cache | 1 || Qcache_total_blocks | 4 |+-------------------------+----------+8 rows in set (0.00 sec)
mysql> select * from varchartest;+----+------+----------+| id | name | utf8name |+----+------+----------+| 1 | j | joshua || 2 | j | jeroen || 3 | d | david |+----+------+----------+3 rows in set (0.00 sec)mysql> show status like '%qcache%';+-------------------------+----------+| Variable_name | Value |+-------------------------+----------+| Qcache_free_blocks | 1 || Qcache_free_memory | 16766864 || Qcache_hits | 3861 || Qcache_inserts | 976 || Qcache_lowmem_prunes | 0 || Qcache_not_cached | 486 || Qcache_queries_in_cache | 1 || Qcache_total_blocks | 4 |+-------------------------+----------+8 rows in set (0.00 sec)
mysql> select * from varchartest;+----+------+----------+| id | name | utf8name |+----+------+----------+| 1 | j | joshua || 2 | j | jeroen || 3 | d | david |+----+------+----------+3 rows in set (0.00 sec)mysql> show status like '%qcache%';+-------------------------+----------+| Variable_name | Value |+-------------------------+----------+| Qcache_free_blocks | 1 || Qcache_free_memory | 16766864 || Qcache_hits | 3860 || Qcache_inserts | 976 || Qcache_lowmem_prunes | 0 || Qcache_not_cached | 486 || Qcache_queries_in_cache | 1 || Qcache_total_blocks | 4 |+-------------------------+----------+8 rows in set (0.00 sec)
mysql> select * from varchartest;+----+------+----------+| id | name | utf8name |+----+------+----------+| 1 | j | joshua || 2 | j | jeroen || 3 | d | david |+----+------+----------+3 rows in set (0.00 sec)mysql> show status like '%qcache%';+-------------------------+----------+| Variable_name | Value |+-------------------------+----------+| Qcache_free_blocks | 1 || Qcache_free_memory | 16766864 || Qcache_hits | 3860 || Qcache_inserts | 976 || Qcache_lowmem_prunes | 0 || Qcache_not_cached | 486 || Qcache_queries_in_cache | 1 || Qcache_total_blocks | 4 |+-------------------------+----------+8 rows in set (0.00 sec)
56
Tip 14: Query caching (3)
mysql> select * from varchartest;+----+------+----------+| id | name | utf8name |+----+------+----------+| 1 | j | joshua || 2 | j | jeroen || 3 | d | david |+----+------+----------+3 rows in set (0.00 sec)mysql> show status like '%qcache%';+-------------------------+----------+| Variable_name | Value |+-------------------------+----------+| Qcache_free_blocks | 1 || Qcache_free_memory | 16766864 || Qcache_hits | 3861 || Qcache_inserts | 976 || Qcache_lowmem_prunes | 0 || Qcache_not_cached | 486 || Qcache_queries_in_cache | 1 || Qcache_total_blocks | 4 |+-------------------------+----------+8 rows in set (0.00 sec)
mysql> select * from varchartest;+----+------+----------+| id | name | utf8name |+----+------+----------+| 1 | j | joshua || 2 | j | jeroen || 3 | d | david |+----+------+----------+3 rows in set (0.00 sec)mysql> show status like '%qcache%';+-------------------------+----------+| Variable_name | Value |+-------------------------+----------+| Qcache_free_blocks | 1 || Qcache_free_memory | 16766864 || Qcache_hits | 3861 || Qcache_inserts | 976 || Qcache_lowmem_prunes | 0 || Qcache_not_cached | 486 || Qcache_queries_in_cache | 1 || Qcache_total_blocks | 4 |+-------------------------+----------+8 rows in set (0.00 sec)
mysql> select * from varchartest ORDER BY RAND();+----+------+----------+| id | name | utf8name |+----+------+----------+| 2 | j | jeroen || 1 | j | joshua || 3 | d | david |+----+------+----------+3 rows in set (0.05 sec)mysql> show status like '%qcache%';+-------------------------+----------+| Variable_name | Value |+-------------------------+----------+| Qcache_free_blocks | 1 || Qcache_free_memory | 16766864 || Qcache_hits | 3861 || Qcache_inserts | 976 || Qcache_lowmem_prunes | 0 || Qcache_not_cached | 487 || Qcache_queries_in_cache | 1 || Qcache_total_blocks | 4 |+-------------------------+----------+8 rows in set (0.00 sec)
mysql> select * from varchartest ORDER BY RAND();+----+------+----------+| id | name | utf8name |+----+------+----------+| 2 | j | jeroen || 1 | j | joshua || 3 | d | david |+----+------+----------+3 rows in set (0.05 sec)mysql> show status like '%qcache%';+-------------------------+----------+| Variable_name | Value |+-------------------------+----------+| Qcache_free_blocks | 1 || Qcache_free_memory | 16766864 || Qcache_hits | 3861 || Qcache_inserts | 976 || Qcache_lowmem_prunes | 0 || Qcache_not_cached | 487 || Qcache_queries_in_cache | 1 || Qcache_total_blocks | 4 |+-------------------------+----------+8 rows in set (0.00 sec)
57
SELECT * FROM table WHERE YEAR(created_dt) < YEAR(NOW());
vs
SELECT * FROM table WHERE YEAR(created_dt) < ‘2010’;
Tip 14: Query caching (4)
58
Tip 15
‣ AND GET SOME NICE TITLES WHILE YOU’RE AT IT...
15) Certify yourself as a DBA and/or DBE.
59
Tip 15: Certify yourself (1)
‣ THEY ARE NOT VERY EASY EXAMS, BUT WELL WORTH IT
‣ Oracle Certified MySQL Associate
‣ Oracle Certified Professional MySQL 5.0 Developer
‣ Oracle Certified Professional MySQL 5.0 Database Administrator
‣ Oracle Certified Expert, MySQL 5.1 Cluster Database Administrator
‣ “Old”, but still, get them all!
60
Let’s summarize
(1) Know how to use explain.
(2) Know the most basic my.cnf settings.
(3) Backup on table level.
(4) Don’t use “SELECT *” when you only need 1 or 2 fields.
(5) Use triggers and stored procedures.
(6) Don’t use FULLTEXT searches.
(7) Wildcard searches (%item%) are bad for performance.
(8) Shard your volatile and non-volatile data.
(9) Don’t use a large Primary Key for InnoDB tables.Don’t “Select COUNT(*)” on InnoDB.
(10) Don’t rely on the VARCHAR().
(11) UTF-8 is not the enemy, but it certainly isn’t your friend.Know your cardinality & selectivity.Non-deterministic functions do not go well with query caching. Certify yourself as a DBA and/or DBE.
Any questions?
∂ QUESTIONS?
http://farm1.static.flickr.com/73/163450213_18478d3aa6_d.jpg
Daycamp for developers
http://www.enrise.comhttp://www.daycamp4developers.com/
63
‣ THANK YOU FOR YOUR ATTENTION‣ Please rate my talk: http://joind.in/3662