practical my sql performance optimization

Practical MySQL Practical MySQL Performance Performance OptimizationOptimization

Marian HackMan MarinovMarian HackMan Marinov<[email protected]><[email protected]>

Chief System ArchitectChief System ArchitectSiteGroundSiteGround

Who am I?Who am I? Who am I?Who am I?

❖ Chief System Architect of Siteground.com

❖ Sysadmin since 1996

❖ Organizer of OpenFest, BG Perl Workshops,

LUG-BG and similar :)

❖ Teaching Network Security and Linux System

Administration at Sofia University

❖ MySQL 5.7

❖ Or its equivalent from MariaDB or Percona

Requirements... Requirements...

❖ Why am I making this presentation?

❖ Where is the information coming from?

MySQL Optimization Documentation

❖ I have skipped the SQL examples as they made the presentation too big ☹

DISCLAMERDISCLAMER

https://dev.mysql.com/doc/refman/5.7/en/optimization.html

❖ Manual monitoring

❖ Using Monyog

MonitoringMonitoring

How MySQL uses indexesHow MySQL uses indexes

❖ What type of indexes are there?



B-Tree

Hash

FULLTEXT

Spatial


❖ Which is used for what ?

hash

used for = or <=>

can not be used in ORDER BY

only whole keys can be used for search




hash




b-tree

can be used for =, >, >=, <, <=, or BETWEEN comparisons

can be used with LIKE, only if it doesn't start with %

can not be used in the form

column1 LIKE column2

can limit the size of the index (prefix)


❖ When is index NOT used



when the index does not cover all the columns in the where clause




when the types don't match: integer used for string index(column)





when != is used





when != is used

when you have a multicolumn index but the column you are referencing is not the first one and you don't have a separate index



when you have a Btree index but you use distinct on the column




when you have a single column indexes but you are actually referencing multiple columns in your WHERE/JOIN





when LIKE is used with % in the begining e.g. LIKE '%name'





when LIKE is used with % in the begining e.g. LIKE '%name'

order of columns in indexes MATTERS


1410864 rows

1 row in set (10.46 sec)



Index hintsIndex hints

❖ IGNORE INDEX


❖ IGNORE INDEX USE INDEX❖


❖ IGNORE INDEX

❖ USE INDEX

❖ FORCE INDEX


❖ "Waiting for query cache lock"

Abusing query cacheAbusing query cache


❖ SELECT *,NOW() FROM TABLE




❖ SQL_NO_CACHE




❖ SQL_NO_CACHE

❖ SET SESSION query_cache_type = OFF;




❖ SQL_NO_CACHE

❖ SET SESSION query_cache_type = OFF;

❖ Аs of MySQL 5.6.8 query cache is now disabled by default


❖ Large result sets hammer the network...


❖ Large result sets hammer the network...

❖ For caching large results move the cache, closer to the application by caching the results in ProxySQL


❖ InooDB uses only a single index per query

Optimizing InnoDB queriesOptimizing InnoDB queries


❖ If an indexed column cannot contain any NULL values, declare it as NOT NULL



❖ If an indexed column cannot contain any NULL values, declare it as NOT NULL

❖ START TRANSACTION READ ONLY


❖ use ANALYZE TABLE or myisamchk --analyze

Optimizing MyISAM queriesOptimizing MyISAM queries


❖ myisamchk --sort-index --sort-records=1




❖ avoid complex SELECT queries on MyISAM tables that are updated frequently





❖ MyISAM supports concurrent inserts






concurrent_insert = 0 (disabled)

concurrent_insert = 1 (auto)

concurrent_insert = 1 (always, even with deleted rows)






❖ try to avoid VARCHAR, BLOB, and TEXT in MyISAM tables







❖ Don't split large(columns) tables







❖ Don't split large(columns) tables

❖ use DELAY_KEY_WRITE=1


mysql> SELECT @@optimizer_switch\G

*************************** 1. row ***************************

@@optimizer_switch: index_merge=on,index_merge_union=on,

index_merge_sort_union=on,

index_merge_intersection=on,

engine_condition_pushdown=on,

index_condition_pushdown=on,

mrr=on,mrr_cost_based=on,

block_nested_loop=on,batched_key_access=off,

materialization=on,semijoin=on,loosescan=on,

firstmatch=on,duplicateweedout=on,

subquery_materialization_cost_based=on,

use_index_extensions=on,

condition_fanout_filter=on,derived_merge=on

Controlling the OptimizerControlling the Optimizer

MySQL 5.7only

❖ Merging multiple index scans on a SINGLE table

index_merge=off (default on)

index_merge_union=on

index_merge_sort_union=on

index_merge_intersection=on

low selectivity indexes are very slow with intersect


❖ Selecting different merge algorithms per table (only for 5.7)

Batched Key Access join /*+ BKA(t1) */

Block Nested-Loop join /*+ BNL(t1, t2) */


❖ Optimizing subqueries



Materialization strategy




Exists strategy




Exists strategy

(usually preferred when the sub queries can produce NULL results)




Exists strategy

(usually preferred when the sub queries can produce NULL results)

subquery_materialization_cost_based


❖ use SQL_SMALL_RESUL

❖ If all columns in the index are numeric MySQL will read only the INDEX

❖ Avoid large amounts of values in IN statements

values in an IN() list count as a predicates combined with OR

use eq_range_index_dive_limit to control the index dives in range comparisons(IN) 5.7

range optimization can't be used with NOT IN()

/*+ NO_RANGE_OPTIMIZATION(t4 PRIMARY) */

❖ Index Condition Pushdown 5.7

General optimizationGeneral optimization

❖ Avoid full table scans

--max-seeks-for-key=1000

SSD or NVMe

General optimizationGeneral optimization

# Turn tracing on (it's off by default):

SET optimizer_trace="enabled=on";

SELECT ...; # your query here

SELECT * FROM INFORMATION_SCHEMA.OPTIMIZER_TRACE;

# possibly more queries...

# When done with tracing, disable it:

SET optimizer_trace="enabled=off";

Using the tracerUsing the tracer

"filesort_summary": {

"rows": 100,

"examined_rows": 100,

"number_of_tmp_files": 0,

"sort_buffer_size": 25192,

"sort_mode": "<sort_key, packed_additional_fields>"

}

Using the tracerUsing the tracer

❖ Consider using INSERT with multiple VALUES lists

❖ Updates on tables with many indexes may become slow

❖ Updates with selects in them may block because of query cache lock contention

❖ Deletes also suffer from the above problems

Changing dataChanging data

❖ Splitting less frequently used columns from large(column) tables using foreign keys

Optimizing architectureOptimizing architecture

ACID RainACID RainConcurrency-Related Attacks onConcurrency-Related Attacks on

Database-Backed Web ApplicationsDatabase-Backed Web Applications

ACID RainACID RainConcurrency-Related Attacks onConcurrency-Related Attacks on

Database-Backed Web ApplicationsDatabase-Backed Web Applications

Marian HackMan Marinov<[email protected]>

http://www.bailis.org/papers/acidrain-sigmod2017.pdf

http://www.bailis.org/papers/acidrain-sigmod2017.pdf

THANK YOU THANK YOU THANK YOU THANK YOU

Marian HackMan Marinov<[email protected]>

practical my sql performance optimization

Engineering