mariadb for dbas · pdf filemariadb for dbas installation introduction architecture...
TRANSCRIPT
MariaDB for DBAsMariaDB Training
MariaDB for DBAs
Introduction
IntroductionArchitecture
Installation
Client Utilities
Configuration
Resource Usage
Storage Engines
Table Schema
Views
Partitioning
User & Server Security
Log Files
Monitoring & Troubleshooting
Back-Ups & Restoration
Customizing
MariaDB Replication
High Availability
Optimization
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Introducing MariaDB AbFounders from MySQL the
Company and Community Funded by Founders, Employees,
and Venture Capital Over 100 Employees Several former MySQL Employees
and Community Members in over 14 Countries
3
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Personal IntroductionsInstructor
Name and Background
Participants Name and Company MariaDB Experience How You Use MariaDB Needs Related to Course Topics
4
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Class Schedule & Personal ConcernsStarting and Ending Times Planned Breaks On- Site
Location of Rest Rooms Smoking Areas Snacks and Drinks
LVC Classes Chat with Everyone
5
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Course OutlineOverview & Server
Architecture Installation Client Utilities Configuration Resource Usage
Schema Choices Storage Engines Table Schema Views Partitioning
Core Admin. Tasks User & Server Security Logs Files Monitoring &
Troubleshooting Back-Ups & Restoration
Advanced Topics Customizing MariaDB Replication High Availability Optimization
6
MariaDB for DBAs
Architecture
Introduction
ArchitectureInstallation
Client Utilities
Configuration
Resource Usage
Storage Engines
Table Schema
Views
Partitioning
User & Server Security
Log Files
Monitoring & Troubleshooting
Back-Ups & Restoration
Customizing
MariaDB Replication
High Availability
Optimization
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
MariaDB Terrain
mysql
application (e.g., php)
mysqladmin
mysqld
Archive
InnoDB
MyISAM
cron
DBA
web user
socket file
tcp/ip
socket file
ssh & tcp/ip
Memory
SQL Tier — ClientsStorage Engine
TierServer Tier
Handler Interface
8
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Client ConnectionsTCP/IP Connections available on All Platforms
--skip-networking (disables)
Socket Files available on Unix Platforms - fastest choice Named Pipe and Shared Memory available on Windows
--enable-named-pipe, --shared-memory
MariaDB Connections require little resources and easy to open. Most use External Connection Pools (not needed usually) Set Global Client Connection Limit (max_connections=n)
9
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Client Libraries and the Like
Information on MariaDB Connectors: https://mariadb.com/products/connectors-plugins
Various Libraries for connecting to MariaDB — Most are Wrappers for MariaDB C API, which uses MariaDB Network Protocol
PHP API, Perl DBI, Python, Ruby MariaDB Connector for Java Native C/C++ driver MariaDB ODBC Drive Embedded MariaDB (libmysqld)
10
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
MariaDB Query Process
Thread Cache
Archive
InnoDB
Aria
Memory
Storage Engines
Query Cache
SQL Parser
Optimizer
Clients
11
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Thread Cache
Thread Cache
Query Cache
SQL Parser
Optimizer
Storage Engines
Thread is Assigned to Each Connection
Threads may be Reused from the Thread Cache or Created
12
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Thread Pool
Thread Pool
Query Cache
SQL Parser
Optimizer
Storage Engines
Main Thread Listens for Client Connections User Authentication based on Host, User, and
Password Client Buffers for Session Variables and Network
Communications
13
thread_handling = pool-of-threads thread_pool_size = 3
Excerpt from /etc/my.cnf.d/server.cnf
Documentation on Thread Pool: https://mariadb.com/kb/en/mariadb/thread-pool-in-mariadb/#using-thread-pool-scheduler
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Query Cache
Thread Cache
Query Cache
SQL Parser
Optimizer
Storage Engines
Documentation on Query Cache: https://mariadb.com/kb/en/mariadb/query-cache/
SHOW VARIABLES LIKE 'query_cache_type';
Possible to Stipulate in a Query
Stores SELECT Query Result Sets
Useful for High Read, Low Write Servers
Cache Purged when Related Data Changed
SELECT SQL_CACHE * FROM ...
SELECT SQL_NO_CACHE * FROM ...
14
No Longer Recommended
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
SQL Parser
Thread Cache
Query Cache
SQL Parser
Optimizer
Storage Engines
Converts SQL Text to Binary Format Parses SQL into Tokens (i.e., keyword, table, field, value) Applies Grammar Rules to check Validity (Lexical
Scanner, Grammar Rules Module) Construct a Parse Tree to be passed to Optimizer
15
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Optimizer
Thread Cache
Query Cache
SQL Parser
Optimizer
Storage Engines
The Optimizer Reads the Parse Tree and Determines an Execution Plan
Locate any Related Indexes Compare Efficiency of Index Access to Table Scan Determine JOIN order of Table
Eliminate Unnecessary Tables and WHERE clauses
Find Indexes that can handle GROUP BY and ORDER BY
16
MariaDB for DBAs
Installation
Introduction
Architecture
Installation
Client Utilities
Configuration
Resource Usage
Storage Engines
Table Schema
Views
Partitioning
User & Server Security
Log Files
Monitoring & Troubleshooting
Back-Ups & Restoration
Customizing
MariaDB Replication
High Availability
Optimization
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Installation Methods & PackagesInstallation Tools (MariaDB, Oracle, Percona)
Built and Tested by Experts Installer (e.g., yum) Use Up-to-Date Versions
Distribution Packages (rpm, deb, pkg, dmg)
Easy and Quick to Install Need to Configure Manually May be Old Versions
Build from Source — Useful for Custom Patches
Required for Non-Standard Platforms
18
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Installing with Binary Files
Download MariaDB: https://downloads.mariadb.org/mariadb/ Repository Configuration Tool: https://downloads.mariadb.org/mariadb/repositories/
Package Managers rpm -Ivh yum install dpkg -i apt-get install
Configuration File (datadir, etc.)
Execute manually mysql_install_db
InnoDB Log File Sizes in my.cnf
Service Scripts (i.e., /etc/init.d/mysql start|stop|restart)
19
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Basics Steps to Install Tar FilesDownload and Extract Tar Zip File Configuration File (/etc/my.cnf or c:\my.ini) Add System User, mysql
Data Directory (owner, mysql)
Install the System Tables (scripts/mysql_install_db)
Install the mysql.server script (e.g., /etc/init.d)
MariaDB binaries in $PATH
Start Server Run mysql_secure_installation
20
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Installing with Source FilesSystem Build Environment Install a Compiler (e.g., gcc), Libraries, Headers, etc. Build Scripts for various Platforms (see ./build/) Unix — Normal Configure and make Process (./configure --help)
Windows — get Visual C++ 6.0 or Higher Configure System to use Same Process as Binary
21
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Upgrade PlanConsideration
Review Change Logs (major and minor version upgrades) Major Upgrade is Safest — with Backup and Reload
MySQL 5.1 to 5.5, 5.6; MySQL 5.5 to MariaDB 10.1; MySQL 5.5 to MariaDB 5.5
Minor Upgrade (mysql_upgrade) Required Downtime
Preparation Back-Up Data Update Applications and Libraries Use a Test Server Use Replication — Upgrade Slave, Promote It, then Upgrade Master
22
MariaDB for DBAs
Client Utilities
Introduction
Architecture
Installation
Client UtilitiesConfiguration
Resource Usage
Storage Engines
Table Schema
Views
Partitioning
User & Server Security
Log Files
Monitoring & Troubleshooting
Back-Ups & Restoration
Customizing
MariaDB Replication
High Availability
Optimization
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
mysql Client — Command Line Tool (CLI)
shell> mysql --user root -p
mysql> SHOW DATABASES;
Shell or ssh
Execute Manually any SQL statement Display mysqld settings and status counters
Use as Interactive Session, or Pipe SQL via Shell
24
mysql -p -u user_name --execute "SHOW DATABASES"
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
mysqladmin — Command Line Tool (CLI)
Documentation on mysqladmin: https://mariadb.com/kb/en/mariadb/mysqladmin/
mysqladmin processlist mysqladmin -u root -p create employees mysqladmin extended-status
Shell or ssh
Manage User Accounts, Passwords, Permissions Display mysqld settings and status counters View and Kill Active Connections
Ping or Shutdown mysqld Create and Drop Databases
25
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Other Client Utilitiesmysqldump mysql_upgrade
my_print_defaults mysqlslap
sysbench
mysqlcheck
Percona Toolkit
26
Documentation on mysqldump: https://mariadb.com/kb/en/mariadb/mysqldump/ Documentation on mysql_upgrade: https://mariadb.com/kb/en/mariadb/mysql_upgrade/ Documentation on my_print_defaults: https://mariadb.com/kb/en/mariadb/my_print_defaults/ Documentation on mysqlslap: https://mariadb.com/kb/en/mariadb/mysqlslap/ Documentation on mysqlslap: https://mariadb.com/kb/en/mariadb/mysqlcheck/ Link to Percona Toolkit: https://www.percona.com/software/database-tools/percona-toolkit
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Other Command-Line Tools
* Caution: These MyISAM Tools Directly Access the Files
mysql.server
mysqld_safe
mysql_fix_privilege_tables
mysql_install_db
mysql_tz_to_sql
myisampack *
mysqlbinlog
mysqldumpslow
myisamchk *
pt-query-digest
innochecksum
27
Documentation on mysql.server: https://mariadb.com/kb/en/mariadb/mysqlserver/ Documentation on mysqld_safe: https://mariadb.com/kb/en/mariadb/mysqld_safe/ Documentation on mysql_install_db: https://mariadb.com/kb/en/mariadb/mysql_install_db/ Documentation on mysqldumpslow: https://mariadb.com/kb/en/mariadb/mysqldumpslow/ Documentation on pt-query-digest: https://www.percona.com/doc/percona-toolkit/2.2/pt-query-digest.html
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
HeidiSQL
Site for HeidiSQL: http://www.heidisql.com/
GUI Application (MS Windows)
Connect via SSL or SSH tunnel
Schema and Query Construction Optimization and Profiling User Account Management Task and Backup Scheduling Data Migration and Cleaning Schema Sync and Diff Tools Notifications
28
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
SQLyog
Site for SQLyog: https://www.webyog.com/product/sqlyog
Graphical Interface for Executing Queries
Schema Changes Make Back-Ups Bulk Import Data
29
MariaDB for DBAs
Configuration
Introduction
Architecture
Installation
Client Utilities
ConfigurationResource Usage
Storage Engines
Table Schema
Views
Partitioning
User & Server Security
Log Files
Monitoring & Troubleshooting
Back-Ups & Restoration
Customizing
MariaDB Replication
High Availability
Optimization
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Configuration File LocationsLinux Systems /etc/my.cnf
/etc/my.cnf.d/*.cnf
--defaults-extra-file
~/.my.cnf (clients)
Windows c:\Windows\my.ini
c:\my.ini
%PROGRAMDATA%\MySQL\MySQL Server x.x\my.cnf
%WINDIR%\my.cnf
INSTALLDIR\my.cnf
--defaults-extra-file
31
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Server Defaults
[mysqld] datadir=/data/mysql socket=/var/lib/mysql/mysql.sock user=mysql symbolic-links=0
[mysqld_safe] log-error=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.pid
!includedir /etc/my.cnf.d
Configuration File (my.cnf):
mysqld --print-defaults
mysqld would have been started with the following arguments:
--datadir=/data/mysql --socket=/var/lib/mysql/mysql.sock --user=mysql --symbolic-links=0
Finding Defaults:
32
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Global Variables
SET GLOBAL tmp_table_size = 32*1024*1024;
Can be Changed with SET Statement
Set in Configuration File at Start
Requires SUPER privilege for mysql.plugin table
SHOW GLOBAL VARIABLES;
Global Variables are System Wide
33
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Global StatusSHOW STATUS provides Counters and
Thresholds of Internal Statistics
Check Status Variables to Decide on Adjustments to Global Variables
SHOW STATUS LIKE '%innodb_buffer_pool%';
34
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Session Variables
SHOW SESSION VARIABLES LIKE '%character_set_client%';
+----------------------+-------+ | Variable_name | Value | +----------------------+-------+ | character_set_client | greek |
+----------------------+-------+
SHOW GLOBAL VARIABLES LIKE '%character_set_client%';
+----------------------+--------+ | Variable_name | Value | +----------------------+--------+ | character_set_client | latin1 | +----------------------+--------+
Session Variables are Values for Current Connection Global Variable Values is Default for Session
35
MariaDB for DBAs
Resource Usage
Introduction
Architecture
Installation
Client Utilities
Configuration
Resource Usage
Storage Engines
Table Schema
Views
Partitioning
User & Server Security
Log Files
Monitoring & Troubleshooting
Back-Ups & Restoration
Customizing
MariaDB Replication
High Availability
Optimization
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Global Memory
37
Global Memory Buffers are Allocated once when mysqld Starts table_open_cache table_definition_cache query_cache_size thread_cache
Permissions Tables Global Memory Buffers are also Allocated for Storage Engines
MyISAM: key_buffer_size InnoDB: innodb_buffer_pool_size, innodb_additional_mem_pool_size, innodb_log_buffer_size
Storage Engines may Allocate internally Other Memory
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Session Memory
38
Buffers Allocated as required for each Client Connection Some Allocated more than once for Joins and Sorting Released when Query is Finished or Client Session Closed Persistent Connections need to be Reset Client Session Memory Variables:
thread_stack read_buffer_size join_buffer_size bulk_insert_buffer_size
binlog_cache_size net_buffer_length
read_rnd_buffer_size max_heap_table_size max_allowed_packet sort_buffer_size
tmp_table_size myisam_sort_buffer_size
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Estimating Memory Usage
39
(sum(session_buffers) * max_connections)
+ sum(global_buffers) = total memory usage
Worst Case, Maximum Memory Usage Scenario:
Chances of All Clients Allocating All Possible Session Buffers at once is usually Small
Normal Practice is to Commit Over Memory Requirements
Out-of-Memory (OOM) may occur if mysqld is near System Memory Limits
Add Memory or Use a Crash-Safe Storage Engine
Use SWAP space, but Avoid letting MySQL Swap for Normal Load
Basic Formula for Estimating Memory Usage
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Temporary Disk Space
40
MariaDB Uses Temporary Disk Space Large Implicit Temporary Tables for Resolving Queries Some Sort Operations (ORDER BY, GROUP BY)
tmpdir defaults to system temporary location Some Unix platforms mount /tmp as tmpfs
slave_load_tmpdir for Replicating LOAD DATA INFILE Defaults to Location of tmpdir — Never use tmpfs Slave Exports from binlog to here, then LOAD DATA
INFILE
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Disk Speed
41
Disk Speed is important when a Data Set won’t Fit in Memory
Disk Reads are Alleviated by Caching Some Storage Engines Implement their own
Caches Some Rely on OS Disk Caching Mechanism
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Disk Redundancy — Preparing for Disk Failure
42
RAID, SAN or other Redundant, Distributed Storage Solutions
Good Disk Failure Protection
MariaDB Replication Slave
Because Asynchronous, will lose some data (see sync_binlog)
DRBD Slower Writes
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
File Handles
43
Logs and Replication Files need to be Open Client Connections each need a File (socket = file_handle)
Storage Engines may have Specific Needs: MyISAM uses a file per table.frm, and multiple for
table.MYI and table.MYD InnoDB needs only a few for Global Table-Space and
Logs (unless using innodb_file_per_table)
Implicit Disk-Based Temporary Tables are Files SELECT INTO OUTFILE uses a file
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Disk Data Directory
44
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Disk Data Directory
45
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Thread Cache
46
SHOW GLOBAL STATUS LIKE 'threads%';
+-------------------+-------+
| Variable_name | Value |
+-------------------+-------+
| Threads_cached | 0 |
| Threads_connected | 3 |
| Threads_created | 604 |
| Threads_running | 1 |
+-------------------+-------+
Use the SHOW STATUS for Thread Cache information:
SHOW VARIABLES LIKE 'thread_cache%';
+-------------------+-------+
| Variable_name | Value |
+-------------------+-------+
| thread_cache_size | 0 |
+-------------------+-------+
Use the SHOW VARIABLES for Thread Cache Variables:
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Server CPU
47
Since mysqld is multi-threaded, it can use many CPU cores
A Client Connection is a single thread — a Query uses only One Core
Query Concurrency is very Schema and Traffic Dependent
MariaDB for DBAs
Storage Engines
Introduction
Architecture
Installation
Client Utilities
Configuration
Resource Usage
Storage EnginesTable Schema
Views
Partitioning
User & Server Security
Log Files
Monitoring & Troubleshooting
Back-Ups & Restoration
Customizing
MariaDB Replication
High Availability
Optimization
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Storage Engine Basics
49
CREATE TABLE table1 (col1 INT) ENGINE = AriaDB;
ALTER TABLE table1 ENGINE = MyISAM;
Multiple Storage Engines Allowed They Determine Storage Medium (disk,
memory, etc.) Control Atomicity Consistency
Isolation Durability (ACID) Lock at certain Levels (table, page, row)
Some offer Special Features (Foreign Keys, GIS, ColumnStore)
Some Provide Optimization Storage Engine is Set for Table:
SHOW ENGINES;
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Pluggable Storage Engines
[mysqld] plugin_dir=/usr/lib/mysql/plugin/ plugin_dir=/usr/lib64/mysql/plugin/
UNINSTALL PLUGIN BLACKHOLE;
Excerpt from my.cnf or my.ini configuration file
Requires INSERT privilege for mysql.plugin table
Documentation on INSTALL PLUGIN: https://mariadb.com/kb/en/mariadb/install-plugin/
INSTALL PLUGIN BLACKHOLE SONAME 'ha_blackhole.so';
There are Storage Engines with different Features and Purposes
Multiple Engines on a Server and within a Query are Permitted
Engines can be Installed and Removed
50
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Advantages & Disadvantages of InnoDB
51
Read Uncommitted Read Committed Repeatable Read (no fantom rows) Serializable
Fully Transactional,
ACID Compliant
Data & Indexes Cached by mysqld
in Buffer Pool
Row-Level Locking for High-
Concurrency
Supports Four Isolation Levels
Supports Foreign Keys and Multi-
Version Concurrency
Control
Reliable Crash Recovery
Slower than MyISAM, but
Higher Concurrency
Advantages Disadvantages
Isolation Levels
Documentation for InnoDB & XtraDB: https://mariadb.com/kb/en/mariadb/xtradb-and-innodb/
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
InnoDB Architecture
52
Additional Memory Pool
Buffer Pool *
Data Files * **
Redo Log (File 1/2)
Redo Log (File 2/2)
Memory
Disk
* Cached Data & Indexes, Stored in Pages ** Transaction Data Writes *** Data, Index, and Undo Log Files
INSERT, UPDATE, DELETE
1
COMMIT
CHECKPOINT
Log Buffer * *
2
3
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
InnoDB Clustered Index
53
root
node node
leafleaf leaf
PAGE (16KB) PAGE (16KB) PAGE (16KB)
B-Tree
HEADER PRIMARY KEY ROW DATA
leaf
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
InnoDB Secondary Index
54
root
node node
leafleaf leaf
PAGE (16KB) PAGE (16KB) PAGE (16KB)
B-Tree
HEADER VALUES OF INDEXED COLUMNS PRIMARY KEY
leaf
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Pros & Cons of MyISAM
55
Documentation for InnoDB & XtraDB: https://mariadb.com/kb/en/mariadb/xtradb-and-innodb/
Indexes are Cached by mysqld in key_buffer Don't Disable — System Tables use MyISAM Storage Engine (InnoDB in Future)
Not Transactional
Table-Level Locking
Disadvantages
Poor Crash Recovery
(Relies on OS to Flush Data to
Disk)
Data Caching Relies on OS Disk Cache
High Read, Low Write
Traffic
Concurrent Writes
Advantages
Many Column Data Types
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Aria
56
Replacement for MyISAM Buffers Pages before Writing Rows
Index & Pages Cached (aria_pagecache_buffer_size)
Crash Safe — Statement Commit & Rollback (notes a log file)
Used for Temporary Tables Variable aria_used_for_temp_tables should be ON
Documentation for Aria: https://mariadb.com/kb/en/mariadb/aria-storage-engine/
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Memory
57
Temporary, but Fast Table Structure on Disk Data and Indexes in RAM and the Like Uses Table-Level Locking
Data won’t survive mysqld Restart — Schema Remains
Allows Only Fixed-Length Records — Variable-Length Columns Padded to Full Width
Some Data Types are Not Supported (e.g., VARCHAR(255), TEXT, BLOB)
Uses a Hash Index Type — Only Equality Comparisons (e.g., No LIKE Operator)
Documentation for Memory: https://mariadb.com/kb/en/mariadb/memory-storage-engine/
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Blackhole — /dev/null
58
Useful for Development Servers for Testing only Query Execution
Can be Used for Replication since Queries are written to Binary Log
Master can Replicate Data without Storing Locally Relay Slave can use Filters Rules Use on Slave to Prevent Storing Locally Replicated Data
Useful for Solving Performance Problems Remove Temporarily and Instantly Excessive Log Table
Activity Benchmark to determine Non-Storage-Engine Bottlenecks
Documentation for Blackhole: https://mariadb.com/kb/en/mariadb/blackhole/
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Archive
59
Useful for Storing Large Amounts of Archived Data or Logging Tables with Frequent Writes and Infrequent reads
Minimal Memory Footprint All Data is Compressed on Disk with zlib Supports only INSERT and SELECT statements Doesn’t support DELETE, UPDATE or REPLACE Indexes Are Not Stored No Data Caching, Except the OS Disk Cache Non-Transactional
Documentation for Archive: https://mariadb.com/kb/en/mariadb/archive/
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Spider
60
Partitions Large Tables across Multiple Servers Transparent — No Changes Necessary to Client Applications Allows for more Parallelism, Scale-out.
Parallel Partition Scanning — Make MariaDB Faster than Other RDBMS
ACID Compliant & Transactional Not Yet Production Ready Schema Creation & Changes Cumbersome
Documentation on Spider: https://mariadb.com/kb/en/mariadb/spider/
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
ColumnStore (Columnar)
61
Massively Parallel, Distributed Data Engine Powerful Analytics Data Warehouse Fast Import (ETL)
cpimport
Columnar Architecture No Indexes Each Column is Stored Separately
WHERE Clauses Parsed Differently
Documentation on ColumnStore: https://mariadb.com/kb/en/mariadb/mariadb-columnstore/
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Connect
62
Maps Diverse Data to Tables Flat Files including CSV Tables in External ODBC Databases
JOIN Mapped Data to Database Tables
Generated tables (PIVOT, etc.) Plug-in API for Customized Mapping
Documentation on Connect: https://mariadb.com/kb/en/mariadb/introduction-to-the-connect-engine/
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Other Storage Engines
63
EXAMPLE — Example Storage Engine CSV — Comma-Separated Value Storage Engine FEDERATED — Federated Storage Engine
Documentation on EXAMPLE: https://mariadb.com/kb/en/mariadb/example-storage-engine/ Documentation on CSV: https://mariadb.com/kb/en/mariadb/csv-overview/ Documentation on FEDERATED: https://mariadb.com/kb/en/mariadb/federated-storage-engine/
MariaDB for DBAs
Table Schema & Maintenance
Introduction
Architecture
Installation
Client Utilities
Configuration
Resource Usage
Storage Engines
Table SchemaViews
Partitioning
User & Server Security
Log Files
Monitoring & Troubleshooting
Back-Ups & Restoration
Customizing
MariaDB Replication
High Availability
Optimization
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Creating Databases & Tables
65
Documentation for CREATE DATABASE: https://mariadb.com/kb/en/mariadb/create-database/ Documentation for CREATE TABLE: https://mariadb.com/kb/en/mariadb/create-table/
CREATE TABLE table1
(col1 INT AUTO_INCREMENT KEY, col2 INT DEFAULT '100', col3 VARCHAR(255) CHARACTER SET 'greek', col4 DATETIME) ENGINE = Aria;
Creating Databases is Simple CREATE DATABASE db1;
Creating Tables can be Simple or Complex
List Column Names, Data Types, Default Values, Character Sets and Collation, etc.
Define Indexes and Specify Storage Engine
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Altering Tables
66
Documentation for ALTER TABLE: https://mariadb.com/kb/en/mariadb/alter-table/
ALTER TABLE table1
ADD COLUMN col5 CHAR(8), DROP COLUMN col3, CHANGE COLUMN col4 col4 DATE;
ALTER TABLE is used to change a Table's Schema ADD COLUMN to Add a Column DROP COLUMN to Drop a Column — Deletes Data CHANGE COLUMN and MODIFY COLUMN to Alter a Column
Basic Syntax Example for ALTER TABLE Statement
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
InnoDB & MyISAM Row Formats
67
Fixed-Length Data Types Less Data File Fragmentation Heaviest Disk Space Usage
Static
Fixed and Variable Length Data Types More Data File Fragmentation More Efficient Disk Space Usage
Dynamic
Decompressed as Needed Minimal Disk Space Usage Read-Only
Compressed
INT, CHAR, etc.
VARCHAR, TEXT, etc.
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Setting SQL Mode
68
List of SQL Modes: https://mariadb.com/kb/en/mariadb/sql_mode/
SET sql_mode='TRADITIONAL';
SHOW VARIABLES LIKE 'sql_mode' \G
Variable_name: sql_mode Value: STRICT_TRANS_TABLES,
STRICT_ALL_TABLES, NO_ZERO_IN_DATE, NO_ZERO_DATE, ERROR_FOR_DIVISION_BY_ZERO,
TRADITIONAL, NO_AUTO_CREATE_USER, NO_ENGINE_SUBSTITUTION
Determines the SQL Server Environment
Supported SQL Syntax Type of Data Validation
SQL Mode is Initially Empty Use SET Statement to set
SQL Mode
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Column Attributes
69
Documentation on Default Column Values: https://mariadb.com/kb/en/mariadb/data-types/
CREATE TABLE people (name VARCHAR(20) DEFAULT 'unknown');
Default Values can be Specified in the CREATE TABLE Statement
Columns can be NULL, unless defined NOT NULL NULL means "No Value", "Not Applicable", or
"Unknown" Use NULL when Value is Not an Empty String or 0 NOT NULL Reduces Storage in some Engines
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Data Types for Integers
70
Documentation on Integer Data Types: https://mariadb.com/kb/en/mariadb/data-types/
Use UNSIGNED when Appropriate
INT(n) Specifies Display Precision, Not Storage Precision
Size and Precision is Storage Engine Dependent Define Handling of Out-of-Range Values with sql_mode
Default Mode: Values are Truncated Silently Strict Mode: Errors are Generated
TINYINT
SMALLINT
MEDIUMINT
INTEGER,INT
BIGINT
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
AUTO_INCREMENT
71
Documentation on AUTO_INCREMENT: https://mariadb.com/kb/en/mariadb/auto_increment/
SELECT MAX(auto_inc_field) FROM table;
Use LAST_INSERT_ID()to get Value Generated for Client Connection
SERIAL is a synonym for "BIGINT UNSIGNED NOT NULL AUTO_INCREMENT UNIQUE"
MyISAM the Counter can be Set Back Manually or If Counter Value Wraps
InnoDB prepares AUTO_INCREMENT counters when mysqld starts
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Floating and Fixed Point
72
Documentation on FLOAT Data Type: https://mariadb.com/kb/en/mariadb/float/ Documentation on DECIMAL Data Type: https://mariadb.com/kb/en/mariadb/decimal/ Documentation on DOUBLE Data Type: https://mariadb.com/kb/en/mariadb/double/
FLOAT and DOUBLE are Approximate Types - uses 4 and 8 bytes IEEE storage format
DECIMAL(m,d) - maximum total number of digits, number of decimal points
An Exact Value type, up to 65 digits precision, 4 bytes storage for each multiple of nine digits
NUMERIC is a synonym for DECIMAL
REAL is a synonym for DOUBLE - Unless in REAL_AS_FLOAT SQL mode
FLOAT
DOUBLE
DECIMAL
NUMERIC
REAL
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
String Data Types
73
Documentation on String Data Types: https://mariadb.com/kb/en/mariadb/string-data-types/
All String Data Types have a Character Set CHAR(n) — number of characters, not bytes, wide VARCHAR(n)— Changes to CHAR in Implicit Temporary Tables and mysqld internal buffers
TEXT —Not Supported by the MEMORY Storage Engine; Implicit Temporary Tables may Convert to MyISAM
CHAR
VARCHAR
TINYTEXT
TEXT
MEDIUMTEXT
LONGTEXT
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Character Set & Collation
74
Documentation on Character Sets and Collation: https://mariadb.com/kb/en/mariadb/data-types-character-sets-and-collations/
SELECT * FROM table1 ORDER BY col1 COLLATE latin1_german2_ci;
Character Set may be Global or for Schema, Table or Column
Multi-Byte Character Sets Increase Disk Storage and Working Memory Requirements (e.g, UTF-8 Requires 3 or 4 bytes per Character)
Collations affect String Comparison (Character Order) Collations can be Changed for Query:
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Binary Data Types
75
Documentation on BINARY Data Type: https://mariadb.com/kb/en/mariadb/binary/ Documentation on VARBINARY Data Type: https://mariadb.com/kb/en/varbinary/ Documentation on BLOB Data Type: https://mariadb.com/kb/en/blob/
BINARY and VARBINARY are Case-Sensitive Versions of CHAR and VARCHAR
No Character Set and Collation for Binary Types - ordered by bytes
Blobs are Used often to Store Files in a Database Files on Disk are often Faster Blobs are Included in Transactions, Replication, and
Backups Blobs Inflate mysqld memory usage
BINARY
VARBINARY
TINYBLOB
BLOB
MEDIUMBLOB
LONGBLOB
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Date & Time Data Types
76
DATE — from 1000-01-01 to 9999-12-31 (YYYY-MM-DD)
TIME — from -838:59:59 to 838:59:59
DATETIME — Same Range with Time (YYYY-MM-DD HH:mm:ss) TIMESTAMP — Unix timestamp, in seconds from
1970-01-01 Many Apps Store UNIX_TIMESTAMP() values in unsigned integer
field
YEAR — Accepts YY or YYYY
DATE
TIME
DATETIME
TIMESTAMP
YEAR
Documentation on Date and Time Data Types: https://mariadb.com/kb/en/mariadb/date-and-time-data-types/Documentation on Microseconds in MariaDB: https://mariadb.com/kb/en/mariadb/microseconds-in-mariadb/
SELECT CURTIME(4); +---------------+ | CURTIME(4) |
+---------------+ | 05:33:09.1061 | +---------------+
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Manipulating Date & Time
77
Documentation on Date and Time Functions: https://mariadb.com/kb/en/mariadb/date-and-time-functions/
ADDDATE()
ADDTIME()
CONVERT_TZ()
CURDATE()
CURTIME()
DATE()
DATE_ADD()
DATE_FORMAT()
DATE_SUB()
DATEDIFF()
DAYNAME()
DAYOFMONTH()
DAYOFWEEK()
DAYOFYEAR()
EXTRACT()
FROM_DAYS()
FROM_UNIXTIME()
GET_FORMAT()
HOUR()
LAST_DAY()
MAKEDATE()
MAKETIME()
MICROSECOND()
MINUTE()
MONTH()
MONTHNAME()
NOW()
PERIOD_ADD()
PERIOD_DIFF()
QUARTER()
SEC_TO_TIME()
SECOND()
STR_TO_DATE()
SUBDATE()
SUBTIME()
SYSDATE()
TIME()
TIME_FORMAT()
TIME_TO_SEC()
TIMEDIFF()
TIMESTAMP()
TIMESTAMPADD()
TIMESTAMPDIFF()
TO_DAYS()
UNIX_TIMESTAMP()
UTC_DATE()
UTC_TIME()
UTC_TIMESTAMP()
WEEK()
WEEKDAY()
WEEKOFYEAR()
YEAR()
YEARWEEK()
Functions for Date and Time Manipulation
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Listing Acceptable Values
78
Documentation on ENUM: https://mariadb.com/kb/en/mariadb/enum/ Documentation on SET: https://mariadb.com/kb/en/mariadb/set-data-type/
CREATE TABLE colors (primary_colors ENUM('red', 'yellow', 'blue'));
ENUM is an Enumerated List of String Values - uses a 2-byte integer index
CREATE TABLE colors
(primary_colors SET('red','yellow','blue'));
INSERT INTO colors VALUES('red'), ('red,blue');
Simple Example for an ENUM Column
Simple Example for a SET Column
SET is a Specified List of String Values — Can Hold Multiple Specified Values
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Dynamic Columns
79
SELECT item_name, COLUMN_GET(item_attributes, 'color' AS CHAR) AS color FROM clothes;
+-----------------+-------+ | item_name | color | +-----------------+-------+
| MariaDB T-shirt | blue | +-----------------+-------+
COLUMN_GET()
COLUMN_ADD()
COLUMN_LIST()
COLUMN_DELETE()
Documentation on Dynamic Columns: https://mariadb.com/kb/en/dynamic-columns/ Tutorial on Dynamic Columns: http://radar.oreilly.com/2015/04/dynamic-columns-in-mariadb.html
CREATE TABLE clothes
(item_name VARCHAR(32) PRIMARY KEY,
item_attributes BLOB);
INSERT INTO clothes
VALUES ('MariaDB T-shirt',
COLUMN_CREATE('color', 'blue', 'size', 'XL'));
Stores Key/Value Pairs within a BLOB Column with COLUMN_CREATE()
Used to Store many Attributes which are Unknown in Advance
Manipulated with Special Dynamic Column Functions:
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Virtual Columns
80
Documentation on Virtual Columns: https://mariadb.com/kb/en/virtual-columns/
CREATE TABLE sales (sales_id INT AUTO_INCREMENT KEY, qty INT,
price DECIMAL(6,2), total DECIMAL(6,2) AS (qty * price) VIRTUAL);
Two Virtual Column Types: PERSISTENT (stored) VIRTUAL (generated only)
All Data Types Supported Use PERSISTENT for Indexes - Cannot
be Primary Key
Used with InnoDB, Aria, MyISAM, CONNECT
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Table Maintenance SQL Statements
81
ANALYZE TABLE — Check and Store Index Cardinal Values
Run Periodically OPTIMIZE TABLE — Defragment MyISAM Data
Files and Sort Indexes Maps Simply to a Full Table Rebuild
for InnoDB CHECK TABLE — Check Table for Errors
Arguments: QUICK, FAST, CHANGED, MEDIUM, EXTENDED
REPAIR TABLE — Fix a Corrupted Table Arguments: QUICK, EXTENDED, USE_FRM
CHECKSUM TABLE
Arguments: QUICK, EXTENDED
Documentation on CHECK TABLE: https://mariadb.com/kb/en/mariadb/sql-commands-check-table/ Documentation on CHECKSUM TABLE: https://mariadb.com/kb/en/checksum-table/ Documentation on REPAIR TABLE: https://mariadb.com/kb/en/repair-table/ Documentation on ANALYZE TABLE: https://mariadb.com/kb/en/analyze-table/ Documentation on OPTIMIZE TABLE: https://mariadb.com/kb/en/optimize-table/
MariaDB for DBAs
Views
Introduction
Architecture
Installation
Client Utilities
Configuration
Resource Usage
Storage Engines
Table Schema
Views
Partitioning
User & Server Security
Log Files
Monitoring & Troubleshooting
Back-Ups & Restoration
Customizing
MariaDB Replication
High Availability
Optimization
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Purpose of Views
83
Adapt and Standardize Table Schema for Applications or Across Application Upgrades
Restricting Visible Table Data to Specific Users and Applications
Useful to Simplify, Split, or Factor Complex Reporting Queries
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
View Example & Caveats
84
CREATE VIEW emp_names AS
SELECT emp_id, name_first, name_last
FROM employees;
SELECT * FROM view_example LIMIT 5;
+--------+------------+-----------+
| emp_no | name_first | name_last |
+--------+------------+-----------+
| 10021 | Ramzi | Erde |
| 10087 | Xinglin | Eugenio |
| 10122 | Ohad | Esposito |
| 10131 | Magdalena | Eldridge |
| 10145 | Akemi | Esposito |
+--------+------------+-----------+
An SQL Statement Represented as a Table
Views are Not Materialized The SELECT Query always Re-
Executes
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Creating and Changing Views
85
Documentation on CREATE VIEW: https://mariadb.com/kb/en/create-view/
CREATE VIEW view_loans AS SELECT id, loan_amount, (loan_months / 12) AS loan_years FROM loan_table;
CREATE VIEW Statement Creates a View based on a SELECT statement
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Creating and Changing Views
86
ALTER VIEW view_loans AS
SELECT ...;
ALTER VIEW Statement Deletes View and Creates New One
DROP VIEW view_loans;
DROP VIEW Statement Deletes a View
Documentation on ALTER VIEW: https://mariadb.com/kb/en/mariadb/alter-view/ Documentation on DROP VIEW: https://mariadb.com/kb/en/mariadb/drop-view/
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Showing Views
87
SHOW CREATE VIEW view_loans \G
Use SHOW TABLES to List Views
SHOW FULL TABLES WHERE Table_type = 'VIEW';
Use SHOW CREATE to Inspect Schema of a View
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
View Security
88
CREATE VIEW view_loans ...
SQL SECURITY = INVOKER;
Views may be Writable — INSERT, UPDATE and DELETE Affect Underlying Table
Conditions of Non-Writable View ALGORITHM Set to TEMPTABLE UNION, HAVING, some Joins, and Sub-Queries Data Aggregation (AVG, GROUP BY, DISTINCT) Derived Fields — Field using a Function or Calculation
Updatable Views WITH CHECK OPTION Constrains Writes to Rows
in View
Set Security Based on View Creator (DEFINER) or User (INVOKER)
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Setting the Algorithm
89
Documentation on View Algorithms: https://mariadb.com/kb/en/mariadb/view-algorithms/
CREATE VIEW view_loans ALGORITHM = MERGE AS SELECT ...
View Algorithm Controls Query Execution Method
TEMPTABLE View Data put in Temporary Table, then Query Executed Possibly Slower since No Index for Temporary Table
MERGE Query and Original View SELECT are Merged Possibly Faster since Underlying Table Index Used
UNDEFINED The Optimizer choose what’s Best (Default)
MariaDB for DBAs
Partitioning
Introduction
Architecture
Installation
Client Utilities
Configuration
Resource Usage
Storage Engines
Table Schema
Views
PartitioningUser & Server Security
Log Files
Monitoring & Troubleshooting
Back-Ups & Restoration
Customizing
MariaDB Replication
High Availability
Optimization
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Value of Partitions
91
Documentation for Partitions: https://mariadb.com/kb/en/mariadb/create-table/#partitions
table1.frm table1.par
table1#P#p0.MYD
table1#P#p0.MYI
table1#P#p1.MYD
table1#P#p1.MYI
table1#P#p2.MYD
table1#P#p2.MYI
table1#P#p3.MYD
table1#P#p3.MYI
Partitioned Table *
* Files Generated for a MyISAM Table with Four Partitions
For Tables Larger than Memory Easier Maintenance with a Single
Partition Instead of Entire Table (Less Data, Fewer Locks, Faster Operations)
Partition Pruning — Better Performance for Queries Involving only Specific Partitions
Bulk Management of Data — Archiving & Fast Elimination of Data
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Factors in Partitioning
92
Custom Partitioning Function is defined for a Table to return a Partitioning Key
Key Determines which Partitions Rows are Stored Key must be Integer or NULL Stored Procedures, Stored Functions, UDFs, Plug-ins, User
Variables & Sub-Queries Not Allowed in Function
A Column may be Used Directly as Key without a Function Maximum of 1024 Partitions per Table Foreign Keys are Disabled in an InnoDB Table FULLTEXT and Spatial Data Types are Disabled in MyISAM
CONNECT Storage Engine Doesn't Support Partitioning
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
RANGE Partitions
93
MAXVALUE indicates the Maximum Value Possible
Documentation for Partitions: https://mariadb.com/kb/en/mariadb/create-table/#partitions
Partition p0
Distribution of Rows
(100, 'Bob Smith', 4),(102, 'Wendy Jones', 6),(105, 'Sarah Jane', 4),(109, 'Bill Johnson', 7)
(101, 'Tim Schmidt', 16),(103, 'John Wells', 12),(107, 'Cindy James', 18)
(104, 'Lena Denison', 25),(106, 'Tom Smith', 20),(108, 'Elaine Wilson', 30),(110, 'Susan Hill, 120)
Partition p1
Partition p2
employees tableCREATE TABLE employees ( id INT, name VARCHAR(20), dept_id INT NOT NULL) ENGINE = MyISAM PARTITION BY RANGE(dept_id) (
PARTITION p0 VALUES LESS THAN (10), PARTITION p1 VALUES LESS THAN (20), PARTITION p2 VALUES LESS THAN MAXVALUE);
Partition by a Range of Column Values
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
RANGE Columns
94
Integer, Date, and String Data Types Allowed for Partition Keys
TEXT, BLOB, DECIMAL, and FLOAT Not Allowed
Partition p0
Distribution of Rows
(40, 'Bob Smith', 4),(85, 'Wendy Joans', 25),(99, 'Sarah Jane', 30),(120, 'Bill Johnson', 8)
(125, 'Tim Schmidt', 16),(100, 'John Wells', 15),(150, 'Cindy James', 40)
(210, 'Lena Denison', 25),(200, 'Tom Smith', 50)
Partition p1
Partition p2
employees tableCREATE TABLE employees ( id INT AUTO_INCREMENT KEY, name VARCHAR(20), dept_id INT) ENGINE = MyISAM PARTITION BY RANGE COLUMNS(id, dept_id) (
PARTITION p0 VALUES LESS THAN (100,10), PARTITION p1 VALUES LESS THAN (200,20), PARTITION p2 VALUES LESS THAN (MAXVALUE,MAXVALUE));
Partition by Column Tuples within a Range Tuples are like Sets of Values for Columns
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
LIST Partitions
95
Values Not Listed are Rejected No MAXVALUE
Partitionactive_clients
Distribution of Rows
(110, 'Harold Wilson', 1),(116, 'Leslie Haynes', 1),(120, 'Mike Brown', 1),(126, 'Tim Simms', 1)
(115, 'Jim Berns', 0),(120, 'Ed Gibson', 0),(122, 'Sally King', 0)
(106, 'Betty Mills', -1),(125, 'Joe Cohen', 2)
Partition inactive_clients
Error
clients tableCREATE TABLE clients (
client_id INT,
name VARCHAR(20), active INT NOT NULL)
ENGINE = MyISAM
PARTITION BY LIST(active) (
PARTITION active_clients VALUES IN (1),
PARTITION inactive_clients VALUES IN (0));
Partition by a List of Specific Column Values
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
LIST COLUMNS Partitions
96
Partition employees_east
Distribution of Rows
(110, 'Harold Wilson', 'Toykyo'),(116, 'Leslie Haynes', 'Tokyo'),(120, 'Mike Brown', 'Sydney'),(126, 'Tim Simms', 'Sydney')
(115, 'Jim Berns', 'London'),(120, 'Ed Gibson', 'New York'),(122, 'Sally King', 'New York)
(106, 'Betty Mills', 'Londonn'),(125, 'Joe Cohen', 'Chicago')
Partition employees_west
Error
employees table
CREATE TABLE employees (
id INT AUTO_INCREMENT KEY,
name VARCHAR(20),
office VARCHAR(20) NOT NULL)
ENGINE = MyISAM
PARTITION BY LIST COLUMNS(office)
(PARTITION employees_west
VALUES IN ('London', 'New York'),
PARTITION employees_east
VALUES IN ('Tokyo', 'Sydney'));
Partition by List of Specific String, Integer or Date Values
One or More Columns Allowed
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
HASH Partitions
97
Partitions contain Same Number of Rows Use COALESCE to Reduce Number of
Partitions
Distribution of Rowsemployees table
Partition p0
Partition p1
Partition p2
Partition p3
(100, 'Bob Smith', '2014-05-15'),(101, 'Tim Schmidt', '2014-05-20'),(102, 'Wendy Jones', 2014-06-01')
(103, 'John Wells', '2014-09-15'),(104, 'Lena Denison', '2014-12-01'),(105, 'Sarah Jane', '2015-01-15')
(106, 'Tom Smith', '2015-02-14'),(107, 'Cindy James', '2015-05-01'),(108, 'Elaine Wilson', '2015-09-20')
(109, 'Bill Johnson', '2016-02-24'),(110, 'Susan Hill, '2016-03-15'),(111, 'Rob Kane, '2016-08-03')
ALTER TABLE employees
COALESCE PARTITION 3;
CREATE TABLE employees (
id INT AUTO_INCREMENT KEY,
name VARCHAR(20), dept_id INT,
join_date DATE)
ENGINE = MyISAM
PARTITION BY HASH(id) PARTITIONS 4;
Partition Determined by a Hash from a Given Expression
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
KEY Partitions
98
Primary Key or Specified Columns Used for Partition Key
CREATE TABLE employees ( id INT AUTO_INCREMENT KEY, name VARCHAR(20), dept_id INT,
join_date DATE) ENGINE = MyISAM PARTITION BY KEY(dept_id, join_date) PARTITIONS 4;
Partition with a Hash, Using an Expression Generated Automatically
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Composite or Sub-Partitioning
99
CREATE TABLE patients ( id INT,
name VARCHAR(20), birth_date DATE) ENGINE = MyISAM PARTITION BY RANGE( YEAR(birth_date) ) SUBPARTITION BY HASH( QUARTER(birth_date) ) SUBPARTITIONS 4 ( PARTITION elderly VALUES LESS THAN (1950),
PARTITION middle_age VALUES LESS THAN (1975), PARTITION young VALUES LESS THAN MAXVALUE );
patients table
elderly#p0
elderly#p1
elderly#p2
elderly#p3
middle_age#p0
middle_age#p1
middle_age#p2
middle_age#p3
young#p0
young#p1
young#p1
young#p3
Used to further Divide Partitions Each Partition Contains same Number of
Sub-Partitions Partitions and Sub-Partitions may use
different Methods
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Partition Specific Queries
100
Supports Queries and Data Manipulation Statements SELECT, DELETE, INSERT, REPLACE, UPDATE, LOAD DATA, and LOAD XML
SELECT COUNT(*) FROM employees
PARTITION(employees_west);
Partition may be Specified in SQL Statements
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Partition Maintenance
101
ALTER TABLE patients REBUILD PARTITION elderly;
Use ALTER TABLE to Perform Specific Partition Maintenance (ANALYZE, CHECK, REBUILD, REPAIR, and OPTIMIZE) Hard Disk Space is Reclaimed
REPAIR TABLE clients; +--------------+--------+----------+----------+
| Table | Op | Msg_type | Msg_text | +--------------+--------+----------+----------+ | test.clients | repair | status | OK | +--------------+--------+----------+----------+
Use Normal Full Table Maintenance Statements (ANALYZE, CHECK, OPTIMIZE, and REPAIR TABLE)
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Partition Changes
102
ALTER TABLE clients
ADD PARTITION (PARTITION suspended_clients VALUES IN (-1) );
Adding Partitions Provide Full Definition of New
Partition
ALTER TABLE clients DROP PARTITION inactive_clients;
Dropping Partitions Data Contained is Dropped
Partition is Lost
ALTER TABLE clients REMOVE PARTITIONING;
Removing Partitioning Data is Not Lost
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Partition Changes
103
Alterations are Allowed on RANGE, LIST Partitions
ALTER TABLE table1
REORGANIZE PARTITION partition_a, partition_b
INTO(PARTITION partition_ab
VALUES LESS THAN (1000));
ALTER TABLE table1
REORGANIZE PARTITION p0 INTO
(PARTITION n0 VALUES LESS THAN (1000),
PARTITION n1 VALUES LESS THAN (2000));
Splitting Partitions
Merging Partitions
ALTER TABLE employees_test
REORGANIZE PARTITION middle_age
INTO(PARTITION older VALUES LESS THAN (1975));
Renaming a Partition Use REORGANIZE PARTITION with
New Name, but Same Parameters
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Exchanging Partitions with Tables
104
Both Partition and Table need Same Structure No Foreign Keys in Table, Nor Other Referencing
Tables Data is Moved; Partition is Preserved
Use DROP PARTITION to Remove PartitionALTER TABLE clients DROP PARTITION inactive_clients;
ALTER TABLE clients EXCHANGE PARTITION inactive_clients
WITH TABLE clients_archive;
Move a Partition or Sub-Partition to an Unpartitioned Table
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Storage Engine Considerations
105
Storage Engine can be Specified for a Partitioned Table
ENGINE Clause precedes PARTITION Clause Partitions in Table must use Same Storage Engine MERGE, CSV and FEDERATED Don't Support
Partitioning DATA DIRECTORY and INDEX DIRECTORY Clauses Don't
Effect Partitioned InnoDB Tables
MariaDB for DBAs
User & Server Security
Introduction
Architecture
Installation
Client Utilities
Configuration
Resource Usage
Storage Engines
Table Schema
Views
Partitioning
User & Server SecurityLog Files
Monitoring & Troubleshooting
Back-Ups & Restoration
Customizing
MariaDB Replication
High Availability
Optimization
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Security — a Multi-Layered Challenge
107
Other Processes
mysqld Data Storage
Network Considerations Private Network Wifi Access SSL for Data In Motion DNS Controlled
ClientsServer
Port 3306
Server Considerations Data Location Non-Root Users Passwords Other Processes
Apps
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
MariaDB User Accounts
108
Documentation for CREATE USER: https://mariadb.com/kb/en/mariadb/create-user/
Privileges are Based on User and Host Combined
CREATE USER 'ted'@localhost
IDENTIFIED BY 'rover123';
Authentication Based on User, Host, and Password Empty String is User Wildcard Percent Sign (%) is a Host Wildcard
Host is an IP or Host Name — DNS, /etc/hosts localhost is used by Local Socket on Linux Systems
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Granting Privileges — Databases and Tables
109
Documentation for GRANT: https://mariadb.com/kb/it/grant/
GRANT SELECT, DELETE,
UPDATE(col2,col3) ON demo.tester TO 'bob'@'localhost';
Users given Permission with GRANT Statement
Privilege Levels — Global, Database, Table, Column, or Routine ON *.* — Access to All Databases and Tables
ON database_name.* — Access to Database Named
ON database_name.table_name — Access to Table Named
privilege(column) ON TABLE db_name.table_name — Column by Privilege
ON PROCEDURE database_name.routine_name — Access to Routine
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
More User SQL Statements
110
Documentation for RENAME USER: https://mariadb.com/kb/en/rename-user/ Documentation for SET PASSWORD: https://mariadb.com/kb/en/set-password/ Documentation for DROP USER: https://mariadb.com/kb/en/drop-user/
SET PASSWORD FOR 'bob'@'localhost' = PASSWORD('fido456');
User Names and Hosts can be Changed
RENAME USER 'robert'@'127.0.0.1' TO 'bob'@'localhost';
Users Passwords can be Changed
DROP USER 'bob'@'localhost';
Users can be Deleted
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Permissions Tables
111
Permissions Checked when Client Connects and Query Received
mysql Database contains Permissions or Grants Tables
— Restrict Access
Grants Tables Loaded into Memory at Start Possible to INSERT, UPDATE, and DELETE Manually Tables
in mysql FLUSH PRIVILEGES Required to Reload Manual Changes
mysqldump will Copy mysql Database by Default
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Privileges Available (SHOW PRIVILEGES)
112
Documentation on User Privileges: https://mariadb.com/kb/en/mariadb/grant/#privilege-levels
Basic Privileges
USAGE
SELECT
INSERT
UPDATE
DELETE
SHOW DATABASES
Developer Privileges
CREATE
ALTER
INDEX
DROP
Customizing Privileges
CREATE ROUTINE
ALTER ROUTINE
EXECUTE
EVENT
TRIGGER
Special Privileges
CREATE TEMPORARY TABLES
CREATE VIEW
SHOW VIEW
LOCK TABLES
Replication Privileges
REPLICATION CLIENT
REPLICATION SLAVE
Administrator Privileges
ALL [PRIVILEGES]
SUPER
CREATE USER
GRANT OPTION
PROCESS
FILE
RELOAD
SHUTDOWN
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Checking User Privileges
113
Documentation for SHOW GRANTS: https://mariadb.com/kb/en/show-grants/
SHOW GRANTS FOR 'bob'@'localhost'; SHOW GRANTS FOR 'bob'@'home';
Check Regularly user Table in mysql Database for User Accounts
Check for Obsolete Accounts Check for Blank Passwords
SELECT User, Host, Password FROM mysql.user;
Use the SHOW GRANTS statement for User-Host Account
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Revoking User Privileges
114
Documentation for REVOKE: https://mariadb.com/kb/en/revoke/
REVOKE DELETE,UPDATE ON db1.table1 FROM 'bob'@'localhost';
Use REVOKE Statement to Revoke Privileges — Need GRANT OPTION
List Privileges to Revoke or ALL PRIVILEGES REVOKE Doesn’t Remove a User-Host Account — Use DROP
USER
Check Results with SHOW GRANTS after Revoking
REVOKE ALL PRIVILEGES ON db1.table1 FROM 'bob'@'localhost';
GRANT SELECT ON database_name.table_name TO 'bob'@'localhost';
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Limiting Users
115
Documentation for User Account Resource Limits: http://dev.mysql.com/doc/refman/5.6/en/user-resources.html
GRANT SELECT ON db_name.table_name TO 'bob'@'home_bob' WITH MAX_QUERIES_PER_HOUR 20 MAX_CONNECTIONS_PER_HOUR 10
MAX_USER_CONNECTIONS 2 MAX_UPDATES_PER_HOUR 5;
User-Host Accounts may be Limited by Resources
Usage Related to Limits are kept in mysql Database Counters Reset when Server Starts or FLUSH PRIVILEGES Executed
User Counters may be Reset Specifically — Not MAX_USER_CONNECTIONS
FLUSH USER_RESOURCES;
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Pluggable Authentication Module
116
Documentation on PAM: https://mariadb.com/kb/en/mariadb/pam-authentication-plugin/
[mysqld] plugin-load=authentication_pam.so
Uses Plug-in to Authenticate Client Connections Allows Credentials from Other Sources — Linux Passwords,
LDAP, Kerberos, Active Directory
CREATE USER user IDENTIFIED WITH authentication_pam AS 'mysql, root=developer, users=data_entry';
Specify PAM when Creating Users
Excerpt from my.cnf or my.ini configuration file
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Proxy Users
117
GRANT PROXY ON 'manager'@'localhost' TO 'bob'@'localhost';
SELECT USER(), CURRENT_USER();
+---------------+-------------------+
| USER() | CURRENT_USER() |
+---------------+-------------------+
| bob@localhost | manager@localhost |
+---------------+-------------------+
MariaDB Perceives User as Proxied User
Documentation on Proxy Users: http://dev.mysql.com/doc/refman/5.5/en/proxy-users.html
REVOKE PROXY ON 'manager'@'localhost' FROM 'bob'@'localhost';
PROXY Permission give a User-Account Permission of Another Account
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
User Roles
118
Documentation on User Roles: https://mariadb.com/kb/en/roles-overview/
User Role is a Set of Privileges available for a Group An Alternative to Assigning Privileges Individually to
Each User Account Useful for Assigning Privileges to Many Users or to
External Users
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Creating a User Role
119
Documentation on CREATE ROLE: https://mariadb.com/kb/en/mariadb/create-role/
CREATE ROLE 'sales_staff';
GRANT SELECT, INSERT, UPDATE ON sales.* TO 'sales_staff';
Use CREATE ROLE to Create Group Privileges
Use GRANT to Grant Privileges to Role
No Host is given for a Role
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Listing & Showing Privileges of Roles
120
Query information_schema to Get List of Roles Created
Use SHOW GRANTS to Check Privileges for Role
SHOW GRANTS FOR 'admin_remote';
+------------------------------------------------------+
| Grants for admin_remote |
+------------------------------------------------------+
| GRANT USAGE ON *.* TO 'admin_remote' |
| GRANT INSERT, UPDATE ON `admin`.* TO 'admin_remote' |
+------------------------------------------------------+
No Host is Given for a Role
SELECT * FROM
information_schema.APPLICABLE_ROLES \G
********* 1. row ********
GRANTEE: root@localhost
ROLE_NAME: admin_remote
IS_GRANTABLE: YES
IS_DEFAULT: NO
There is No SHOW ROLES Statement
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Granting a Role to a User
121
Use GRANT to Designate Users for Role
GRANT 'sales_staff' TO 'bob'@localhost;
SELECT * FROM mysql.roles_mapping;
+-----------+-------+--------------+--------------+ | Host | User | Role | Admin_option | +-----------+-------+--------------+--------------+ | localhost | bob | admin_remote | N | | localhost | root | admin_remote | Y |
+-----------+-------+--------------+--------------+
Check mysql.roles_mapping for Roles Granted
Executed while Logged in as an Administrator
Executed while Logged in as an Administrator
GRANT 'admin_remote' TO 'bob'@localhost WITH ADMIN OPTION;
Use WITH ADMIN OPTION so Grantee may Grant Role
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Assuming and Relinquishing a Role
122
Documentation on CURRENT_ROLE( ): https://mariadb.com/kb/en/mariadb/current_role/ Documentation on SET ROLE: https://mariadb.com/kb/en/mariadb/set-role/
SET ROLE sales_staff;
User Assumes a Role for Session with SET ROLE SELECT CURRENT_ROLE();
+----------------+ | CURRENT_ROLE() | +----------------+
| sales_staff | +----------------+
Executed while Logged in as bob@localhost
Executed while Logged in as user bob@localhost
User Relinquishes a Role by Setting it to None — Or by Ending Session
SET ROLE NONE;
Executed while Logged in as bob@localhost
SET DEFAULT ROLE sales_staff;
User can Set Default Role to NONE or Preferred Role
Executed while Logged in as user bob@localhost
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Revoking a Role
123
Documentation on Revoking Roles: https://mariadb.com/kb/en/mariadb/revoke/#roles Documentation on DROP ROLE: https://mariadb.com/kb/en/mariadb/drop-role/
REVOKE 'admin_remote' FROM 'bob'@localhost;
Check mysql.roles_mapping for a List of Roles Granted
Use REVOKE to take Role Option from User
Executed while Logged in as an Administrator
Executed while Logged in as an Administrator
SELECT * FROM mysql.roles_mapping;
+-----------+---------+--------------+--------------+ | Host | User | Role | Admin_option | +-----------+---------+--------------+--------------+
| localhost | ted | admin_remote | N | | localhost | bob | admin_remote | Y | | localhost | root | admin_remote | Y | +-----------+---------+--------------+--------------+
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Dropping a Role
124
Documentation on Revoking Roles: https://mariadb.com/kb/en/mariadb/revoke/#roles Documentation on DROP ROLE: https://mariadb.com/kb/en/mariadb/drop-role/
SELECT User FROM mysql.user
WHERE is_role='Y';
DROP ROLE 'sales_staff';
Check mysql.user for List of Roles
Use DROP ROLE to take Eliminate a Role
Executed while Logged in as an Administrator
Executed while Logged in as an Administrator
MariaDB for DBAs
Log Files
Introduction
Architecture
Installation
Client Utilities
Configuration
Resource Usage
Storage Engines
Table Schema
Views
Partitioning
User & Server Security
Log Files
Monitoring & Troubleshooting
Back-Ups & Restoration
Customizing
MariaDB Replication
High Availability
Optimization
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
MariaDB Logs
126
Error Log
General Query Log
SQL Error Log
Slow Query Log
Binary Log
mysqld
Error Log General Query Log SQL Query Log Slow Query Log Binary Log
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Error Log
127
Set log-warnings to 2 for VerboseDocumentation on Error Log: https://mariadb.com/kb/en/mariadb/error-log/
Error Log
General Query Log
SQL Query Log
Slow Query Log
Binary Log
mysqld
[mysqld]
log-error = /path/file_name log-warnings = 1
Contain Startup, Shutdown, Error Messages
Unix uses stderr, Sent to host_name.err in datadir
Some Unix systems Redirect to System Log
Windows uses host_name.err in datadir, or System Event Log
Excerpt from my.cnf or my.ini configuration file
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
General Query Log
128
Documentation on General Query Log: https://mariadb.com/kb/en/mariadb/general-query-log/
Error Log
General Query Log
SQL Error Log
Slow Query Log
Binary Log
mysqld
Excerpt from my.cnf or my.ini configuration file
[mysqld] general-log
general_log_file='/path/host.log'
All Queries Received from All Clients Order Received — Not Execution Potential Problems with Log
Possible Disk I/O bottleneck on High-Traffic Servers
Log can Quickly become Huge Contains Queries in Plain Text — a
Security Risk
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
SQL Error Log
129
Documentation on SQL Query Log: https://mariadb.com/kb/en/mariadb/sql_error_log-plugin/
Error Log
Slow Query Log
Binary Log
mysqld SQL Error Log
General Query Log
Logs SQL Errors Error Messages SQL Statement
Part of MariaDB Audit Plugin Used for Detecting SQL Injections
Logs User, Host, & Time
INSTALL PLUGIN sql_error_log SONAME 'sql_errlog';
Requires INSERT Privilege on mysql.plugin Table
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Slow Query Log
130
Documentation on Slow Query Log: https://mariadb.com/kb/en/slow-query-log/
Error Log
General Query Log
SQL Error Log
Slow Query Log
Binary Log
mysqld
Relevant Entries from my.cnf or my.ini configuration file
[mysqld] slow_query_log = ON
slow_query_log_file = /path/file long_query_time = 0.5 log_queries_not_using_indexes log_slow_admin_statements
Queries which take more seconds to execute than long_query_time
Contains Queries in Plain Text Use mysqldumpslow to Read
Log
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Binary Log
131
Documentation on Binary Log: https://mariadb.com/kb/en/binary-log/
Error Log
General Query Log
Binary Log
Slow Query Log
SQL Error Logmysqld
[mysqld] log-bin
expire_logs_days = 7
Writes in the Order Executed Useful for Recovering Lost Data Used for Replication Move from to Separate Disk for
Security Set Expiration or Purge
Occasionally
Excerpt from my.cnf or my.ini configuration file
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Managing Log Files
132
Documentation on FLUSH: https://mariadb.com/kb/en/mariadb/flush/ Documentation on mysqladmin: https://mariadb.com/kb/en/mariadb/mysqladmin/
Rotated Regularly the Log Files Move manually the Existing Log Files
and then Flush Logs
FLUSH LOGS;
Back-Ups Include Logs in Back-ups Synchronize Binary Logs with Back-ups
Error Log Stop mysqld, Move the Log, then Restart mysqld
mysqladmin flush-logs
An Alternative Method from the Command-Line
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Managing Binary Logs
133
Documentation on PURGE BINARY LOGS: https://mariadb.com/kb/en/mariadb/sql-commands-purge-logs/
Inspect Current Binary Logs FLUSH LOGS triggers a Binary Log
Rotation Binary Log Files use Incrementally
Numbered Extensions Binary Log Files can be Purged
explicitly Binary Logs can be set to Expire in
Configuration File (expire_logs_days) Allow Time for all Slaves to Read the
Binary Log Files
PURGE BINARY LOGS BEFORE '2016-06-01 10:00:00';
SHOW BINARY LOGS;
PURGE BINARY LOGS TO 'mariadb-bin.000010';
Purges All Binary Logs Before One Named
Purges All Binary Logs Before One which Contains Entries for Date Given
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Audit Plugin Log
134
Documentation on Audit Plugin: https://mariadb.com/kb/en/mariadb/about-the-mariadb-audit-plugin/
Includes Table Event Logging (Triggers, Stored Procedure Calls) Optional Field Substitution of Placeholders in Query
Log to Improve Security Filtering Audit Logs by Role & User Accounts Records Privilege Changes Password Change Logging
MariaDB for DBAs
Monitoring & Troubleshooting
Introduction
Architecture
Installation
Client Utilities
Configuration
Resource Usage
Storage Engines
Table Schema
Views
Partitioning
User & Server Security
Log Files
Monitoring & Troubleshooting
Back-Ups & Restoration
Customizing
MariaDB Replication
High Availability
Optimization
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Command-Line Monitoring Tools
136
Site for mytop: http://jeremy.zawodny.com/mysql/mytop/ Site for innotop: https://code.google.com/p/innotop/
mytop -u user -p passwordmytop
A Clone of top for Monitoring MariaDB Threads
Works any Shell, SSH, or Shell Script
innotop
Tool for Monitoring MariaDB Threads
Includes InnoDB Specific Information
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
MonYog
137
Site for MonYog: https://www.webyog.com/product/monyog
User-Friendly, Cross Platform, GUI
No Server-Side Agent Required Manages Easily Multiple Servers Track and Alert many Metrics
Monitor Error Log and Security Vulnerabilities
Replication Lag and Excessive Resource Usage
Slow Queries and Deadlocks Detailed History and Trend
Analysis
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Automated Monitoring Utilities
138
Automated Monitoring can poll MariaDB and Send Alerts
CA-Unicenter Cacti Ganglia HP Openview Hyperic IBM Tivoli
Munin Nagios NewRelic OpenNMS PMM Zabbix
Some are Generic Network Monitors with Plug-ins for MariaDB
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Custom Monitoring Solutions
139
Custom Monitoring Solution or Method is sometimes Required
Check if mysqld is active with mysqladmin ping or C function, mysql_ping()
Look for Performance Hot Spots with SHOW GLOBAL STATUS
Parse the mysqld Error Log for “Error” or “Warning” of other Keywords
Use a Scheduler (e.g., cron) to run Monitor Tools — Email Results or Create SNMP Alerts
Kristian Koehntopp (http://vvv.koehntopp.de/rrd/)
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Error Log
140
Error Log is Stored by Default in Data Directory (datadir)
Error Log Contains Information about Errors from Queries
Shows Warnings - Depends on --log-warnings
Shows mysqld Stops, Starts, and Error Events Crashes will Appear as Abrupt Stop without Shutdown
Messages Check System Log for Information on Shutdown or
Crash
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
MariaDB Status
141
SHOW PROCESSLIST;
SHOW GLOBAL STATUS;
SHOW ENGINE INNODB STATUS \G
Documentation on SHOW PROCESSLIST: https://mariadb.com/kb/en/mariadb/show-processlist/ Documentation on SHOW STATUS: https://mariadb.com/kb/en/show-status/ Documentation on SHOW ENGINE INNODB STATUS: https://mariadb.com/kb/en/show-engine-innodb-status/
Check Global Status Unusually Large Values - Note Typical Results
Check Processes Long Running Queries - Lock & Load Problems
Intermittent or Short Term Problems - Try Multiple Snapshots or Polling
Check Status of Storage Engines Used Useful for InnoDB, though Returns Complex Results
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Server Status
142
See if Reached Limits of CPU, Disk, Memory, or Network Usage
Track System Statistics to Identify Trends or Patterns - Use a Monitoring Program
Watch for Unpredictable and Reoccurring Problems Check What is Running on Server — Check cron Check if Other Servers Affected, if more than One
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Application Status
143
Check if New Applications Installed Recently Code Changes or Updates Applied Recently — If it
Handles Deadlocks Check Logs for Applications
MariaDB Connection or Disconnection Problems Unexpected Error Codes from MariaDB
Check if Application Traffic and Dataset within Expected Bounds
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
MariaDB Support
144
Web Page for MariaDB Support: http://www.mariadb.com/products/mysql-support
24 x 7 On-Line, Live Assistance General Consulting Performance Tuning Code Review Login Support Bug and Hot fixes On-Site Health Check
MariaDB for DBAs
Back-Ups & Restoration
Introduction
Architecture
Installation
Client Utilities
Configuration
Resource Usage
Storage Engines
Table Schema
Views
Partitioning
User & Server Security
Log Files
Monitoring & Troubleshooting
Back-Ups & RestorationCustomizing
MariaDB Replication
High Availability
Optimization
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Backup Principles
146
High Availability Can’t Prevent User Errors Copy Data, Logs, and Configuration Files Daily Snapshots (mysqldump, InnoDB Hot Backup, LVM) and
Real-Time Backups (Replication)
Store Backups in Multiple Locations - Onsite for fast access; Offsite for security
Synchronize Binary Logs with Backup Files
Back-Up Methods Physical Back-Up Logical Back-Up Binary Back-Up
Test Backup and Recovery
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Making Physical Back-Ups
147
Produces a Binary Copy of Data Faster than Dumping to SQL Text File Recovered Only to the Same Storage Engine Not Useful for Migrations Corrupted Files are Undetected Multiple Options
Copy Manually Data Directory (Stopping mysqld is Necessary) LVM on Linux for Volume Snapshot (Data Complete and Consistent, but still Trigger InnoDB Recovery after
Restore) InnoDB Hot Backup tool, or XtraBackup
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Logical Backups
148
Documentation on mysqldump: https://mariadb.com/kb/en/mariadb/mysqldump/ Documentation on SELECT…INTO OUTFILE: https://mariadb.com/kb/en/mariadb/select-into-outfile/
Produce Text Files with SQL statements to Replay to Rebuild Database
Allows Backup of Databases or Specific Tables — Schema or only Data
An SQL dump is independent of Storage Engine, and can be Restored to a different Storage Engine, or Used for Migration
Process can be Slow and Requires Locks Use a Local Drive, not across Network Multiple Options (mysqldump, "SELECT INTO OUTFILE", etc.)
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Advantages & Factors of mysqldump
Documentation on mysqldump: https://mariadb.com/kb/en/mariadb/mysqldump/
mysqldump -u backup_user -p --all-databases > backup.sql mysqldump -u backup_user -p db1 table1 > table.sql mysqldump -u backup_user -p -h production | mysql -u dev_user -h development
Exports Schema and Data (SQL text file, tab-delimited, or XML) Good for Backups & Transferring Data between Servers Requires Locks for Consistency —May Affect Traffic Dump all Databases, or Specific Databases or Tables
Optimize for Importing (extended INSERT, temporarily disable indexes, and ignore foreign key checks)
149
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Create a Back-Up User
150
CREATE USER 'admin_backup'@localhost IDENTIFIED BY 'rover123';
GRANT SELECT, RELOAD, LOCK TABLES, REPLICATION CLIENT, SHOW VIEW, EVENT, TRIGGER ON *.* TO 'admin_backup'@'localhost';
Create a Special User for Making Back-ups with mysqldump
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Recovery with Binary Logs
151
Documentation on mysqlbinlog: https://mariadb.com/kb/en/mariadb/mysqlbinlog/
SHOW BINARY LOGS; SHOW MASTER STATUS;
Restore Databases from Dump file Use the Binary Logs to Execute
Remaining SQL Statements
mysqlbinlog binlog.000005 binlog.000006 | mysql -u admin_user -p
Use mysqlbinlog to Convert Binary Log and Pipe to mysql Client
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Point-in-Time Recovery
152
Documentation on Point-in-Time Recovery: http://dev.mysql.com/doc/refman/5.5/en/point-in-time-recovery.html
mysqlbinlog --stop-datetime='2017-02-10 11:59:59' \ /var/log/mysql/bin.000006 | mysql -u admin_user -p
Give a Stop Time to mysqlbinlog to Recover Until Then (e.g., Noon)
mysqlbinlog --start-datetime='2013-11-03 12:01:00' \ /var/log/mysql/bin.000006 | mysql -u admin_user -p
Give a Start Time to mysqlbinlog to Recover From Then
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Point-in-Time Recovery — Event Positions
153
mysqlbinlog --start-datetime='2013-11-03 11:55:00' \ --stop-datetime='2013-11-03 12:05:00' \ /var/log/mysql/bin.000001 > /tmp/mysql_restore.sql
Determine Event Position Numbers (log_pos)
mysqlbinlog --stop-position=720 /var/log/mysql/bin.0000006 \ | mysql -u admin_user -p
mysqlbinlog --start-position=828 /var/log/mysql/bin.000006 \
| mysql -u admin_user -p
Run mysqlbinlog on Binary Log until Stop Position Execute Binary Log again from a Start Position
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Replication as a Back-up Method
154
Use MariaDB Replication to copy Data to Slave (close to real time, but technically asynchronous)
Recovery can be Fast and Simple Switch Traffic to Slave, Restore Master, Switch Traffic
back to Master Alternatively, Replay Binary Logs with mysqlbinlog tool
Allows other Methods to be executed on Slave without increasing Load on Master
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
XtraBackup (Percona)
155
Documentation on XtraBackup: http://www.percona.com/doc/percona-xtrabackup Installing XtraBackup: http://www.percona.com/doc/percona-xtrabackup/2.1/installation.html
Fast Back-Up Process Transactions Processed during Back-Up Process Low Drain on System Resources Back-Up Files Verified Automatically Faster Restore Process
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Develop a Back-Up PlanTake Inventory of Databases Write a Back-Up Schedule Write a Verification Schedule
Check and Test Back-Ups Practice Restoring
156
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Take Inventory
157
List of Databases & Tables Number of Rows Frequency of Changes How Active Sensitivity of Data
Assessment of company DatabaseTables Rows Changing Active Sensitive
clients 1,159,913 ✓ ✓ ✓
clients_addresses 1,159,913 ✓ ✓ ✓clients_email 1,159,913 ✓ ✓ ✓clients_telephone 1,159,913 ✓ ✓ ✓employees 5,042 ✓ ✓employees_email 5,042 ✓employees_salaries 1,628 ✓ ✓employees_telephones 5,042 ✓org_departments 18
org_divisions 5
org_warehouses 5
ref_job_titles 240
ref_name_titles 9
ref_states 56
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Back-Up Schedule
158
Back-Up Tables Frequency Schedule Secure Off-Site
company — full back-up (company-yyyy-mmm-dd.sql)
all tables weekly Sunday, 1:00 p.m.
✓ ✓
company — client tables (company-clients-yyyy-mmm-dd.sql)
clients, clients_addresses, client_telephones
daily every day, 1:00 a.m.
✓
company — employee tables (company-employees-yyyy-mmm-dd.sql)
employees, employees_email, employees_salaries, employees_telephones
daily every day, 1:15 a.m.
✓
Which Databases & Tables Labels and File Naming Patterns
Frequency — Days & Times Location — Security & Off-Site
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Verification Schedule
159
Regularly Verify Back-Ups Practice Restoring Databases, Tables, Rows — Frequency Retention Periods
Restoration TestsBack-Up Verify Tables Rows Retention
company — full back-up weekly monthly two months
company — client tables weekly semi-monthly semi-monthly one month
company — employee tables weekly semi-monthly semi-monthly one monthBack-ups will be verified on a regular basis. For testing and practicing purposes, databases, tables, and rows will be restored regularly in a test environment.
MariaDB for DBAs
Customizing
Introduction
Architecture
Installation
Client Utilities
Configuration
Resource Usage
Storage Engines
Table Schema
Views
Partitioning
User & Server Security
Log Files
Monitoring & Troubleshooting
Back-Ups & Restoration
Customizing
MariaDB Replication
High Availability
Optimization
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Method and Value of Stored Routines
161
Documentation on CREATE PROCEDURE: https://mariadb.com/kb/en/create-procedure/ Documentation on CREATE FUNCTION: https://mariadb.com/kb/v/create-function/
Procedures and Functions are Supported Database Level Functions — Isolating Certain Functionality
Databases Used as Namespaces (e.g., CALL db_name.my_proc())
Routines are Dropped with Database
Library of Common Functions to make Complex Logic Accessible
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Procedures vs. Functions
162
Stored Procedures Called Directly (e.g., CALL show_user()) Can Replace SQL statements, Encapsulate Complex Logic Can Recurse (see, max_sp_recursion_depth and thread_stack) Returns One or More Results Sets
User Defined Functions Called within other SQL Statements (e.g., SELECT DELTA_PCT(n, n)) Performs Smaller Data Manipulation, Calculations, or Conversion Cannot Recurse Returns a Single Scalar Value
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
User Defined Function Example
163
CREATE FUNCTION DELTA_PCT (old_nbr INT, new_nbr INT) RETURNS DECIMAL(6,2)
NO SQL COMMENT 'Fills need for percentage change function.' RETURN (new_nbr - old_nbr) / old_nbr;
SELECT DELTA_PCT(100, 125);
Identify Need: No Function for
Percentage of Change
Determine Any Input and Output
Develop SQL Statements
Stored Procures: https://mariadb.com/kb/v/stored-procedure-overview/ User Defined Functions: https://mariadb.com/kb/v/user-defined-functions/
GRANT EXECUTE ON my.* TO 'tim'@localhost;
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Stored Procedure Example
164
DELIMITER // CREATE PROCEDURE SHOW_USERS()
CONTAINS SQL SQL SECURITY DEFINER COMMENT 'Fills need for SHOW USERS statement.' BEGIN SELECT User AS 'user',
SUBSTRING(Host, 1, 16) AS 'host', IF(Password != '', '','no') AS 'no pwd', IF(is_role = 'Y', 'yes', '') AS 'role', default_role AS 'default role'
FROM mysql.user ORDER BY User, Host; END // DELIMITER ;
CALL my.show_users;Stored Procures: https://mariadb.com/kb/v/stored-procedure-overview/ User Defined Functions: https://mariadb.com/kb/v/user-defined-functions/
Identify Need: No SHOW USERS Statement
Determine Any Input and Output Develop SQL Statements
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Block and Scope
165
Block of SQL Statements Entered Between BEGIN and END
BEGIN SET max_connections = 10; ... END
SET for Non-Local Variables (i.e., external, user, system)
BEGIN DECLARE variable1 INT DEFAULT 100 ... END
CREATE PROCEDURE procedure1()
BEGIN SET @variable1 = CURDATE(); UPDATE table1 SET col1 = @variable1; END
Local Variables Declared Between BEGIN and END
Variable Scope can be Local or Outside the Block
Documentation on BEGIN…END: https://mariadb.com/kb/v/begin-end/
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Flow Control — Conditions & Switches
166
Documentation on IF statement: https://mariadb.com/kb/v/if-statement/
Conditional BranchingDECLARE value1 INT
CASE value1 WHEN 10 THEN statements WHEN 20 THEN statements ELSE statements
END CASE
IF condition THEN statements
ELSEIF condition THEN statements ELSE statements
END IF
Switches or Case Statements
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Flow Control — Loops
167
Documentation on LOOP: https://mariadb.com/kb/v/loop/
Simple LOOP Syntax Example
CREATE PROCEDURE procedure1 BEGIN LOOP
statements END LOOP END //
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Flow Control — Loop Methods
168
Documentation on REPEAT: https://mariadb.com/kb/v/repeat-loop/ Documentation on WHILE: https://mariadb.com/kb/v/while/ Documentation on ITERATE: https://mariadb.com/kb/v/iterate/ Documentation on LEAVE: https://mariadb.com/kb/v/leave/
CREATE PROCEDURE procedure3
DECLARE c1 INT; label3: LOOP SET c1 = c1 +1; IF c1 < 5 ITERATE label3;
END IF; LEAVE label3; END LOOP label3 END //
ITERATE & LEAVE — Label Required
CREATE PROCEDURE procedure2
label1: WHILE condition DO statements END WHILE label1
label2: REPEAT statements
UNTIL condition END REPEAT label2
END //
REPEAT & WHILE Syntax— Label Optional
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Error Handling
169
Documentation on DECLARE HANDLER: https://mariadb.com/kb/v/declare-handler/ Documentation on SIGNAL: https://mariadb.com/kb/v/signal/
DECLARE condition_name CONDITION FOR condition_value
DECLARE is Used also for Conditions
SIGNAL condition_value SET signal_information
SIGNAL for Complete Error Handling
DECLARE handler_type HANDLER FOR condition_value statement
DECLARE can Declare Handlers
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Cursors
170
Documentation on DECLARE CURSOR: https://mariadb.com/kb/v/declare-cursor/
BEGIN <variable_declarations> DECLARE cursor1 CURSOR FOR
SELECT col1,col2 FROM table1; <handler_declarations>
OPEN cursor1; read_loop: LOOP
FETCH cursor1 INTO a, b; ... END LOOP; CLOSE cursor1;
END //
DECLARE Used to Define a Cursor
OPEN, FETCH, CLOSE, then Operate the Cursor
Cursors are also Supported in Triggers and Events
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Triggers
171
Documentation on Triggers: https://mariadb.com/kb/v/trigger-overview/
Executes Automatically SQL Statements — Not for Views or Temporary Tables
Useful for Checking, Filtering, Adjusting Data before Writing
Maintaining Related Data (History Tables, Materialized Views) Mimics Foreign Keys and Constraints for All Storage
Engines
Executes Once per Row, Not Once per SQL Statement Separate Triggers Allowed for Before and After SQL
Statements Stored in .TRG Files in the Database Folder
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Trigger Example
172
Documentation on CREATE TRIGGER: https://mariadb.com/kb/en/create-trigger/ Documentation on DROP TRIGGER: https://mariadb.com/kb/v/drop-trigger/
CREATE TRIGGER salary_change
BEFORE UPDATE ON company.employees_salaries FOR EACH ROW INSERT INTO admin_log.co_salary_chg (user_info, salary_id, amount_before, amount_after, chg_date)
VALUES(USER(), OLD.salary_id, OLD.amount, NEW.amount, NOW());
Action to Trigger: an SQL Statement of Multiple Statements with BEGIN…END
Can't Create Triggers on mysql Database
Timing of Trigger: BEFORE or AFTER
Type of SQL Statement to Start Trigger: INSERT, UPDATE or DELETE
CREATE TABLE admin_log.co_salary_chg
(change_id INT AUTO_INCREMENT KEY, user_info VARCHAR(141), salary_id INT, amount_before DECIMAL(9,2), amount_after DECIMAL(9,2), chg_date DATETIME);
May want to surpress warnings (e.g., binary_format = 'mixed')
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Event Scheduler
173
Documentation on Event Scheduler: https://mariadb.com/kb/v/events/
Executes Tasks based on Predefined Schedule Alternative to cron or similar Utilities — Portable with
Database
Enable and Activate the Event Scheduler
event_scheduler = ON
Entry in Configuration File (e.g., my.cnf)
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Creating & Monitoring Events
174
Documentation on CREATE EVENT: https://mariadb.com/kb/v/create-event/ Documentation on SHOW EVENTS: https://mariadb.com/kb/v/show-events/
CREATE EVENT cleanup_log ON SCHEDULE EVERY 1 HOUR DO DELETE FROM admin_log.user_logins WHERE login_time < DATE_SUB(NOW(), INTERVAL 1 HOUR);
Schedule One-Time or Reoccurring Events Specify Optionally Start and End Dates and Times
Check and Monitor Regularly EventsSHOW EVENTS; SHOW SCHEDULER STATUS;
MariaDB for DBAs
MariaDB Replication
Introduction
Architecture
Installation
Client Utilities
Configuration
Resource Usage
Storage Engines
Table Schema
Views
Partitioning
User & Server Security
Log Files
Monitoring & Troubleshooting
Back-Ups & Restoration
Customizing
MariaDB ReplicationHigh Availability
Optimization
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Purpose of MariaDB Replication
176
Load Balancing (Scaling SELECT Queries) Move Slow, Heavy Queries to Slave Take Slave Off-Line to Make Back-ups
Multiple Data Centers Need Fast Reads Gain Redundancy (High Availability)
Fail Over - Promote Quickly a Slave to Master Fail Over Isn’t Automatic — Requires External
Monitoring Minimal Downtime for Upgrades or Schema Changes
Apply Changes to a Slave Promote Slave to Master and Redirect Traffic Apply Changes to Master and Switch
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Replication Terrain
177
mysqld Data Storage
Master
Binary Log
mysqld Data Storage
Slave 1
Relay Log
Data Storage
Slave 2/Master 2
Relay Log
Binary Log
mysqldSlave 2A
Slave 2B
Slave 2C
INSERT UPDATE DELETE
CREATE ALTER DROP
Dump Thread
Client Threads
SQL Thread
IO Thread
IO Thread
IO Thread
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
MariaDB Replication Factors
178
One Master, Multiple Slaves No true Multi-Master Solution, but Circular Replication
Close to Real Time, but Asynchronous Semi-Synchronous Replication Mode Crash-Safe Slaves with Transactional Storage Engines
A Slave may also be a Master Set log_slave_updates in Configuration File Apply Optionally Replication Filtering Rules or Storage
Engine Changes on Intermediate Slaves
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Replication — Threads
179
Master binlog Dump Thread Pushes binlog Events to Slave Visible in SHOW PROCESSLIST as "Binlog Dump"
Slave IO Thread — Visible in SHOW SLAVE STATUS Requests and receives binlog events from the Master Writes them to the local relay log
Slave SQL Thread — Visible in SHOW SLAVE STATUS
Reads the Relay Log and Executes Queries on Local Data Checks the Query Result Codes Match those Recorded by Master
Slave Multiple Execution Threads Multi-Threaded Slave separates events based on Database Names Updates are Applied in Parallel, Not Sequence
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Parallel Replication
180
Documentation on Parallel Replication: https://mariadb.com/kb/en/mariadb/documentation/replication/standard-replication/parallel-replication/
Replication Process on Slaves Events Received from Master by IO Thread and Queued
in Relay Log Each Relay Log Entry is Retrieved by the SQL Thread Each Transaction is Applied to the Slave
On Non-Parallel Systems, Application Performed Sequentially by SQL Thread
On Parallel Systems, Application Performed in Pool of Separate Replication Worker Threads
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Topologies
181
Master to Slave Simplest Solution and Used Most Widely Allows Off-Loading of SELECT Traffic to Slave
Master1 to MasterN ... to Master1 (circular) Servers Replicate in a Circle, with binlog Events
Traversing the ring until Originating Server Does Not Alleviate Heavy Write Load Needs Careful setup of server-id and
auto_increment_offset, auto_increment_increment settings
Master to Slave to Slaves Can build Complex Trees Useful for Replication Rules or Storage Engine Changes
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Master Configuration
182
GRANT REPLICATION SLAVE ON *.* TO 'maria_replicator'@'52.34.19.24' IDENTIFIED BY 'rover123';
Enable Binary Log — Choose Binary Log Format Set server-id in Configuration File to Unique Value Create Replication User Account on Master
Make a Consistent Snapshot of Data on Master
mysqldump -p -u admin_backup --master-data --flush-logs \ --all-databases > full-dump.sql
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Slave Configuration
183
Documentation on Slave Options: https://mariadb.com/kb/en/mariadb/replication-and-binary-log-server-system-variables/ Documentation on CHANGE MASTER TO: https://mariadb.com/kb/en/mariadb/change-master-to/
CHANGE MASTER TO
MASTER_HOST='35.161.145.71',
MASTER_PORT=3306,
MASTER_USER='maria_replicator',
MASTER_PASSWORD='rover123';
Set server-id in Configuration File to Unique Value Add read-only in Configuration File to Prevent Writes Set Optionally Replication Rules — Covered Later in Class
Restart MariaDB Load Data from Master
Execute START SLAVE on Slave
mysql -p -u root < full-dump.sql
scp -i ./.ssh/mariadb.pem [email protected]:/home/ec2-user/full-dump.sql .
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Monitoring Replication
184
Documentation on SHOW MASTER STATUS: https://mariadb.com/kb/en/show-master-status/ Documentation on SHOW SLAVE STATUS: https://mariadb.com/kb/en/mariadb/show-slave-status/
SHOW MASTER STATUS;
Check Regularly Status on Master — Includes binlog number and position
SHOW SLAVE STATUS \G
Slave_IO_State: Waiting for master to send event Slave_IO_Running: Yes
Slave_SQL_Running: Yes Last_Errno: 0 Last_Error: Seconds_Behind_Master: 300
Check More Often Status of Replication on Slave
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Replication Files
185
Documentation mysqlbinlog: https://mariadb.com/kb/en/mariadb/using-mysqlbinlog/
Binary Log Files (Master) Master Records Write-Queries to File Rotated when Flushed or Periodically to New Log File — File
Name Pattern (.000001)
Relay Log File (Slave) Record of Master binlog Events Rotated when Flushed or Periodically — File Name Pattern (.000001)
Replication Configuration Recorded in master.info (Slave)
Name of Relay Log File Recorded in relay-log.info (Slave)
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Slave Configuration Files
186
52.89.128.176 maria_replicator rover123 3306 60
0 ...
master.info
196803 mariadb-bin.000040
196513 30217 9 ...
relay-log.info
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Replication File Maintenance & Back-Ups
187
Replication Files Updated & Purged Automatically — Don’t Edit or Move Manually
Include Replication Files when Making Binary Back-ups
Use --raw option with mysqlbinlog to Back-up Binary Log
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Binary Log Format
188
Documentation on Binary Log Format: https://mariadb.com/kb/en/mariadb/binary-log-formats/
Statement Based (SBR) — Original Queries are Replicated
Least Data Sent over Wire and Tested for Years Non-Deterministic Statements Executed on Slave — Slave Load is Increased vs. RBR
Row Based (RBR) - Table Rows are Replicated Only Non-Deterministic Statements Executed on Master — Slave Load is Reduced vs. SBR More Data sent over Wire — Not Supported by All Engines
Mixed (default) - Smart Switching between SBR and RBR[mysqld]
binlog-format=MIXED
Checksum (--binlog-checksum) in Binary and Relay Logs to detect Errors
Includes Errors in Memory, Disk, Network and Database Can be Implemented for each Slave
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Slave Filtering Rules — Database Level
189
Documentation on Slave Options: https://mariadb.com/kb/en/mariadb/replication-and-binary-log-server-system-variables/
Exclude Specific Databases (e.g., mysql)
CHANGE REPLICATION FILTER REPLICATE_IGNORE_DB = (), REPLICATE_DO_DB = (sales,inventory);
CHANGE REPLICATION FILTER REPLICATE_IGNORE_DB = (mysql);
Include Specific Databases
Excluding can Cause Problems with Joins
SET GLOBAL replicate_ignore_db = ''; SET GLOBAL
replicate_do_db = 'sales,inventory';
SET GLOBAL replicate_ignore_db = 'mysql';
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Slave Filtering Rules — Table Level
190
Documentation on Slave Options: https://mariadb.com/kb/en/mariadb/replication-and-binary-log-server-system-variables/
Ignore Specific Tables
CHANGE REPLICATION FILTER REPLICATE_IGNORE_DB = (employees), REPLICATE_DO_TABLE = (employees.names, employees.contacts);
CHANGE REPLICATION FILTER REPLICATE_IGNORE_TABLE = (employees.salary);
CHANGE REPLICATION FILTER
REPLICATE_IGNORE_DB = (sales), REPLICATE_DO_TABLE = (sales.europe_%), REPLICATE_IGNORE_TABLE = (sales.europe_uk_%);
Include Specific Tables
Wildcards for Multiple Tables
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
MariaDB Replication - Asynchronous
191
Master Doesn’t Wait for Slaves IO Thread may be Slow to Receive binlog Packets
Network Congestion or Disconnects
SQL Thread may be Slow in Processing Relay Log Events
Load on Slave or Network Problems
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Semi-Synchronous
192
Documentation on Semi-Synchronous Replication: https://mariadb.com/kb/en/mariadb/semisynchronous-replication/
Implemented with an optional Plug-In A COMMIT on Master can Wait for a Slave to
Acknowledge it’s Received Transaction Master Waits for Slave to Write
Transaction to Relay Log, Not to Execute Transaction — Slave SQL Thread may still Lag
One Slave Response needed for Master to Continue (Semi-Synchronous, Not Synchronous)
Can Affect Significantly Performance of Master
INSTALL PLUGIN rpl_semi_sync_master SONAME 'semisync_master.so';
INSTALL PLUGIN rpl_semi_sync_slave SONAME 'semisync_slave.so';
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Lagging Slave
193
Documentation on Time-Delayed Replication: http://dev.mysql.com/doc/refman/5.6/en/replication-delayed.html
When Slave SQL Thread is Slow or Disabled due to Errors, Slave is said to Lag behind Master
Slave SQL Thread must Execute Serially Queries that were Executed in Parallel on the Master
Slave Multiple Execution Threads Multi-Threaded Slave separates events based on
Database Names Updates are Applied in Parallel, Not Sequence
Time-Delayed Replication - Set Provides a Buffer to Stop Replication of Mistakes
CHANGE MASTER TO MASTER_DELAY = seconds;
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Troubleshooting Problems with Replication
194
Check Slave Error Log for Errors affecting Replication Look for Disconnects from Network Problems Binary or Relay Log Event Corruption will cause Slave
SQL Thread to Stop Different Query Error Codes on Slave indicate it’s Not
Synchronized with Master
Tools like Maatkit can help with Replication Troubleshooting and Recovery
May Need to Rebuild Slave from a fresh Snapshot (Back-up of Master or Another Slave)
MariaDB for DBAs
High Availability
Introduction
Architecture
Installation
Client Utilities
Configuration
Resource Usage
Storage Engines
Table Schema
Views
Partitioning
User & Server Security
Log Files
Monitoring & Troubleshooting
Back-Ups & Restoration
Customizing
MariaDB Replication
High AvailabilityOptimization
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
High Availability Goals and Concepts
196
Never Lose Data because of a Crash — Users Should Never Notice
Remain Operational Despite Unforeseen Problems
System Redundancy of Software and Hardware Write Data to Multiple Devices and Locations
(e.g., RAID, Replication, Galera, DRBD) A Second Database Server Designated as a Hot
Spare
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
High Availability Goals — Reality Check
197
Availability Down-Time90% 1 Nine 36.5 days per year 99% 2 Nines 3.65 days per year 99.9% 3 Nines 8.76 hours per year99.99% 4 Nines 52 minutes per year 99.999% 5 Nines 5 minutes per year 99.9999% 6 Nines 31 seconds per year
Availability = Up-Time / (Up-Time + Down-Time)
Availability = Mean Time Before Failure / (Mean Time Before Failure + Mean Time To Recovery)
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
MariaDB Replication
198
Close to Real Time — Asynchronous or Semi-Synchronous
Mono-Threaded on Slaves Semi-Synchronous Replication Mode
A Slave may also be a Master Set log_slave_updates in Configuration File
Master High Availability Manager Monitors the Master Automates Master Fail-Over and Slave Promotion Integrates with Pacemaker & Heartbeat
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
MariaDB MaxScale
199
Tutorials on MaxScale: https://github.com/mariadb-corporation/MaxScale/tree/develop/Documentation/Tutorials Documentation on MaxScale: https://github.com/mariadb-corporation/MaxScale/blob/develop/Documentation/Getting-Started/MariaDB-MaxScale-Installation-Guide.md
Application-to-Database Dynamic Data Routing Platform
Insulates Client Applications from Complexities of Backend Database Cluster
Improves Database Availability, Security and Scalability
Database-to-Database Dynamic Data Routing Platform
Simplifies Interoperability across Databases
MaxScale
MariaDB Master
MariaDBSlave
MariaDBSlave
Client Client Client
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
MariaDB Replication Manager
200
Download MariaDB Replication Manager: https://github.com/mariadb-corporation/replication-managerl
High Availability Solution Monitor and Administer MariaDB Replication and
MariaDB Enterprise Clusters On-Demand Slave to Master Promotion (i.e.,
Switchover) Electing a New Master on Failure Detection — MaxScale
will Route Queries to New Master
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
MariaDB Galera Cluster
201
Documentation for Galera: http://galeracluster.com/documentation-webpages/
Virtually Synchronous Replication Uses Only InnoDB Storage Engine True Multi-Master Solution Conflict Detection & Resolution upon Commit Automatic Provisioning Nodes Isolated Easily for Maintenance
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Distributed Replicated Block Device
202
DRBD User Guide: http://www.drbd.org/en/doc/users-guide-84/p-introl
DRBD is a Linux Kernel Module, Providing Synchronous Replication of a Block Device between Two Servers — Hot Spare Server
If Primary Server Fails, Secondary Server is Used — Immediately and Seamlessly
DRBD Disk Writes over Network slows MariaDB
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Shared-Disk Architecture
203
More information on Shared Storage: https://mariadb.com/sites/default/files/SharedStorage.pdfl
Active – Passive replication Failover requires MariaDB Crash
Recovery — Often File System Crash Recovery Non-Transactional — Only MyISAM Storage
Engine Combined with Pacemaker/Heartbeat
for Auto Failover Virtual IP often used to Fail Over
In theory the SAN is a SPOF
MariaDB for DBAs
Optimization
Introduction
Architecture
Installation
Client Utilities
Configuration
Resource Usage
Storage Engines
Table Schema
Views
Partitioning
User & Server Security
Log Files
Monitoring & Troubleshooting
Back-Ups & Restoration
Customizing
MariaDB Replication
High Availability
Optimization
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Optimization Concepts & Considerations
205
Performance Optimizing relates to Resolving Bottlenecks to Handle More Traffic or Data at a Time
Efficiency Optimizing has to do with the Best Use of Hardware, Bandwidth, and Budget
Consider Carefully Schema and Indexes Don’t Spend too much Time on
Optimizing without Data — Need Real Data and Traffic to Optimize Effectively
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Query Tuning
206
Don’t Tune Queries on Production Server Tune Queries on a Test Server Use a Complete Copy of Data Use Indexes to Improve Read Performance
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Query Analysis
207
Documentation on EXPLAIN: https://mariadb.com/kb/en/mariadb/explain/ Documentation on EXPLAIN EXTENDED: https://mariadb.com/kb/en/mariadb/explain/#explain-extended Documentation on mysqldumpslow: https://mariadb.com/kb/en/mariadb/mysqldumpslow/
EXPLAIN SELECT * FROM employees WHERE MONTH(birth_date) = 8 \G id: 1 select_type: SIMPLE
table: employees type: ALL possible_keys: NULL key: NULL
key_len: NULL ref: NULL rows: 299587 Extra: Using where
Use the Slow Log to find Problem Queries (--log-slow-queries)
Use mysqldumpslow Utility for Manageable Reports
Use EXPLAIN to see how MariaDB Executes a Troublesome Query and if Indexes are Used
Use EXPLAIN EXTENDED and SHOW WARNINGS to see how MariaDB Rearranges a Query before Execution
EXPLAIN EXTENDED SELECT …; SHOW WARNINGS;
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Fields from EXPLAINField Descriptionid Query identifierselect_type Type of SELECT performed
table The table in use (each table in a JOIN will use one row)
*type The table access strategy
possible_keys Any available indexes that could resolve the query
key Index MariaDB chose from possible_keyskey_len The portion of the chosen index to be used (width in bytes)ref Columns from the index, or a constant, usedrows Number of rows the query is expected to touchExtra Extra information relating to the access strategy
208
Documentation on EXPLAIN: https://mariadb.com/kb/en/mariadb/explain/ Documentation on EXPLAIN EXTENDED: https://mariadb.com/kb/en/mariadb/explain/#explain-extended
* See Next Slide for List of Types
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Scan or Index Method TypesField Description
const A constant value can be read once and cached
eq_ref One index access per outer query row
ref Multiple index accesses per outer query row
index_merge Multiple indexes used, merging into a single result set
range Multiple index accesses to return all rows within a range
index Full index scan (every index entry is read sequentially)
all Full table scan (every record is read sequentially)
209
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Choosing Data Types
210
Documentation for PROCEDURE ANALYSE( ): https://mariadb.com/kb/en/mariadb/procedure-analyse/
Appropriate Type (e.g., INT for numbers not VARCHAR)
Accurate Type (e.g., DECIMAL for currency, not DOUBLE)
Smallest Useful Type (e.g., CHAR for known text length, not VARCHAR)
Use NOT NULL, if practical
Analyze Columns with PROCEDURE ANALYSE()
SELECT col1 FROM table1
WHERE col1 = 'test' PROCEDURE ANALYSE() \G
Field_name: db1.table1.col1 Min_value: 7450554
Max_value: 8592336 Min_length: 7 Max_length: 7 Empties_or_zeros: 0
Nulls: 0 Avg_value_or_avg_length: 7955635.7647 Std: 342328.4478 Optimal_fieldtype: MEDIUMINT(7) UNSIGNED NOT NULL
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Optimize Tables
211
Appropriate Data Types - Smaller is Better Shorten Variable Length Columns (i.e., VARCHAR)
— May Expand to Full Width in Memory Use PROCEDURE ANALYSE() for Data Type
Recommendations
DESCRIBE employees;
+------------+---------------+------+
| Field | Type | Null |
+------------+---------------+------+
| emp_no | int(11) | NO |
| first_name | varchar(255) | YES |
| last_name | varchar(255) | YES |
+------------+---------------+------+
SELECT first_name, last_name FROM employees PROCEDURE ANALYSE() \G *************************** 1. row *********************
Field_name: employees.employees.first_name
Min_value: Aamer
Max_value: Zvonko
Min_length: 3
Max_length: 14
Empties_or_zeros: 0
Nulls: 0
Avg_value_or_avg_length: 6.2157
Std: NULL
Optimal_fieldtype: VARCHAR(14) NOT NULL
*************************** 2. row *********************
Field_name: employees.employees.last_name
Min_value: Aamodt
Max_value: Zykh
Min_length: 4
Max_length: 16
Empties_or_zeros: 0
Nulls: 0
Avg_value_or_avg_length: 7.1539
Std: NULL
Optimal_fieldtype: VARCHAR(16) NOT NULL
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Indexes for Performance Optimization
212
MariaDB can Resolve Queries Faster with Indexes Faster Retrieval of Matching Rows, Faster Sorting of
Result Sets Without Indexes, MariaDB does Full Table Scan
Indexes can be for a Column or Multiple Columns (i.e., Composites)
Index for Speed, but Avoid Indexing Excessively or Arbitrarily
Remove Unused or Redundant Indexes
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Table Indexes
213
SHOW CREATE TABLE clients \G
*************************** 1. row ***********************
Table: clients
Create Table: CREATE TABLE 'clients' (
'client_id' int(11) NOT NULL AUTO_INCREMENT,
'name_first' varchar(255) DEFAULT NULL,
'name_last' varchar(255) DEFAULT NULL,
PRIMARY KEY ('client_id'),
KEY 'client_names'
('name_last'(10),'name_first'(5)))
ENGINE=InnoDB DEFAULT CHARSET=latin1
Use SHOW CREATE TABLE to see a Table's Indexes
Documentation on CREATE INDEX: https://mariadb.com/kb/en/mariadb/create-index/
Two Methods for Creating an Index for an Existing Table
ALTER TABLE employees ADD INDEX emp_names (name_last, name_first);
CREATE INDEX client_names ON clients (name_last(10), name_first(5));
Use ALTER TABLE or CREATE INDEX to add an Index
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Creating Indexes
214
CREATE TABLE people
(id INT, name CHAR(20), age TINYINT UNSIGNED, KEY aged_name(age, name));
ALTER TABLE employees
ADD INDEX full_name (name_last, name_first);
CREATE INDEX short_name
ON employees (name_last(8), name_first(5));
Indexes are Created with CREATE TABLE and ALTER TABLE
Create an Index for a new Table with CREATE TABLE
Add an Index to a Table with ALTER TABLE or CREATE INDEX
Keep Indexes as Small as Practical For Indexing Strings, Use Prefix
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Better JOIN Construction
215
SELECT last_name, city, country_name FROM clients
JOIN country_names USING(country_id) WHERE clients.country_id IN('de','fr','it','uk') ORDER BY country_name, city, last_name;
Foreign Keys used with Joins should be Indexed Index Columns in ON and USING Clauses
Keep GROUP BY and ORDER BY columns in one Table for better Indexing
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Optimize Tables
216
Minimize Table Size on Disk and in Memory Use Partitioning Spread Large Tables across Multiple Servers if Application
Permit Archive Table Data when Possible and Appropriate
Index Appropriately Remove Duplicate or Unused Indexes Don’t Index a Column if First Field in Another Index
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Buffer Pool
217
Use the SHOW STATUS statement to see Variables related to Buffer Pool
SHOW VARIABLES LIKE '%innodb_buffer_pool%';
+---------------------------------------+-----------+
| Variable_name | Value |
+---------------------------------------+-----------+
| innodb_buffer_pool_instances | 1 |
| innodb_buffer_pool_populate | OFF |
| innodb_buffer_pool_restore_at_startup | 0 |
| innodb_buffer_pool_shm_checksum | ON |
| innodb_buffer_pool_shm_key | 0 |
| innodb_buffer_pool_size | 134217728 |
+---------------------------------------+-----------+
SHOW STATUS LIKE '%innodb_buffer_pool%';
+-----------------------------------------+-----------+
| Variable_name | Value |
+-----------------------------------------+-----------+
| Innodb_buffer_pool_pages_data | 8032 |
| Innodb_buffer_pool_bytes_data | 131596288 |
| Innodb_buffer_pool_pages_dirty | 0 |
| Innodb_buffer_pool_bytes_dirty | 0 |
| Innodb_buffer_pool_pages_flushed | 850451 |
| Innodb_buffer_pool_pages_LRU_flushed | 119 |
| Innodb_buffer_pool_pages_free | 1 |
| Innodb_buffer_pool_pages_made_not_young | 0 |
| Innodb_buffer_pool_pages_made_young | 207678 |
| Innodb_buffer_pool_pages_misc | 158 |
| Innodb_buffer_pool_pages_old | 2944 |
| Innodb_buffer_pool_pages_total | 8191 |
| Innodb_buffer_pool_read_ahead_rnd | 0 |
| Innodb_buffer_pool_read_ahead | 0 |
| Innodb_buffer_pool_read_ahead_evicted | 0 |
| Innodb_buffer_pool_read_requests | 215938795 |
| Innodb_buffer_pool_reads | 37986 |
| Innodb_buffer_pool_wait_free | 0 |
| Innodb_buffer_pool_write_requests | 4817642 |
+-----------------------------------------+-----------+
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Optimize Hardware
218
Benchmark Systems using Real Data and Real Traffic
Search for Performance Limits Related to Equipment Upgrade, Fine Tune, or Buy More Hardware Buy Fast Disks for Large Databases More RAM for Faster Query Processing - Enough Memory
for Most or All Results Sets More CPU Cores Helps High Traffic Sites
Ideally, Run Only MariaDB on Server (e.g., Web, Email Service on Separate Server)
MariaDB for DBAs
Conclusion
Introduction
Architecture
Installation
Client Utilities
Configuration
Resource Usage
Storage Engines
Table Schema
Views
Partitioning
User & Server Security
Log Files
Monitoring & Troubleshooting
Back-Ups & Restoration
Customizing
MariaDB Replication
High Availability
Optimization
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
On-Line Resources
220
MariaDB Documentation and Knowledge Base: http://mariadb.com/kb
Documentation Knowledge Base Open-Source Community
Post Questions on Forums Ask Questions on IRC — FreeNode
Version 3.3, Slide Copyright MariaDB Ab. Commercial in ConfidenceMariaDB for DBA's
Other Training Courses
221
Training Courses: https://mariadb.learnupon.com/store Upcoming Courses: https://mariadb.learnupon.com/store/sessions
Public In-Person Courses Private In-Person Courses Live Virtual Training Courses Self-Paced On-Line Courses
MariaDB for DBAs
Thanks for Participating