my first 100 days with a mysql dbms
Post on 04-Aug-2015
288 Views
Preview:
TRANSCRIPT
My First 100 days with a MySQL DBMS
Presented by : Gustavo René Antúnez DBA Team Lead April, 2015
2
• 15 Years of Data infrastructure management consul4ng
• 200+ Top brands • 6000+ databases under
management • Over 200 DBA’s, in 26 countries • Top 5% of DBA work force, 9
Oracle ACE’s, 2 MicrosoJ MVP’s
• Oracle, MicrosoJ, MySQL partners, Netezza, Hadoop and MongoDB plus UNIX Sysadmin and Oracle apps
About Pythian
• Flexible Environment – Work remotely from anywhere in the world.Flexible
hours • Outstanding People:
– Work with the industry top minds-‐in fact, the top 5%. • Career growth and development:
– Enhance exis4ng skills or learn new ones.Work with leading-‐edge technologies. Experiment and explore using our in-‐house sandbox. Generous training allowance . Professional development days. Internal speaker series.
• Fun, fun ,fun – Blog during work hours-‐please ! Enjoy monthly cheese
tas4ngs with our resident cheese sommelier. Take a day off and volunteer for your favourite charity.
3
#PythianLife
4
Welcome to CLV15
Where do I come From–Oracle DBA
• Started with Version 9.2 in 2004 –Speaker at Oracle Open World, Oracle
Developers Day and Collaborate –Co-‐President of ORAMEX (Mexico Oracle
User Group) –Web Events Chair for IOUG Cloud
Compu4ng Special Interest Group (SIG); –Movie Fana4c & Music Lover –Bringing the best from México (Mexihtli) to
the rest of the world and in the process photographing it :)
– rene-‐ace.com –@rene_ace
• #CLV15
5
Where do I come from?
6
How did you get to be a DBA
7
6th Happiest Job of 2014! It was #1 in 2014
8
http://www.forbes.com/sites/susanadams/2014/03/20/the-happiest-and-unhappiest-jobs-in-2014/
Work-life balance
Relationship with boss and co-workers
Daily tasksJob resources
Field will grow by 15% between
2012 and 2022
DBA can be the key driver of
success
Happiest Job of 2034?
Oxford University: THE FUTURE OF EMPLOYMENT: HOW SUSCEPTIBLE ARE JOBS TO COMPUTERISATION?
• 47 percent of American jobs are at high risk of being taken by computers within the next two decades. – 1st Wave
• Computers will start replacing people in especially vulnerable fields like transporta4on/logis4cs, produc4on labor, and administra4ve support.
– 2nd Wave • Dependent upon the development of good ar4ficial
intelligence. This could next put jobs in management, science and engineering, and the arts at risk.
9
How can you tell an Oracle DBA has touched your MySQL Installation?
MYSQL_HOME=/home/oracle/products/mysql-‐version
10
What is a Relational Database?
11
• Accessible and Rela4onal • Data Management • Organised Data Store • Systema4c Data Architecture • Security Enabled • Tuneable Performance • Ability to u4lise Opera4ng System and its resources at
kernel level system soJware.
What is MySQL?
12
• Popular soJware in open source world with GNU license
• Easy to manage , easy to use features • Mul4 plaqorm support • Adoptable to popular soJware such as PHP and Java • De-‐facto standard for web and e-‐commerce
applica4ons and unbeatable choice for startups
MySQL Version History
13
MySQL 5.7: Improvements in latest DMR
• InnoDB for beter transac4onal throughput, availability, IO
•Replica.on for beter scalability and availability
• Fabric for high availability and sharding •Performance Schema for new and improved
performance metrics •Op.mizer for beter EXPLAINing, parsing,
query performance •GIS with na4ve InnoDB spa4al indexes and
Boost.Geometry integra4on
Terminology
14
Oracle MySQL
Database (files) Database Server Instance
Database Instance (memory) Database Server Instance
Schema User Database
User User
Table Space Table Space
Storage Engine
MySQL Architecture
15
MySQL Architecture
16
• SQL Interface • Mechanisms to receive commands and transmit
results to the user. • Parser
• Lex-‐YACC script that is compiled with Bison • Constructs a query structure used to represent the
query statement (SQL) in memory as a tree structure (abstract syntax tree)
MySQL Architecture
17
• Query Op.mizer • Is a SELECT-‐PROJECT-‐JOIN
• Query Execu.on • Set of library methods designed to implement a par4cular query.
• mysql_insert() method is designed to insert data. • mysql_select() method designed to find and return data
matching the WHERE clause • located in a variety of source code files under a file of a
similar name (e.g., sql_insert.cc or sql_select.cc)
MySQL Architecture
18
• Query Cache • Caches not only the query structure but also the query results
• Buffer Manager/Cache and Buffers • Responsible for ensuring that the most frequently used data (or structures, as you will see) are
available in the most efficient manner possible • Table Cache
• Designed to store metadata about the tables in memory. • Record Cache
• Enhances sequen4al reads from the storage engines • Key Cache
• Buffer for frequently used index data (used exclusively for MyISAM tables ) • Privilege Cache
• used to store grant data on a user account • Hostname Cache
• Contains the hostnames of all the connec4ons to the server
MySQL Architecture
19
• The Storage Manager • Interfaces with the opera4ng system to write data to
the disk efficiently • Transac.on Manager
• Provides a locking facility to ensure that mul4ple simultaneous users access the data in consistent way
Option Files
20
• Beter known as configura4on files
• Lists specific startup op4ons • Common Loca4ons of Op4on
File • Windows
• INSTALLDIR\my.ini • INSTALLDIR\my.cnf
• Linux & Mac • /etc/my.cnf • /etc/mysql/my.cnf
# The following options will be passed to all MySQL clients[client]
#password = your_password
port = 3306
socket = /tmp/mysql.5.6.16.sock
# Here follows entries for some specific programs# The MySQL server [mysqld] port= 3306 socket= /tmp/mysql.5.6.16.sock log-error = /Users/ligaya/mysql_installs/mysql-5.6.16-osx10.7-x86_64/data/error.log
server-id= 1 table_open_cache = 4000
How to find your Option Files
21
$strace -e stat64 mysqld --print-defaults$lsof -p <mysqld_pid)
Generic Unix/Linux Binary File Layout
22
Directory Contents
bin Client programs and mysqld server
data Log files and databases
docs Manual in Info format
man Unix manual pages
include Include (header) files
lib Libraries
scripts mysql_install_db
share Misc support files including error messages, sample option files and SQL for database installation
sql-bench Benchmarks
Linux RPM File Layout
23
Directory Contents
/usr/bin Client programs and scripts
/usr/sbin mysqld server
/var/lib/mysql log files and databases
/usr/share/info Manual in Info format
/usr/share/man Unix manual pages
/usr/include/mysql
Include (header) files
/usr/lib/mysql Libraries
/usr/share/mysql Misc support files including error messages, character set files, sample option files and SQL for database installation
/usr/share/sql-bench Benchmarks
Disk Space Usage
24
• Database directories • Hold data files associated with storage engine types
• Table format files (*.frm) • Keeps the defini4on (schema) of the table
• Data and index files • Tables under same top-‐ level directory used for database • MyISAM storage engine creates one file per table for
each data and index.
It can be as easy…
25
[root@localhost ~]# groupadd mysql [root@localhost ~]# useradd -r -g mysql mysql [root@localhost ~]# yum localinstall mysql-community-release-el6-6.noarch.rpm [root@localhost ~]# yum repolist enabled | grep "mysql.*-community.*" [root@localhost ~]# yum repolist all | grep mysql [root@localhost ~]# yum install mysql-community-server [root@localhost ~]# service mysqld start Starting mysqld:[ OK ] [root@localhost ~]# service mysqld status mysqld (pid 407) is running. [root@localhost ~]# mysql_secure_installation
Generic Binary Installa4on
Storage Engines
26
• General-‐purpose storage engine • As of 5.5 is the default engine • Main Features:
• ACID compliant • Row level locking with consistent reads Foreign keys • Clustered primary key index • FullText Indexes
InnoDB
Storage Engines
27
• Default storage engine prior to 5.5 Stored on disk as 3 files • .frm (Format file) • .MYD (Data file) • .MYI (Index file)
• Features • FullText indexes • Geospa4al datatypes support
• There is a limit of 232 (~4.295E+09) rows in a MyISAM table
MyISAM
Storage Engines
28
• Memory • CSV • Archive • Blackhole • 3rd Party
Others
Storage Engines
29
mysql> mysql> SHOW ENGINES\G*************************** 1. row *************************** Engine: MEMORY Support: YES Comment: Hash based, stored in memory, useful for temporary tablesTransactions: NO XA: NO Savepoints: NO*************************** 2. row *************************** Engine: MyISAM Support: YES Comment: MyISAM storage engineTransactions: NO XA: NO Savepoints: NO*************************** 3. row *************************** Engine: InnoDB Support: DEFAULT Comment: Supports transactions, row-level locking, and foreign keysTransactions: YES XA: YES Savepoints: YES
9 rows in set (0.28 sec)
Creating a table with a specific storage engine
30
mysql> CREATE TABLE test1 (col1 INT, col2 CHAR(30), PRIMARY KEY (col1)) ENGINE = INNODB;Query OK, 0 rows affected (0.29 sec)
mysql> CREATE TABLE test2 (col1 INT, col2 CHAR(30)) ENGINE = MYISAM;Query OK, 0 rows affected (0.36 sec)
Creating a table with a specific storage engine
31
mysql> SHOW TABLE STATUS FROM rene;+------------+-----------------+----------+----------------+---------+| Name | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length |Index_length | Data_free | Auto_increment | Create_time | Update_time | Check_time |Collation | Checksum | Create_options | Comment |+--------+--------+---------+------------+------+----------------+-------------+------------+-----------------+----------+----------------+---------+| test1 | InnoDB | 10 | Compact | 0 | 0 | 16384 | 0 | 0 | 0 | 1 | 2015-01-08 18:21:05 | NULL | NULL |utf8_general_ci | NULL | | || test2 | MyISAM | 10 | Fixed | 0 | 0 | 0 | 26740122787512319 | 1024 | 0 | NULL | 2015-01-08 18:35:47 | NULL | NULL |utf8_general_ci | NULL | | |+--------+--------+---------+------------+------+----------------+-------------2 rows in set (0.03 sec)
Storage Engines
32
Feature MyISAM Heap BDB InnoDB
Transactional No No Yes Yes
Lock granularity Table Table Page Row
Storage A data file (.MYD) and an index file (.MYI) for each table
In-memory A single data and index file (.db) for each table
A set of data files for all the tables
Portable Yes N/A No Yes
Contrasting Architecture
33
Area MySQL Oracle
Memory Caches • MyISAM key caches
• InnoDB data cache
• InnoDB log cache
• Dictionary cache
Data cache (variants)
Log buffer
Shared Pool
Java Pool Redo/Undo Logs • InnoDB Undo Space
• InnoDB Logs
• Undo Tablespace
• Redo Logs
• Archive Logs Data Storage • Tablespaces
• Table/Index Files
• Format files
• Tablespaces
• Datafiles
Optimizer • Cost-based • Cost-based
Error Log
34
• Default loca4on: host_name.err in datadir • Logs
• Start and stops • Cri4cal errors • InnoDB Monitor data (if turned on) • MyISAM tables that need to be checked or repaired • Some OS’s -‐ stack trace if mysqld crashes
log-‐error[=file_name]
Slow Query Log
35
• First line of offense for tuning queries • Why?
• Performance usually • Enable dynamically or with -‐-‐slow-‐query-‐log
• Default file loca4on: host_name-‐slow.log in the datadir • Can also go to a table
• Mul4ple op4ons for controlling it • Aggregate its data with mysqldumpslow u4lity
log-‐error[=file_name]
Explain
36
• Prior to 5.6 EXPLAIN [explain_type] SELECT select_options
• 5.6 EXPLAIN [explain_type] explainable_stmt
Displays query execu4on plan SyntaxColumn Meaning
id The SELECT identifier
select_type The SELECT type
table The table for the output row
type The join type
possible_keys The possible indexes to choose
key The index actually chosen
key_len The length of the chosen key
ref The columns compared to the index
rows Estimate of rows to be examined
Extra Additional information
Explain
37
mysql> explain extended select j1.c1 from j1, j2, j3 where j1.c1 = j2.c1 and j3.c1 = j1.c1;+----+-------------+-------+------+---------------+------+---------+------+------+-------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |+----+-------------+-------+------+---------------+------+---------+------+------+-------------+| 1 | SIMPLE | j1 | ALL | NULL | NULL | NULL | NULL | 2 | || 1 | SIMPLE | j2 | ALL | NULL | NULL | NULL | NULL | 3 | Using where || 1 | SIMPLE | j3 | ALL | NULL | NULL | NULL | NULL | 4 | Using where |+----+-------------+-------+------+---------------+------+---------+------+------+-------------+3 rows in set, 1 warning (0.00 sec)
Example
User Accounts
38
They are divided into 2 parts
• username • hostname • Eg:
• ‘root’@‘localhost’ • ‘’@‘127.0.0.1’ • ‘root’@‘192.168.1.%’ • ‘lig’@’%’
• Stage 1: Connec4ng • Who are you?
• host • username
• Stage 2: Request • For each request
• What are you doing and are you allowed to do that? (Grants)
Backups
39
They are divided into 2 parts• Logical Backups • What is Saved?
• Logical structure • Content
• Machine Independent • Slower • Server Up/Warm • Commonly used
• mysqldump • Editable text files • Very litle Scalability
[mysql@localhost Desktop]$ mysqldump -u pythian – p $pw --extended-insert=true --add-drop-database--databases rene | gzip > /export/full_dump.gz
Backups
40
They are divided into 2 parts• Physical Backup • Raw copies • Faster • Compact • Usually server is down or locked • Commonly used
• Filesystem Snapshot • MySQL Enterprise Backup
• Official solu4on for “Hot” backups
• XtraBackup (3rd party solu4on from Percona)
a) FLUSH TABLES WITH READ LOCK;
b) Take the filesystem snapshot c) UNLOCK TABLES;
MySQL SYS Schema
41
• SYS is similar to: • Oracle V$ catalog views • MicrosoJ SQL DMVs (Dynamic Mgmnt Views) • IBM DB2 SYSIBM catalog
• Helps simplify DBA tasks • Monitor health, growth rates, and other key metrics • Spot, diagnose, and tune performance problems
• Provides easy to understand insights into • IO hot spots, Costly SQL statements • Dynamic table, index, and schema sta4s4cs • Wait 4me analysis, Locking • InnoDB sta4s4cs
Schema Objects in Oracle and MySQL
42
Oracle MySQL
Package N/A
PL/SQL procedure Routine
PL/SQL function Routine
Role N/A
Sequence AUTO_INCREMENT for a column
Synonym N/A
MySQL Case Sensitivity
43
• Oracle: • Column, index, stored procedure,
and trigger names as well as column aliases are case insensi4ve on all plaqorms
• MySQL • Databases correspond to directories
within the data directory.Case sensi4vity of the database and table names is determined by the case sensi4vity of the underlying opera4ng systems
44
MySQL Replication - How it works?
45
MySQL Replication - How it works?
46
• On the master • Replica4on events writen to a
special log called binary log • stores data that
replica2on slave will be reading later
• Replica2on slave connects to a master, master creates a new thread for the connec2on
• On the replica • Two threads are started on the slave
• IO thread • Reads binary log events from the
master as they come in and just copies them over to a local log file called relay log
• SQL thread • Reads events from a relay log
stored locally on the replica4on slave and then applies them as fast as possible.
MySQL Replication Types
47
MySQL Best Features
48
• Free, Almost Free, Lowest Total Cost of Ownership, Open Source
• Easy setup and low profile replica4on features • Scalability and Flexibility • High Performance • High Availability • Robust Transac4onal Support • Works with very limited resources • Plaqorm availability
MySQL Features - Cons
49
• Tablespace management • Role access management • Premature Stored Procedures and Triggers • Advanced Par44oning • Advanced Views • Lack of Parallelism • Struggle with Large and Cri4cal Data • Limited Backup and Recovery op4ons for VLDB • Some other features in 5.6, 5.7 …
50
FIT-ACER
• F – Focus (SLOW DOWN! Are you ready?)
• I – Identify server/DB name, time, authorization
• T – Type the command (do not hit enter yet)
• A – Assess the command (SPEND TIME HERE!)
• C – Check the server / database name again
• E – Execute the command
• R – Review and document the results
51
53
To contact us
sales@pythian.com
1-877-PYTHIAN
To follow us
http://www.pythian.com/blog
http://www.facebook.com/pages/The-Pythian-Group/163902527671
@pythian
http://www.linkedin.com/company/pythian
Thank you – Q&A
top related