MySQL for Oracle DBAs

Mark LeithSupport Engineer, MySQL AB

mleith@mysql.com

1

2

Introduction

9 years database experience, starting with Oracle5 years MySQL experienceExtensive experience with database toolsWorking within the MySQL Support Group, for the Americas region

AgendaMySQL ArchitectureBackup and RecoveryManaging space and tablesManaging MemoryTracing Sessions / AuditingMonitoring Tools and TipsConnecting MySQL to OracleQuestions

3

Architecture

Product ArchitectureInternal Memory StructuresFilesystem Layout

BinariesDatabase / Table DatafilesLog Files

Permissions / Ownership

4

Product Architecture

5

Internal Memory Structures

6

Filesystem Layout - BinariesWindows default: C:\Program Files\MySQL\MySQL Server x.x\binTarball default: /usr/local/mysql/bin RPM default: /usr/lib/mysql, /usr/bin & /usr/sbin (depending on RPM) MySQL Server executables

mysqld, mysqld-debug, mysqld-max

MySQL Client Utilities mysql, mysqldump, mysqlcheck, mysqlbinlog ..

Upgrade and Security scripts mysql_upgrade, mysql_fix_privilege_tables, mysql_secure_installation ..

7

Filesystem Layout - DatafilesWindows default: C:\Program Files\MySQL\MySQL Server x.x\dataTarball default: /usr/local/mysql/data RPM default: /var/lib/mysql

Databases use “datadir”Default “mysql” database (SYSTEM Tablespace)One directory per databaseTable frm, data and index files within respective database directories

InnoDB uses “innodb_data_home_dir” Single tablespace file (ibdata1) located in “datadir” by defaultSet multiple datafiles with “innodb_datafile_path”frm files still created within each database directory under datadirRespect datadir conventions when using “innodb_file_per_table”

8

Filesystem Layout - Log filesWindows Default: C:\Program Files\MySQL\MySQL Server x.x\dataTarball default: /usr/local/mysql/data RPM default: /var/log & /var/lib/mysql

Error Log (Alert Log)Set path with “log_error”, defaults to [hostname].err in “datadir”Binary Logs (Archive Redo Logs)Set with “log_bin”Record of all INSERT/UPDATE/DELETE activity<=5.0 Statement Based>=5.1 Statement Based and Row BasedInnoDB Transaction logs (Online Redo Logs)ib_logfile[x] set with “innodb_log_files_in_group” (default 2)Provides “Durability” to ACID complianceControl “checkpoint” times by altering the size of “innodb_log_file_size”

9

10

Permissions / OwnershipNo concept of “ownership”Permissions stored in the “mysql” database“schema” == “database” rather than “users objects”

mysql.userUsername, host, password hash, global privileges, grant optionmysql.dbDatabase level privilegesmysql.tables_privTable level privilegesmysql.columns_privColumn level privilegesmysql.procs_privProcedure / Function privileges

Backup and Recovery

Cold BackupFilesystem Snapshot (cp / rsync)

Warm Backup mysqldump

Hot Backup mysqldump, mysqlhotcopy

Using SlavesRecovery tips

11

12

Backup - Cold BackupFull cold backup

All database directories and contents (including “mysql”)my.cnf/ini fileIf using InnoDB

All InnoDB datafilesAll InnoDB Transaction Redo Log files

Incremental Cold BackupAll of the above as a starting pointBinary logging enabled (”log-bin”)Only requires the latest binary logs since the last snapshotBeware of recovery times - applying the binary logs is a lot slower than file copying

Backup - Warm Backupmysqldump

Uses FLUSH TABLES WITH READ LOCKLocks all other sessions

13

/* Backup db1, db2 and db3 to the same file, mixed MyISAM / InnoDB / other engines */

mysqldump -u root --opt --databases db1, db2, db3 > /backup/file.sql

/* Backup all databases to seperate files per database schema */#!/bin/bash

DATABASES=`mysql -u root -N -e 'SHOW DATABASES'`

OPTIONS1='--extended-insert --no-autocommit --quick --set-charset'OPTIONS2='--disable-keys --skip-add-locks --skip-comments'DUMPOPTIONS="$OPTIONS1 $OPTIONS2"

for D in $DATABASES ; do echo dumping $D ; echo "use $D;" > $D-data.sql echo "set autocommit=0;" >> $D-data.sql mysqldump -u root $DUMPOPTIONS --databases $D \ >> $D-data.sql ; done

Backup - Hot Backupmysqldump

InnoDB onlyUses transactions and multi-versioning

/* Backup InnoDB databases to seperate files per database schema */

#!/bin/bash

DATABASES=`mysql -u root -N -e 'SHOW DATABASES'`

OPTIONS1='--extended-insert --no-autocommit --quick --set-charset --disable-keys'OPTIONS2=' --skip-add-locks --skip-comments --single-transaction --master-data'DUMPOPTIONS="$OPTIONS1 $OPTIONS2"

for D in $DATABASES ; do echo dumping $D ; echo "use $D;" > $D-data.sql echo "set autocommit=0;" >> $D-data.sql mysqldump -u root $DUMPOPTIONS --databases $D \ >> $D-data.sql ; done

14

Backup - Hot Backupmysqlhotcopy

MyISAM and ARCHIVE tables only

LOCK TABLES ..; FLUSH TABLES; [cp | scp]; UNLOCK TABLES;

Requires Perl DBI

/* Hot Backup MyISAM / ARCHIVE table files to seperate directories per database schema */

#!/bin/bash

DATABASES=`mysql -u root -N -e 'SHOW DATABASES'`

DUMPOPTIONS='-u root --method=cp --noindices --flushlog'

for D in $DATABASES ; do echo copying $D ; DIRNAME=$D-`date '+%d%m%y'` mkdir ~/backup/$DIRNAME mysqlhotcopy $DUMPOPTIONS $D ~/backup/$DIRNAME done

15

Backup - Slave Hot BackupRequires no downtime on the master systemAllows full cold backups easily, giving faster recovery timesAdds a server that can also be used to spread load

16

Master Slave

server-id 1 server-id 2

mysqldumpmysqlhotcopy

cpscp

rsyncLVM

Application Users

Reporting Users(Dataguard)

17

Recovery Tipsmysqlbinlog

Perform point in time recovery directly to MySQLmysqlbinlog bin.000001 bin.000002 bin.000003 --stop-datetime=’2006-03-26 12:00:00’ | \mysql -u root -p

Output statements for single databases directly to MySQLmysqlbinlog bin.000001 bin.000002 bin.000003 --stop-datetime=’2006-03-26 12:00:00’ \--database sakila | mysql -u root -p

Output statements to plain text SQL files

mysqlbinlog bin.000001 bin.000002 bin.000003 --stop-datetime=’2006-03-26 12:00:00’ \--database sakila > /tmp/sakila-binlog-dump.sql

Recovery Tips

18

Always use binary logging if possible

On recovery, make an initial copy of all current database filesWork on original set, leaving copy as secondary backup

Take regular full backups, to minimize recovery time

InnoDB Log file size controls full checkpoint activityIf using mysqlhotcopy, with --noindices, to rebuild indexes:

mysqlcheck --all-databases --auto-repair --use-frm

Managing Space and Tables

19

MyISAM Storage EngineGathering stats, managing space

InnoDB Storage EngineGathering stats, managing space

Partitioning Partition Management

20

Managing MyISAM SpaceAll table files stored within specific database directoryRow data held within [tablename].MYDIndex data held within [tablename].MYITable structure held within [tablename].frm

Deleted row space retained for future INSERT/UPDATE

Supports FULLTEXT Indexes (Oracle Text)

Table level locking (SELECT = Shared, INSERT, UPDATE, DELETE = Write)

Data warehousing storage engine

medusa:/usr/local/mysql/data/sakila root# ls -l film_text*-rw-rw---- 1 mysql wheel 119616 Apr 4 20:51 film_text.MYD-rw-rw---- 1 mysql wheel 205824 Apr 4 20:51 film_text.MYI-rw-rw---- 1 mysql wheel 8642 Apr 4 20:51 film_text.frm

Managing MyISAM SpaceSHOW TABLE STATUS

mysql> SHOW TABLE STATUS LIKE 'film_text'\G*************************** 1. row *************************** Name: film_text Engine: MyISAM Version: 10 Row_format: Dynamic Rows: 950 Avg_row_length: 119 Data_length: 119616Max_data_length: 281474976710655 Index_length: 205824 Data_free: 6136 Auto_increment: NULL Create_time: 2006-04-03 14:16:02 Update_time: 2006-04-04 20:51:02 Check_time: 2006-04-04 20:51:17 Collation: utf8_general_ci Checksum: NULL Create_options: Comment: 1 row in set (0.00 sec)

Variable Length rows (VARCHAR etc.)“Fixed” = CHAR etc. static sized

Data size in bytes (~117KB)

Max size for the internal row pointer

Index size in bytes (~201KB)

Space reserved but unused(~6KB)

21

22

Managing MyISAM SpaceINFORMATION_SCHEMA TABLES (DBA_ / USER_ tables)SELECT s.schema_name,CONCAT(IFNULL(ROUND((SUM(t.data_length)+SUM(t.index_length)) /1024/1024,2),0.00),"Mb") total_size,CONCAT(IFNULL(ROUND(((SUM(t.data_length)+SUM(t.index_length))-SUM(t.data_free))/1024/1024,2),0.00),"Mb") data_used,CONCAT(IFNULL(ROUND(SUM(data_free)/1024/1024,2),0.00),"Mb") data_free,IFNULL(ROUND((((SUM(t.data_length)+SUM(t.index_length))-SUM(t.data_free)) /((SUM(t.data_length)+SUM(t.index_length)))*100),2),0) pct_used,COUNT(table_name) total_tablesFROM INFORMATION_SCHEMA.SCHEMATA sLEFT JOIN INFORMATION_SCHEMA.TABLES t ON s.schema_name = t.table_schemaWHERE s.schema_name = "sakila"GROUP BY s.schema_nameORDER BY pct_used DESC\G*************************** 1. row *************************** schema_name: sakila total_size: 6.62Mb data_used: 6.62Mb data_free: 0.01Mb pct_used: 99.91total_tables: 221 row in set (0.08 sec)

23

Managing MyISAM SpaceINFORMATION_SCHEMA TABLES (DBA_ / USER_ tables)

SELECT table_name, ROUND(((t.data_length+t.index_length)-t.data_free)/(t.data_length+t.index_length)*100) pct_used FROM information_schema.tables tWHERE table_schema = 'sakila'HAVING pct_used < 99ORDER BY table_name\G*************************** 1. row ***************************table_name: film_text pct_used: 981 row in set (0.05 sec)

To reclaim space to the filesystem run “OPTIMIZE TABLE”Creates a copy of the table, drops the original, and renames the copyRequires a table lockIf frequent activity on the table run “ANALYZE TABLE”Requires a table lock

Managing MyISAM Spacemyisamchk -dvv

medusa:/Users/markleith/mysql/mysql-5.1.7/data/sakila root# ../../bin/myisamchk -dvv film_text

MyISAM file: film_textRecord format: PackedCharacter set: latin1_swedish_ci (8)File-version: 1Creation time: 2006-04-18 15:57:45Status: changedData records: 1000 Deleted blocks: 0Datafile parts: 1000 Deleted data: 0Datafile pointer (bytes): 6 Keyfile pointer (bytes): 6Datafile length: 119616 Keyfile length: 205824Max datafile length: 281474976710654 Max keyfile length: 288230376151710719Recordlength: 782

table description:Key Start Len Index Type Rec/key Root Blocksize1 2 2 unique short 1 29696 10242 5 254 fulltext ? packed 0 49152 1024 1 4 float 0

Field Start Length Nullpos Nullbit Type1 1 1 2 2 2 no zeros 3 4 767 varchar 4 771 10 1 1 blob

Should not be used whilst table is being used LOCK TABLE .. WRITE; or shutdown

24

25

Managing InnoDB SpaceRow level locking Supports all transaction isolation levelsTables, Indexes and rollback created in a single tablespaceMultiple datafiles, with last datafile allowed to autoextend

medusa:/usr/local/mysql/data root# ls -l ib*-rw-rw---- 1 mysql wheel 5242880 Apr 8 21:25 ib_logfile0-rw-rw---- 1 mysql wheel 5242880 Apr 8 21:25 ib_logfile1-rw-rw---- 1 mysql wheel 18874368 Apr 8 21:25 ibdata1-rw-rw---- 1 mysql wheel 10485760 Apr 8 21:25 ibdata2

} Redo Logs

} Data Files

Filesystem InnoDB OracleDisk PartitionFileInodeFilesystem BlockDisk Block

TablespaceSegment (Table/Index)InodeExtent (64 pages, 1Mb)Page (UNIV_PAGE_SIZE 16Kb)

Tablespace(s)Segment (Table/Index)

ExtentDB_BLOCK_SIZE

Clustered Indexes (Index Organized Tables)Pages and extents managed like a filesystem

SHOW TABLE STATUSmysql> SHOW TABLE STATUS LIKE 'rental'\G*************************** 1. row *************************** Name: rental Engine: InnoDB Version: 10 Row_format: Compact Rows: 16305 Avg_row_length: 97 Data_length: 1589248Max_data_length: 0 Index_length: 1261568 Data_free: 0 Auto_increment: 16050 Create_time: 2006-04-03 14:16:02 Update_time: NULL Check_time: NULL Collation: utf8_general_ci Checksum: NULL Create_options: Comment: InnoDB free: 4096 kB; (`customer_id`) REFER `sakila/customer`(`customer_id`) ON 1 row in set (0.10 sec)

New 5.0 InnoDB compact row format

Data size in bytes (~1.5Mb)

No Maximum (64Tb for tablespace)

Index size in bytes (~1.2Mb)

Unused space not reported by table

Managing InnoDB Space

Freespace reported by tablespace(4Mb)

26

27

Managing InnoDB SpaceInnoDB Table and Tablespace Monitors

Dumped to the error log, or console (stdout) if no log-error

Dump the InnoDB internal Data Dictionary informationSwitched on with special InnoDB tables

create table innodb_table_monitor (a int) engine = innodb;

create table innodb_tablespace_monitor (a int) engine = innodb;

InnoDB Table Monitor

InnoDB Tablespace Monitor

28

Managing InnoDB SpaceInnoDB Tablespace Monitor

================================================060403 14:20:31 INNODB TABLESPACE MONITOR OUTPUT================================================FILE SPACE INFO: id 0size 1152, free limit 832, free extents 3not full frag extents 4: used pages 202, full frag extents 2first seg id not used 0 114SEGMENT id 0 86 space 0; page 209; res 28 used 28; full ext 0fragm pages 28; free extents 0; not full extents 0: pages 0SEGMENT id 0 87 space 0; page 209; res 1 used 1; full ext 0fragm pages 1; free extents 0; not full extents 0: pages 0SEGMENT id 0 88 space 0; page 209; res 16 used 16; full ext 0fragm pages 16; free extents 0; not full extents 0: pages 0SEGMENT id 0 89 space 0; page 209; res 1 used 1; full ext 0fragm pages 1; free extents 0; not full extents 0: pages 0SEGMENT id 0 90 space 0; page 209; res 16 used 16; full ext 0fragm pages 16; free extents 0; not full extents 0: pages 0SEGMENT id 0 91 space 0; page 209; res 1 used 1; full ext 0fragm pages 1; free extents 0; not full extents 0: pages 0SEGMENT id 0 92 space 0; page 209; res 13 used 13; full ext 0fragm pages 13; free extents 0; not full extents 0: pages 0......

29

Managing InnoDB SpaceInnoDB Tablespace Monitor

......SEGMENT id 0 84 space 0; page 2; res 96 used 53; full ext 0fragm pages 32; free extents 0; not full extents 1: pages 21

SEGMENT id 0 84 space 0; page 2;The Internal data dictionary id for the tablespace segment“space 0” is the system tablespace (ibdata files)“page 2” is where the inode of the segment is

res 96 used 53;“res” is total number of reserved pages“used” is the number of pages with data allocated

full ext 0The number of extents that are completely used

fragm pages 32;The first 32 pages are fully allocated

free extents 0;The number of extents that have no pages at all used

not full extents 1: pages 21The number of extents that have pages used but are not full“pages” is how many pages are used within these extents

30

Managing InnoDB SpaceInnoDB Table Monitor

TABLE: name sakila/rental, id 0 26, columns 11, indexes 5, appr.rows 16305 COLUMNS: rental_id: DATA_INT len 4 prec 0; rental_date: DATA_INT len 8 prec 0; inventory_id: DATA_INT len 3 prec 0; customer_id: DATA_INT len 2 prec 0; return_date: DATA_INT len 8 prec 0; staff_id: DATA_INT len 1 prec 0; last_update: DATA_INT len 4 prec 0; DB_ROW_ID: DATA_SYS prtype 8 len 6 prec 0; DB_TRX_ID: DATA_SYS prtype 8 len 6 prec 0; DB_ROLL_PTR: DATA_SYS prtype 8 len 7 prec 0; INDEX: name PRIMARY, id 0 44, fields 1/9, type 3 root page 207, appr.key vals 16305, leaf pages 53, size pages 97 FIELDS: rental_id DB_TRX_ID DB_ROLL_PTR rental_date inventory_id customer_id return_date staff_id last_update INDEX: name rental_date, id 0 45, fields 3/4, type 2 root page 208, appr.key vals 17655, leaf pages 28, size pages 29 FIELDS: rental_date inventory_id customer_id rental_id INDEX: name idx_fk_inventory_id, id 0 46, fields 1/2, type 0 root page 210, appr.key vals 4467, leaf pages 16, size pages 17 FIELDS: inventory_id rental_id INDEX: name idx_fk_customer_id, id 0 47, fields 1/2, type 0 root page 211, appr.key vals 589, leaf pages 16, size pages 17 FIELDS: customer_id rental_id INDEX: name idx_fk_staff_id, id 0 48, fields 1/2, type 0 root page 212, appr.key vals 1, leaf pages 13, size pages 14 FIELDS: staff_id rental_id FOREIGN KEY CONSTRAINT sakila/fk_rental_staff: sakila/rental ( staff_id ) REFERENCES sakila/staff ( staff_id ) FOREIGN KEY CONSTRAINT sakila/fk_rental_inventory: sakila/rental ( inventory_id ) REFERENCES sakila/inventory ( inventory_id ) FOREIGN KEY CONSTRAINT sakila/fk_rental_customer: sakila/rental ( customer_id ) REFERENCES sakila/customer ( customer_id ) FOREIGN KEY CONSTRAINT sakila/fk_payment_rental: sakila/payment ( rental_id ) REFERENCES sakila/rental ( rental_id )

31

Managing InnoDB SpaceInnoDB Table Monitor

TABLE: name sakila/rental, id 0 26, columns 11, indexes 5, appr.rows 16305 COLUMNS: rental_id: DATA_INT len 4 prec 0; rental_date: DATA_INT len 8 prec 0; inventory_id: DATA_INT len 3 prec 0; customer_id: DATA_INT len 2 prec 0; return_date: DATA_INT len 8 prec 0; staff_id: DATA_INT len 1 prec 0; last_update: DATA_INT len 4 prec 0; DB_ROW_ID: DATA_SYS prtype 8 len 6 prec 0; DB_TRX_ID: DATA_SYS prtype 8 len 6 prec 0; DB_ROLL_PTR: DATA_SYS prtype 8 len 7 prec 0;

“len” is column length in bytes, “prec” is the columns precision (DECIMAL etc.)“DB_ROW_ID”, “DB_TRX_ID” and “DB_ROLL_PTR” internal columns for transactions / multi-versioning

INDEX: name PRIMARY, id 0 44, fields 1/9, type 3 root page 207, appr.key vals 16305, leaf pages 53, size pages 97 FIELDS: rental_id DB_TRX_ID DB_ROLL_PTR rental_date inventory_id customer_id return_date staff_id last_update

“fields 1/9” is fields specifically specified in the index (1) and total fields in the index (9) “type” is index type: 3 = PRIMARY, 2 = UNIQUE, 0 = KEY, 1 = AUTO GENERATED PK“leaf pages 53” is the number of pages with data allocated (”used” in tablespace monitor)“size pages 97” is the total number of allocated pages (”res” in tablespace monitor)

Rows: 16305Avg_row_length: 97Data_length: 1589248Max_data_length: 0Index_length: 1261568Data_free: 0

16,384 * 97 = 1,589,248 (1.5Mb)

16,384 * (29+17+17+14) = 1,261,568 (1.2mb)

16,384 * (97-53) = 720,896 (~700Kb)

32

Managing InnoDB SpaceAllocates the first 32 pages individually

After 32 pages allocated, allocates 1Mb extents To reclaim space to the tablespace run “OPTIMIZE TABLE”Reorganization process similar to MyISAMRequires a table lock

“ANALYZE TABLE” rarely needed, as InnoDB updates stats Not Uniform extent allocation, fragmentation highly possibleTo reorganize the tablespace:

mysqldump all tablesStop MySQLmv the InnoDB tablespace datafiles and log files to a backup locationStart MySQLLoad the mysqldump

33

Partitioning (5.1)No extra costsAll storage engines support partitioning“Local” indexes only

Indexes only created locally to each partition“Global” indexes scheduled for 5.2

Indexes created across the entire partitioned tablePartition pruning already available at the optimizer levelAllow far greater table maintenance flexibility, with partition level locks

34

Partitioning (5.1)RANGE partitioning: (As in Oracle 8i RANGE partitions)

Assigns rows to partitions based on column values falling within a given range.

LIST partitioning: (As in Oracle 9i LIST partitions)Similar to partitioning by range, except that the partition is selected based on columns matching one of a set of discrete values.

HASH partitioning: (Not possible in Oracle)A partition is selected based on the value returned by a user-defined expression that operates on column values in rows to be inserted into the table. The function may consist of any expression valid in MySQL that yields a non-negative integer value.

KEY partitioning: (As in Oracle 8i HASH partitions)Similar to partitioning by hash, except that only one or more columns to be evaluated are supplied, and the MySQL server provides its own hashing function. These columns can contain other than integer values, since the hashing function supplied by MySQL guarantees an integer result regardless of the column data type.

Subpartitioning: (As in Oracle 9i composite partitions)Subpartition tables that are partitioned by RANGE or LIST. Subpartitions may use either HASH or KEY partitioning.

35

Partitioning (5.1)Moving to Partitioned tables

CREATE TABLE `RENTAL` ( `rental_id` int(11) NOT NULL auto_increment, `rental_date` datetime NOT NULL, `inventory_id` mediumint(8) unsigned NOT NULL, `customer_id` smallint(5) unsigned NOT NULL, `return_date` datetime default NULL, `staff_id` tinyint(3) unsigned NOT NULL, `last_update` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP, PRIMARY KEY (`rental_id`), UNIQUE KEY `rental_date` (`rental_date`,`inventory_id`,`customer_id`), KEY `idx_fk_inventory_id` (`inventory_id`), KEY `idx_fk_customer_id` (`customer_id`), KEY `idx_fk_staff_id` (`staff_id`), CONSTRAINT `fk_rental_staff` FOREIGN KEY (`staff_id`) REFERENCES `staff` (`staff_id`) ON UPDATE CASCADE, CONSTRAINT `fk_rental_inventory` FOREIGN KEY (`inventory_id`) REFERENCES `inventory` (`inventory_id`) ON UPDATE CASCADE, CONSTRAINT `fk_rental_customer` FOREIGN KEY (`customer_id`) REFERENCES `customer` (`customer_id`) ON UPDATE CASCADE) ENGINE=InnoDB DEFAULT CHARSET=utf8

PARTITION key must be a part of the PRIMARY KEY, if any, if not, any columnPARTITION key must be on an INT column, or expression returning an INTDo not support FOREIGN KEY constraints

36

Partitioning (5.1)Moving to Partitioned tables

DROP TABLE rental_parts;CREATE TABLE `rental_parts` ( `rental_id` int(11) NOT NULL auto_increment, `rental_date` datetime NOT NULL, `inventory_id` mediumint(8) unsigned NOT NULL, `customer_id` smallint(5) unsigned NOT NULL, `return_date` datetime default NULL, `staff_id` tinyint(3) unsigned NOT NULL, `last_update` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP, PRIMARY KEY (`rental_id`, `rental_date`), UNIQUE KEY `rental_date` (`rental_date`,`inventory_id`,`customer_id`), KEY `idx_inventory_id` (`inventory_id`), KEY `idx_customer_id` (`customer_id`), KEY `idx_staff_id` (`staff_id`)) ENGINE=InnoDB DEFAULT CHARSET=utf8PARTITION BY RANGE (YEAR(`rental_date`))( PARTITION rental_2002 VALUES LESS THAN (2003), PARTITION rental_2003 VALUES LESS THAN (2004), PARTITION rental_2004 VALUES LESS THAN (2005), PARTITION rental_2005 VALUES LESS THAN (2006), PARTITION rental_2006 VALUES LESS THAN MAXVALUE);INSERT INTO rental_parts (SELECT * FROM rental);

DROP TABLE rental;RENAME TABLE rental_parts TO rental;

37

Partitioning (5.1)INFORMATION_SCHEMA.PARTITIONS

************************** 1. row *************************** table_schema: sakila table_name: rentalpartition_name: rental_2002 table_rows: 0 data_length: 16384 index_length: 65536*************************** 2. row *************************** table_schema: sakila table_name: rentalpartition_name: rental_2003 table_rows: 0 data_length: 16384 index_length: 65536*************************** 3. row *************************** table_schema: sakila table_name: rentalpartition_name: rental_2004 table_rows: 0 data_length: 16384 index_length: 65536

*************************** 4. row *************************** table_schema: sakila table_name: rentalpartition_name: rental_2005 table_rows: 16272 data_length: 1589248 index_length: 1818624*************************** 5. row *************************** table_schema: sakila table_name: rentalpartition_name: rental_2006 table_rows: 182 data_length: 16384 index_length: 655365 rows in set (0.02 sec)

SELECT table_schema, table_name, partition_name, table_rows, data_length, index_lengthFROM information_schema.partitions WHERE table_name = 'rental’AND table_schema = ‘sakila’

38

Partitioning (5.1)Partition Management - ALTER TABLE

REBUILD PARTITION (No Oracle equivalent)Used to rebuild/reorganize a partitionALTER TABLE rental REBUILD PARTITION rental_2005;

OPTIMIZE PARTITION (No Oracle equivalent)Used like OPTIMIZE TABLE for each storage engine, at the partition levelALTER TABLE rental OPTIMIZE PARTITION rental_2004;

ANALYZE PARTITION (analyze table emp partition(p1) compute statistics;)Reads and stores the key distributions for partitionsALTER TABLE rental ANALYZE PARTITION rental_2005;

CHECK PARTITION (No Oracle equivalent)Checks specific partitions for errors (like CHECK TABLE)ALTER TABLE rental CHECK PARTITION rental_2003;

REPAIR PARTITION (No Oracle equivalent)Used for MyISAM partitions, if they become corrupt (like REPAIR TABLE)ALTER TABLE rental REPAIR PARTITION rental_2003;

39

Partitioning (5.1)ADD PARTITION (Same as Oracle)

Add a partition to the table (to the end of the ALTER TABLE rental ADD PARTITION (PARTITION rental_2007 VALUES LESS THAN (2008);

DROP PARTITION (Same as Oracle)Remove a partition from the tableALTER TABLE rental DROP PARTITION rental_2002;

REORGANIZE PARTITION (ALTER TABLE .. SPLIT / MERGE PARTITION)Split a partition in to multiple partitions, or merge several partitions in to one

ALTER TABLE rental REORGANIZE PARTITION rental_2002 INTO ( PARTITION rental_pre_2002 VALUES LESS THAN (2002), PARTITION rental_2002 VALUES LESS THAN (2003));

ALTER TABLE rental REORGANIZE PARTITION rental_pre_2002, rental_2002 INTO (PARTITION rental_pre_2003 VALUES LESS THAN (2003));

Partition Management - ALTER TABLE

40

Managing Memory

SHOW [GLOBAL | SESSION] STATUSMyISAM Key CacheInnoDB Buffer PoolQuery Cache Thread Buffers

41

Managing MemorySHOW [GLOBAL | SESSION] STATUS [LIKE ‘string’]

Provides session and global level server statisticsGives information about various aspects of the server such as locks, memory usage, disk usage, network activity etc.Akin to various of the V$ views in partsReset statistics with “FLUSH STATUS”

mysql> SHOW STATUS;+-------------------------------------+-----------------+| Variable_name | Value |+-------------------------------------+-----------------+| Aborted_clients | 0 || Aborted_connects | 0 || Bytes_received | 155372598 || Bytes_sent | 1176560426 || Connections | 30023 |...| Created_tmp_disk_tables | 0 || Created_tmp_tables | 8340 || Created_tmp_files | 60 |

42

Managing MemoryMyISAM Key Cache

Caches only MyISAM index dataKey status indicators:

Key_reads - Index blocks read from diskKey_read_requests - Index blocks read from cacheKey_writes - Index blocks written on diskKey_write_requests - Index blocks written within cache

Monitor overall usage with a hit ratio indicatorSHOW GLOBAL STATUS LIKE ‘key_read%’;100-((key_reads/key_read_requests)*100)

Set with “key_buffer_size” in my.cnf, or interactively:SET GLOBAL key_buffer_size = 50*1024*1024;

Current maximum 4Gb

43

Managing MemoryInnoDB Buffer Pool

Caches InnoDB table and index dataKey status indicators (5.0 or greater):

Innodb_buffer_pool_reads - Pages read from diskInnodb_buffer_pool_read_requests - Pages read from the cacheInnodb_buffer_pool_wait_free - Number of waits to create or flush pageInnodb_buffer_pool_read_ahead_seq - Incremented for full table scans

Monitor overall usage with a hit ratio indicatorSHOW GLOBAL STATUS LIKE ‘innodb_buffer_pool_read%’;100-((Innodb_buffer_pool_reads/Innodb_buffer_pool_read_requests)*100)

Set with “innodb_buffer_pool_size” in my.cnfNot a dynamic variable (requires a restart)Typically 50-80% of dedicated server’s available RAM

44

Managing MemoryQuery Cache

Caches query result sets as pre-formed network packetsKey status indicators:

Qcache_hits - Queries satisfied from the cacheQCache_inserts - Query result sets added to the cacheQcache_lowmem_prunes - Query results flushed due to lack of memoryQcache_free_memory - Bytes free

Monitor overall usage with a hit ratio indicator100-((Qcache_inserts/Qcache_hits)*100)

Set with “query_cache_size” and “query_cache_type”SET GLOBAL query_cache_size = 32*1024*1024;

Control block allocation size with “query_alloc_block_size”Control size of result sets cached with “query_cache_limit”

45

Managing MemoryPer Thread Buffers

Possible to set dynamically at the session level with “SET SESSION variable = ..”

sort_buffer_sizeAllocated per thread for each sort operation (ORDER BY / GROUP BY)Values larger than 2-4Mb rarely benefit performanceMonitor for high “Sort_merge_passes” in SHOW GLOBAL STATUS

read_rnd_buffer_sizeUsed when reading sorted rows after an ORDER BY on an indexed columnLarger result sets get better performance from larger buffer sizeLeave GLOBAL default and use SET SESSION when needed for large queries

read_buffer_sizeAllocated per thread for each full MyISAM table scanValues larger than 2-4Mb rarely benefit performanceMonitor for high “Handler_read_rnd_next” / “Select_scan”

join_buffer_sizeAllocated per thread for each JOIN causing a full table scanPossible many allocated per thread with complex/multiple joinsValues larger than 2-4Mb rarely benefit performance

46

Managing Memory

tmp_table_sizeAllocated per thread for each complex GROUP BY or subquries etc.Monitor with “Created_tmp_tables” and “Created_tmp_disk_tables”32-64Mb depending on memory available

binlog_cache_sizeAlso allocated per thread Caches Binary Log entries for multiple statement transactionsMonitor with “Binlog_cache_use” and “Binlog_disk_cache_use” satus variablesLimit with “max_binlog_cache_size” system variable

thread_cache_sizeGLOBAL level variableCaches threads/sessions with an already allocated stack for faster connectionsWhen sessions disconnect they have their buffers flushed and are re-addedMonitor with ratio of “Threads_created” to “Connections”

Per Thread Buffers

47

Tracing Sessions / AuditingSHOW FULL PROCESSLIST (V$SESSION)

mysql> SHOW FULL PROCESSLIST;+----+------+-----------------------+---------+------------------+--------------+-------------+----------------------------------------+| Id | User | Host | db | Command | Time | State | Info | +----+------+-----------------------+---------+------------------+---------------+-------------+---------------------------------------+| 1 | root | localhost:32820 | merlin | Sleep | 1 | | NULL || 3 | root | localhost:32822 | merlin | Sleep | 507 | | NULL || 4 | root | localhost:32823 | merlin | Sleep | 507 | | NULL || 5 | root | localhost:32824 | merlin | Sleep | 506 | | NULL || 6 | root | localhost:32826 | merlin | Sleep | 21207 | | NULL || 7 | root | localhost:32827 | merlin | Query | 0 | Sending data | SELECT data_id,value |FROM `dc_INTEGER` WHERE `begin_time` =(SELECT MAX(`begin_time`) FROM `dc_INTEGER` | WHERE `collect_id`=90) AND `collect_id`=90 FOR UPDATE || 16 | root | localhost:33447 | NULL | Sleep | 3 | | NULL || 19 | root | localhost:48885 | merlin | Sleep | 7 | | NULL || 27 | root | localhost | NULL | Query | 0 | NULL | SHOW FULL PROCESSLIST |+----+------+-----------------------+---------+------------------+--------------+-------------+----------------------------------------+9 rows in set (0.00 sec)

PROCESS privilege required to view the full session listUse FULL keyword to include currently executing SQL

39

Tracing Sessions / AuditingPROCESSLIST states

Sleeping - The thread is idle

Opening Tables - Trying to open a new tableCopying to tmp table - Creating in memory temporary table

Copying to tmp table on disk - Creating disk based temporary tableSorting for group - Performing a GROUP BY operation

Sorting for order - Performing an ORDER BY operationSending Data - Processing and returning SELECT results

Updating - Currently updating row valuesWriting to net - Server is writing packets to the client

Locked - The session is currently blocked by another sessions lock(s)Killed - Kill flag set, waiting for next opportunity to release thread

49

Tracing Sessions / AuditingGeneral Query Log

Records all connections and statements, successful or failedSet with the “log” server variable

log=/var/log/mysql-query.logDoes incur a slight performance overhead

[markleith@medusa:~] $ sudo tail -n 15 mysql/mysql-5.1.7/data/general.logPassword:./mysql-5.1.7/bin/mysqld-debug, Version: 5.1.7-beta-debug-log. started with:Tcp port: 5017 Unix socket: /Users/markleith/mysql/mysql-5.1.7.sockTime Id Command Argument060419 11:58:49 1 Connect root@localhost on 060419 11:58:57 1 Query SELECT DATABASE() 1 Init DB sakila060419 11:59:00 1 Query show tables060419 11:59:19 1 Query desc staff060419 12:00:04 1 Query select staff_id, first_name, last_name, email, username, passwordfrom staff

51

Tracing Sessions / AuditingNew in 5.1 (True V$SESSION, V$SQLTEXT, sys.aud$)

INFORMATION_SCHEMA.PROCESSLISTmysql.general_log - Set with “--log-output=TABLE,FILE” (or either)mysql.slow_log - Set with “--log-output=TABLE,FILE” (or either)

mysql> SELECT * FROM INFORMATION_SCHEMA.PROCESSLIST;+----+---------+--------------+---------+----------------+---------+--------------+------------------------------------------------------------------------------------+ | ID | USER | HOST | DB | COMMAND | TIME | STATE | INFO |+----+---------+--------------+---------+----------------+---------+--------------+------------------------------------------------------------------------------------+ | 2 | root | localhost | | Query | 0 | preparing | SELECT * FROM INFORMATION_SCHEMA.PROCESSLIST | | 1 | root | localhost | sakila | Sleep | 2434 | | |+----+---------+--------------+---------+----------------+---------+--------------+------------------------------------------------------------------------------------+2 rows in set (0.00 sec)

mysql> SELECT p.id, g.event_time, p.user, p.host, g.command_type, replace(g.argument,'\n', '') statement -> FROM information_schema.processlist p -> JOIN mysql.general_log g ON p.id = g.thread_id -> WHERE p.id = 1 AND g.event_time > NOW() - INTERVAL 3 HOUR ORDER BY event_time;+----+----------------------------+------+--------------+----------------+------------------------------------------------------------------------------------------------------------+| id | event_time | user | host | command_type | statement |+----+----------------------------+------+--------------+----------------+------------------------------------------------------------------------------------------------------------+| 1 | 2006-04-19 11:58:49 | root | localhost | Connect | root@localhost on || 1 | 2006-04-19 11:58:57 | root | localhost | Query | SELECT DATABASE() || 1 | 2006-04-19 11:58:57 | root | localhost | Init DB | sakila || 1 | 2006-04-19 11:59:00 | root | localhost | Query | show tables || 1 | 2006-04-19 11:59:19 | root | localhost | Query | desc staff || 1 | 2006-04-19 12:00:04 | root | localhost | Query | select staff_id, first_name, last_name, email, username, passwordfrom staff |+----+----------------------------+------+--------------+----------------+------------------------------------------------------------------------------------------------------------+

52

Tracing Sessions / AuditingTriggers

DML triggers availble [BEFORE | AFTER] [INSERT | UPDATE | DELETE]

System triggers such as “LOGON” not yet availableOnly one trigger of each type allowed (i.e. BEFORE UPDATE)Can perform multiple actions between BEGIN ... END blocks

CREATE TABLE audit_payment ( audit_id INT AUTO_INCREMENT PRIMARY KEY, payment_id INT, new_staff_id TINYINT, old_staff_id TINYINT, new_amount DECIMAL(5,2), old_amount DECIMAL(5,2), updated_by VARCHAR(64), update_time DATETIME);

CREATE TRIGGER tr_audit_payment AFTER UPDATE ON paymentFOR EACH ROWINSERT INTO audit_payment VALUES (NULL,OLD.payment_id,NEW.staff_id,OLD.staff_id,NEW.amount,OLD.amount,CURRENT_USER(),NOW());

53

Tracing Sessions / AuditingTriggersmysql> select * from payment where payment_id = 16049;+-----------------+-------------------+-----------+-------------+------------+----------------------------+-----------------------------+| payment_id | customer_id | staff_id | rental_id | amount | payment_date | last_update |+-----------------+-------------------+-----------+-------------+------------+----------------------------+-----------------------------+| 16049 | 599 | 2 | 15725 | 2.99 | 2005-08-23 11:25:00 | 2006-02-15 22:24:13 |+-----------------+-------------------+-----------+-------------+------------+----------------------------+-----------------------------+1 row in set (0.00 sec)

mysql> UPDATE payment SET amount = 1.99 WHERE payment_id = 16049;Query OK, 1 row affected (0.03 sec)Rows matched: 1 Changed: 1 Warnings: 0

mysql> select * from audit_payment\G*************************** 1. row *************************** audit_id: 1 payment_id: 16049new_staff_id: 2old_staff_id: 2 new_amount: 1.99 old_amount: 2.99 updated_by: root@localhost update_time: 2006-04-19 13:33:481 row in set (0.00 sec)

Monitoring Tools

54

MySQL Network 2.0

Server Overview

55

Monitoring ToolsAlert overview

56

Monitoring ToolsAlert History

Graphing/Trending

57

Connecting Oracle to MySQLHeterogeneous Services

Connects Oracle to MySQL via MyODBCCan create database links from OracleOracle -> MySQL only

Install and configure MyODBCCreate and edit $ORACLE_HOME/hs/admin and add:

/* Replace "MySQL" below with your DSN specified within your MyODBC setup */

HS_FDS_CONNECT_INFO = MySQL HS_FDS_TRACE_LEVEL = OFF

Edit $ORACLE_HOME/network/admin/listener.ora and add:(SID_DESC =(PROGRAM = hsodbc)(ORACLE_HOME = oracle/product/92) /* Your $ORACLE_HOME */(SID_NAME = MySQL) /* Your DSN */)

58

Connecting Oracle to MySQLHeterogeneous Services

Add the following to the tnsnames.ora file:MYSQL =(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=localhost)(PORT=1521))(CONNECT_DATA=(SID=MYSQL))(HS=OK))

Reload your Oracle listener (lsnrctl reload) and then log onto to the Oracle database. To set up the database link:

CREATE DATABASE LINK mysql CONNECT TO "my_user" IDENTIFIED BY "my_password" USING 'mysql';

59

Connecting Oracle to MySQLHeterogeneous Services

SQL> COUNT(*) FROM film@mysql;

COUNT(*)----------1000

SQL> DESC film@mysql5;Name Null? Type----------------------------------------- -------- ----------------------------film_id NUMBER(10)category_id NOT NULL NUMBER(10)title NOT NULL VARCHAR2(27)description LONGrental_duration NOT NULL NUMBER(3)length NUMBER(10)rating CHAR(5)

SQL>INSERT INTO film@mysql5 VALUES (1000000,1,'test','test',1,1,'PG');

1 row created.---change prompts---mysql> USE sakila

mysql> SELECT max(film_id) FROM film;+------------------+| max(film_id) |+------------------+| 1000000 |+------------------+

60

Questions?