1 copyright 2006 mysql ab the world’s most popular open source database mysql storage engine...

1Copyright 2006 MySQL AB The World’s Most Popular Open Source Database

MySQLStorage Engine Overview

Dr. Charles A. BellSenior Software Developer

[email protected]


Agenda• MySQL AB• Technology Stacks• MySQL Server Architecture• Pluggable Storage Engines• Server Internals• Q & A


• Founded in 1995• Operations in 22 countries• 10+ million product installations• 50,000 downloads each day• Dramatically reduces TCO of database management• Bundled by more than 100 SW and HW companies• Sold by partners such as HP, Dell, Novell, and others

MySQL AB


Chosen by Successful, Modern Companies

• Embedded– ”Batteries included” database in

software applications

– Network elements

• High Volume Web Sites– Dynamic content

– eCommerce

– Gaming & entertainment

– Scale Out

• Enterprise– Data Warehousing

– High-Volume Transactions

– Departmental

– Intranet/Extranet

– Scale Out


MySQL Software Priorities

LowerTCO

* Robert Lemos CNET News.com Feb 4, 2005

Reliability

Study found comparatively fewer

bugs in MySQL*

Ease of Use15 Minute Rule

Up and running in 15 minutes

PerformanceMySQL, Sun and BEA WebLogic Set

New World Records for Speed & Price/Performance in SPEC

Benchmarks


Second Generation Open Source

• MySQL AB is a profitable company– Develops the software in-house; community helps test it– Owns source code, copyrights and trademarks – Targets the “commoditized” market for databases

• “Quid Pro Quo” dual licensing for OEM market– Cost-effective commercial licenses for commercial use– Open source GPL license for open source projects

• Annual MySQL Network subscription for Enterprise, Web and OEM development/testing

– Per server annual subscription– Includes support, alert and update advisors, Knowledge Base,

Certified/Optimized Binaries

• MySQL Support– Worldwide 24 x 7 support– Training and certification– Consulting

“Reasoning's inspection study shows that the code quality of MySQL was six times better than that of comparable proprietary code. ”


Popular Technology Stacks

MySQL

Linux

Apache

MySQL MySQL

Linuxor

SolarisWindows

ApacheTomcatJBoss

IISApacheJBoss

Java .net / C#

Perl

LAMP J2EE .NET


Supported Technology Platforms

OperatingSystem

Web & Application

Server

Database

Hardware

Storage

…

Internet Information

Server Apache Tomcat …

ProgrammingLanguages

Perl

…

…

…

HP-UX


MySQL Architecture


Pluggable Storage Engine Architecture

• MySQL supports several storage engines that act as handlers for different table types.

• Choose, create, or extend a storage engine that best suits your applications unique requirements.

• What is most important to you?- Read Intensive - Replication

- OLTP - Online Backups

- Transactions - Data Warehousing

- Performance - Foreign Keys

- Scalability - Small Footprint

- Level of Concurrency - Row Level Locking

- Indexes Types - Embedded

- Storage Utilization - Table Level Locking

- High Availability - Clustering



• Storage Engines are available on a per table basis• Changing from one storage engine to another can be

done via a simple SQL command:

ALTER TABLE mytable ENGINE=MyISAM;

Innodb MyISAM


* In MySQL 5.0 transactions are supported, however, the partial rollback of a transaction is not supported. Cluster supports the

READ_COMMITTED, REPEATABLE_READ, and SERIALIZABLE transaction isolation levels.

*



Storage Engines – MyISAM

• Default MySQL engine• No practical limits on data storage• Very efficient storage• Easily handles high-speed data loads• Has B-tree, R-tree, and Full-text Indexes• Supported by special index memory caches• Offers compressed data option• Supports geospatial operations• Uses table level locks• Does not do transactions• Backup/point-in-time recovery supported

Fast Facts

Best Use Cases• High-traffic Web sites• Data warehouses

MySQL Server

MyISAM


Storage Engines – InnoDB

• Provides ACID transaction support• 64TB data storage limit per tablespace• Higher storage cost• Slower data load speed than most other engines• Offers MVCC/Snapshot read• Has B-tree and clustered indexes• Supported by special data & index memory caches• Provides foreign key support• Does not offer compressed data option• Uses row level locks and has custom isolation levels• Has crash recovery• Backup/point-in-time recovery supported

Fast Facts

Best Use Cases• Online transaction processing applications

MySQL Server

Innodb


Storage Engines – Cluster (NDB)

• Transaction support• All data and index reside in main memory• Memory limitation removed for tables in 5.1• High data load speed • Offers MVCC/Snapshot read• Has B-tree indexes• Very fast primary key lookup capabilities • Offers 99.999% uptime• Shared nothing architecture• Has high-speed API for access as well as SQL API• Online backup/point-in-time recovery supported

Fast Facts

Best Use Cases• Highly available, always-on/up applications• Fast directory/key lookup applications

MySQL Server

Cluster(NDB)


Storage Engines – Archive

• New in 5.0• Provides automatic data compression • Offers storage savings up to 80%• No practical storage limit• Fastest data load speed of any storage engine• Offers MVCC/Snapshot read• No index support• Has special insert buffer for fast insert speed• Only supports INSERT and SELECT operations• Uses row level locks• Backup/point-in-time recovery supported

Fast Facts

Best Use Cases• Historical data warehouses• Data archiving applications• Data auditing

MySQL Server

ArchiveArchive


Storage Engines – Federated

• New in 5.0• Allows creation of one logical database from many physical • Acts as “pointer” from one database to another target object• No special middleware needed for remote data access• Speed of operations depends on network/misc. factors• Actions constrained by target engine object properties• Security handled through federated table definition• All SQL operations supported (as per target object)

Fast Facts

Best Use Cases• Distributed database environments• Data Mart environments

MySQL Server

Federated


Storage Engines – Other

• Memory tables reside in RAM; data is lost on shutdown • Memory tables support both B-tree and hash indexes• BDB tables offer transaction support with COMMIT/ROLLBACK• Merge tables are collections of underlying MyISAM tables• Merge tables offer one form of data partitioning• Custom storage engines may be plugged into MySQL

Fast Facts

Best Use Cases• Memory: fast lookups for data objects• BDB: Online transaction processing• Merge: Large databases with partitioned data• Custom: special application situations

MySQL Server

Memory BDB

Merge Custom


Storage Engines – Coming Soon

New Transactional Storage Engines:• Maria• Falcon• others still… MySQL Server

Memory BDB

Merge Custom


MySQL Architecture – Parser/Optimizer

• Validates user’s privileges on accessing database objects and executing SQL calls.

• Converts all SQL calls to internal database language.

• Decides how best to service user’s SQL request.

• Supports all storage engines, which means no special coding per storage engine for particular needs.

Parser

Query Translation, Object Privilege

Optimizer

Access Paths,Statistics


Q & A


Query Trees & Query Execution

An Alternative Query Execution Mechanism


Query Trees

(job = ‘Manager’) (name=‘Sales’) (EMP emp.deptno = dept.deptno DEPT)

EMP DEPT

(job = ‘Manager’) (name=‘Sales’)

emp.deptno = dept.deptno

Alternative‘query tree’ for the query to list all the managers that work in the sales department:


Query Trees

• Tree nodes are atomic operations– Project– Restrict– Join– Sort

• Each node has 0-2 inputs– From 0-2 relations directly– From 0-2 children


Query Trees

Π

Φ

Σ Σ

SELECT Col1, Col2FROM A JOIN (SELECT * FROM C WHERE ColB = 7)ON ColCWHERE A.Col3 > 14


Optimization Techniques

• Cost-Based– Statistics, statistics, statistics

• Manual offline computation• Piggy back

– Strategies• Even distribution

• Heuristic– Knowledge of operations– “Works Best”

• Semantic– Knowledge of schema– Decisions based on behavior/relationships


Heuristic Optimization Example

EMP DEPT



EMP DEPT



EMP DEPT



EMP DEPT



EMP DEPT



EMP DEPT


(job = ‘Manager’)(job = ‘Manager’) (name=‘Sales’)

EMP DEPT

X


(emp.deptno = dept.deptno)

EMP DEPT

X



EMP DEPT

X



OptimisedCanonical Query


Heuristic Algorithmvoid QueryTree::HOptimization(){

SplitRestrictWithJoin(Root);SplitProjectWithJoin(Root);SplitRestrictWithProject(Root);pNode = FindRestriction(Root);while (pNode != 0){

PushRestrictions(Root, pNode);nNode = FindRestriction(Root);

}pNode = FindProjection(Root);while (pNode != 0){

PushProjections(Root, pNode);nNode = FindProjection(Root);

}pNode = FindNaturalJoin(Root);while (pNode != 0){

PushNaturalJoins(Root, pNode);nNode = FindNaturalJoin(Root);

}BalanceJoins(Root);PruneTree(0, Root);

}


Optimization Using Query Trees

SELECT P.NameFROM ProfessorProfessor P, TeachingTeaching TWHERE P.Id = T.ProfId

AND P. DeptId = ‘CS’ AND T.Semester = ‘F1994’

Name(DeptId=‘CS’ Semester=‘F1994’(ProfessorProfessor Id=ProfId TeachingTeaching))

Name

DeptId=‘CS’ Semester=‘F1994’

Id=ProfId

ProfessorProfessor TeachingTeaching


Query Optimizer

• Heuristic optimization eliminates most inefficiencies and generates near optimal queries which can be executed directly from the internal representation (query tree) without modification.

• Test– Replace SELECT-PROJECT-JOIN optimizer in MySQL with

Heuristic Query Optimizer.– To find out the results you need to…


Buy my book!

Expert MySQLSince the enormous number of new features made available with MySQL release 5.0, MySQL has been gaining steam as a viable alternative to database behemoths like Oracle and IBM DB2. MySQL users now have the ability to extend MySQL with new SQL commands, optimize query execution, and embed MySQL within low-resource environments like embedded devices and kiosks.

Expert MySQL, by Dr. Charles A. Bell, is the first book to examine these opportunities in detail, showing you how to wield maximum control over this powerful open source database. You’ll learn how to create your own custom storage handlers, ensuring maximum flexibility and speed within your specialized applications. You’ll also gain valuable insight into MySQL’s architecture and learn how to tweak its behavior through custom changes to the source code.


Q & A

1 copyright 2006 mysql ab the world’s most popular open source database mysql storage engine...

Documents