sql, data plans and gettings things fast from our database -- skiphp presentation
DESCRIPTION
Many developers are stumped at how to make SQL really perform and this presentation covers some of the logic behind SQL, how to read EXPLAIN output, and using VISUAL EXPLAIN. Many folks wonder why an entire table needs to be read when you use a LIMIT or why indexes need maintanance.TRANSCRIPT
![Page 1: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/1.jpg)
SQL, Data Plans and SQL, Data Plans and Gettings Things Fast Gettings Things Fast From Our DatabaseFrom Our DatabaseVery few programmers really understand SQL or how to speed queries to their databases. This session covers that basics of relational calculus (no actual math/calculus will be demanded of attendees), how a RDMS like MySQL tries to optimize the query, and introduces query tuning.
Dave [email protected]@stokerslideshare.net/davestokes
![Page 2: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/2.jpg)
SQL Structured Query Language
Structured Query Language (/ sˈɛ kjuː lˈɛ /,[4] or / si kw lˈ ː ə /; (SQL)[5][6][7][8]) is a
special-purpose programming language designed for managing data held in a
relational database management system (RDBMS).
Originally based upon relational algebra and tuple relational calculus, SQL consists of a data definition language
and a data manipulation language. The scope of SQL includes data insert, query, update and delete, schema
creation and modification, and data access control. Although SQL is often described as, and to a great extent is, a
declarative language (4GL), it also includesprocedural elements.
relational algebra is an offshoot of first-order logic and of algebra of sets concerned with operations over
finitary relations, usually made more convenient to work with by identifying the components of a tuple by a name
(called attribute) rather than by a numeric column index, which is called a relation in database terminology.
--Wikipedia
![Page 3: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/3.jpg)
![Page 4: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/4.jpg)
You send SQL to the server …
The mysqld process will take your input and parse it for VALID syntax.
Then it will build a query plan on how best to retrieve the data.
Finally it goes to fetch the data.* MySQL’s NoSQL queries that skip these steps are MUCH faster
![Page 5: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/5.jpg)
GOALs
1. Get the data that you need and only what you need as fast as possible. No ‘SELECT * FROM’
2. Avoid unnecessary disk/memory reads and disk writes.
3. Make data as compact as is usefull, no BIGINTs for zipcodes.
![Page 6: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/6.jpg)
Cost Based Optimizer
C.5.6. Optimizer-Related Issues
MySQL uses a cost-based optimizer to determine the best way to resolve a query. In many cases, MySQL can calculate the
best possible query plan, but sometimes MySQL does not have enough information about the data at hand and has to make
“educated” guesses about the data.
So MySQL wants to get your data as cheaply as possible and plans accordingly.
![Page 7: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/7.jpg)
Query Plan not lockable with MySQL
Each time MySQL gets a query it will optimise it!
It builds a list of statistics over time to help keep track of data & speed retrieval of the data (5.6 lets you save/restore this information)
Clue: You want FAST!
![Page 8: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/8.jpg)
EXPLAIN
EXPLAIN is a tool to ask the server how it wants to optimize the query.
Prepend to a QUERY
![Page 9: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/9.jpg)
Example Table
![Page 10: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/10.jpg)
Example Query
![Page 11: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/11.jpg)
Example EXPLAIN
![Page 12: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/12.jpg)
Example EXPLAIN with \G
No keys (indexes)
Reads all records in table!
Query#
![Page 13: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/13.jpg)
A Quick Word on Indexes
Indexes allow you to go directly to the record(s) you want (think SSN) instead of reading all records to find the one(s) wanted.
But they require maintenance and overhead.
Not a panacea!
![Page 14: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/14.jpg)
select_type
![Page 15: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/15.jpg)
Using WHERE
Only 274 records read!
![Page 16: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/16.jpg)
Previous query w/o INDEX
No index used and all records in table readto find 274 records wanted!
![Page 17: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/17.jpg)
How to find index(es) already in use
![Page 18: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/18.jpg)
OR ...
![Page 19: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/19.jpg)
A more common example
Has to read ALL records in Country
Could use the PRIMARY KEY but doesn’t!
Uses CountryCode
Optimer estimates 8 reads to match all records - 8x239 = 1,912
![Page 20: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/20.jpg)
Slightly more complex query
SELECT a.Name as 'City',
b.Name as 'Country',
a.population
FROM City a
JOIN Country b
ON (a.CountryCode = b.Code)
WHERE a.population > 3000000
AND b.LifeExpectancy > 66
ORDER BY b.name, a.Population
LIMIT 20;
![Page 21: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/21.jpg)
Gee, we added all that stuff after the where and we are still doing the 239x8 reads!
But to GET the records we need to GET the records and then filter!
![Page 22: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/22.jpg)
Visual Explain
MySQL 5.6 and
Workbench 6 use
JSON format output
to generate diagram.
Costs published with
5.7 and 6.1!
![Page 23: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/23.jpg)
Yet a little deeper into complexity
SELECT CONCAT(customer.last_name, ', ', customer.first_name) AS customer,
address.phone, film.title
FROM rental INNER JOIN customer ON rental.customer_id = customer.customer_id
INNER JOIN address ON customer.address_id = address.address_id
INNER JOIN inventory ON rental.inventory_id = inventory.inventory_id
INNER JOIN film ON inventory.film_id = film.film_id
WHERE rental.return_date IS NULL
AND rental_date + INTERVAL film.rental_duration DAY < CURRENT_DATE()
LIMIT 5;
![Page 24: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/24.jpg)
**
****
****
****
****
****
****
* 1.
row
***
****
****
****
****
****
****
id: 1
sel
ect_
typ
e: S
IMP
LE
tabl
e: fi
lm
typ
e: A
LLpo
ssib
le_k
eys
: PR
IMA
RY
k
ey:
NU
LL
ke
y_le
n: N
ULL
r
ef:
NU
LL
row
s: 1
000
E
xtra
: N
ULL
****
****
****
****
****
****
***
2. r
ow *
****
****
****
****
****
****
**
id
: 1
sele
ct_t
ype
: SIM
PLE
ta
ble:
inve
nto
ry
typ
e: r
efpo
ssib
le_k
eys
: PR
IMA
RY,
idx_
fk_
film
_id
k
ey:
idx_
fk_f
ilm_
id
ke
y_le
n: 2
r
ef:
saki
la.fi
lm.fi
lm_
id
row
s: 2
E
xtra
: U
sing
inde
x**
****
****
****
****
****
****
* 3.
row
***
****
****
****
****
****
****
id: 1
se
lect
_typ
e: S
IMP
LE
tabl
e: r
enta
l
typ
e: r
efpo
ssib
le_k
eys
: idx
_fk_
inve
nto
ry_i
d,id
x_fk
_cus
tom
er_
id
ke
y: id
x_fk
_in
vent
ory_
id
ke
y_le
n: 3
r
ef:
saki
la.in
vent
ory.
inve
nto
ry_
id
row
s: 1
E
xtra
: Usi
ng
whe
re
****
****
****
****
****
****
***
4. r
ow *
****
****
****
****
****
****
**
id
: 1
sele
ct_t
ype
: SIM
PLE
ta
ble:
cus
tom
er
typ
e: e
q_r
ef
poss
ible
_ke
ys: P
RIM
AR
Y,id
x_fk
_ad
dres
s_id
k
ey:
PR
IMA
RY
key_
len:
2
re
f: sa
kila
.ren
tal.c
usto
mer
_id
r
ows:
1
Ext
ra:
NU
LL**
****
****
****
****
****
****
* 5.
row
***
****
****
****
****
****
****
id: 1
se
lect
_typ
e: S
IMP
LE
tabl
e: a
ddr
ess
t
ype:
eq
_re
fpo
ssib
le_k
eys
: PR
IMA
RY
k
ey:
PR
IMA
RY
key_
len:
2
re
f: sa
kila
.cus
tom
er.a
ddre
ss_i
d
row
s: 1
E
xtra
: N
ULL
5 ro
ws
in s
et (
0.00
sec
)
![Page 25: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/25.jpg)
A little easier to understand
![Page 26: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/26.jpg)
Compound Indexes
We can use this index searching on
1. City, State, and Zip
2. City, State
3. City
![Page 27: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/27.jpg)
Musts for better queries
1. Read chapter 8 of the MySQL Manual
2. Join on like data types, INTs with INTS
3. Keep columns as small as practical (PROCEDURE ANALYSE)
4. Maintain B-tree index with ANALYSE TABLE when things are quiet
5. Keep looking for improvments
![Page 28: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/28.jpg)
Hard to teach all in a few minutes
![Page 29: SQL, Data Plans and Gettings Things Fast From Our Database -- SkiPHP Presentation](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6ead34a7959b35f8b4761/html5/thumbnails/29.jpg)
MySQL Connect
Four days with the MySQL Engineers, innovative customers (Facebook, Twitter, Playful Play, Verizon, Paypal, & you), and the top professionals from the MySQL Community.
Starts September 27th in San Francisco!