appnexus scalability

1

Database ScalabilitySweep the Query Log & Keep it Clean

2

What Is Scalability?

Webserver tier - Apache, Nginx

Object caches - Memcache

Search servers - Sphinx

Queueing servers - SQS, RabbitMQ

3

Why is DB hard to Scale?

Relational Databases

Single authoritative master

Scaling a write db is hard

Durability

SQL code weight non-obvious

4

ORM Advantages

Middleware, multi-database support

Reusable code, encapsulation & abstraction

Time saving - auto generated model objects

Built for wide audience, reduce need for SQL

Sacrifice a bit of efficiency for code agility

5

ORMs + ScalabilityNegates standard optimization techniques

Difficult to divorce after you are wedded

Limit query tuning at the bare metal

Wide audience not high-scale applications

Bad at deducing JOINs

Fetch 20 cols when you need only 2?

“death by a thousand queries” - Laurie Voss

6

ORM ExampleEx. from http://mattiasgeniar.be

<?php

// First, get all the companies in your database

$companies = $this->getAllCompanies();

$totalValue = 0;

foreach ($companies as $company) {

// For each company there is, retrieve the total value of all their orders

$orders = $company->getOrders();

foreach ($orders as $order) {

$totalValue += (float) $order->getValue();

}

echo "This company made us ". $totalValue ." euro already.";

}

?>

7

ORM Example<?php

// First, get all the companies in your database

SELECT * FROM "company";

$totalValue = 0;

foreach (...) {

// For each company there is, retrieve the total value of all their orders

SELECT * FROM "order" WHERE companyid = $companyid;

foreach (...) {

$totalValue += $row->value;

}

echo "This company made us ". $totalValue ." euro already.";

}

?>

8

ORM Example as SQL

SELECT SUM(o.VALUE) AS TotalValue, c.name AS CompanyName

FROM company AS c

LEFT JOIN "order" AS o ON o.companyid = c.companyid

GROUP BY o.companyid;

9

ORM Articles

ORM is an anti-pattern - Laurie Voss

Bad ORM is infinitely worse than bad SQL - Mattias Geniar

Case against ORM frameworks - Todd Hoff

5 things toxic to scalability - Sean Hull

10

Database Tuning Process

Identify heavy queries in slow log

Find queries in code

Rewrite, run explain & profile

Benchmark with large dataset

Deploy fixed query

11

Being Reactive?

Sweep out the slow query log

Rewrite queries

12

Being Proactive

Keep slow query log quiet

With new code deploys, new slow queries will pop up

Tune early, tune often

13

Use Profiling

set profiling=1;

<run query>

show profile for query 1;

14

Enable Session Profiling(sean@localhost:mysql.sock) [sakila]> set profiling = 1;

Query OK, 0 rows affected (0.00 sec)

(sean@localhost:mysql.sock) [sakila]> select customer_id, last_name from customer where customer_id = 5;

+-------------+-----------+

| customer_id | last_name |

+-------------+-----------+

| 5 | BROWN |

+-------------+-----------+

1 row in set (0.00 sec)

(sean@localhost:mysql.sock) [sakila]> select SQL_NO_CACHE customer_id, last_name from customer where customer_id = 5;

+-------------+-----------+

| customer_id | last_name |

+-------------+-----------+

| 5 | BROWN |

+-------------+-----------+


15

Anatomy of a Profile(sean@localhost:mysql.sock) [sakila]> show profiles;

+----------+------------+--------------------------------------------------------------------------------+

| Query_ID | Duration | Query |

+----------+------------+--------------------------------------------------------------------------------+

| 1 | 0.00015500 | select customer_id, last_name from customer where customer_id = 5 |

| 2 | 0.00054700 | select SQL_NO_CACHE customer_id, last_name from customer where customer_id = 5 |

+----------+------------+--------------------------------------------------------------------------------+

2 rows in set (0.00 sec)

(sean@localhost:mysql.sock) [sakila]> show profile for query 1;

+--------------------------------+----------+

| Status | Duration |

+--------------------------------+----------+

| starting | 0.000062 |

| checking query cache for query | 0.000020 |

| checking privileges on cached | 0.000019 |

| sending cached result to clien | 0.000033 |

| logging slow query | 0.000010 |

| cleaning up | 0.000011 |

+--------------------------------+----------+


16

Non Cached Query Profile

(sean@localhost:mysql.sock) [sakila]> show profile for query 2;

+--------------------------------+----------+

| Status | Duration |

+--------------------------------+----------+

| starting | 0.000052 |

| checking query cache for query | 0.000103 |

| Opening tables | 0.000025 |

| System lock | 0.000014 |

| Table lock | 0.000022 |

| init | 0.000043 |

| optimizing | 0.000022 |

| statistics | 0.000095 |

| preparing | 0.000030 |

| executing | 0.000011 |

| Sending data | 0.000045 |

| end | 0.000013 |

| end | 0.000010 |

| query end | 0.000012 |

| freeing items | 0.000015 |

| closing tables | 0.000015 |

| logging slow query | 0.000010 |

| cleaning up | 0.000010 |

+--------------------------------+----------+


17

Single Column Index

create index d_idx on a (d);

select email from a where d=99;

18

Multi-column Index

create index def_idx on a (d, e, f);

SELECT email FROM a WHERE d=1 and e=2 and f=3;

19

Left-most Prefix Index

create index abc_idx on a (d, e, f);

SELECT name FROM a where d = 1 and e = 2;

SELECT name FROM a WHERE d=1 and e=2 and f=3;

SELECT name FROM a WHERE e=2;

20

Covering Index

create index cover_id on (id, name);

SELECT name FROM a WHERE id = 5;

21

Use EXPLAIN

Illustrates database engine’s path to the data

Shows logical & physical I/Os

Shows sorting, temp tables, index usage

23

EXPLAIN - basic index

(sean@localhost:mysql.sock) [sakila]> create index cust_email on customer(email);


+----+-------------+----------+------+---------------+------------+---------+-------+------+-------------+


+----+-------------+----------+------+---------------+------------+---------+-------+------+-------------+

| 1 | SIMPLE | customer | ref | cust_email | cust_email | 153 | const | 1 | Using where |

+----+-------------+----------+------+---------------+------------+---------+-------+------+-------------+


24

EXPLAIN - covering index

(sean@localhost:mysql.sock) [sakila]> create index cust_email_ln on customer(email, last_name);


+----+-------------+----------+------+---------------+---------------+---------+-------+------+--------------------------+


+----+-------------+----------+------+---------------+---------------+---------+-------+------+--------------------------+

| 1 | SIMPLE | customer | ref | cust_email_ln | cust_email_ln | 153 | const | 1 | Using where; Using index |

+----+-------------+----------+------+---------------+---------------+---------+-------+------+--------------------------+


25

Test With Large Datasets

As data grows, indexing more crucial

Test with million row tables

Slow desktop or test boxes?

26

Can Devops Help?

Teamwork - way of working

Less siloing of departments

DBAs helping identify queries

DBAs helping rewrite queries

Devs on top of new code deploys

Clean slow-log as new queries popup

27

About Sean Hull

[email protected]

Join 8000 & please follow me: @hullsean

www.iheavy.com/blog

www.iheavy.com/signup-scalable-startups-newsletter

mobile: +1-917-442-3939

appnexus scalability

Technology