does your performance tuning need a 12 -step program? · does your performance tuning need a 12...
TRANSCRIPT
DOES YOUR PERFORMANCE TUNING NEED A 12-STEP PROGRAM?
JANIS GRIFFINSENIOR DBA / PERFORMANCE EVANGELIST
• Senior DBA / Performance Evangelist for SolarWindso [email protected] Twitter® - @DoBoutAnythingo Current – 25+ Years in Oracle®, Sybase®, SQL Server®, MySQL®o DBA and Developer
• Former: Database Design & Implementation
• Specializes in performance tuning
• Review database performance for customers and prospects
• Common thread – How do I tune it?
WHO AM I?
2© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
• Challenges Of Tuningo Who should tuneo Which SQLs to tune
• Utilize Response Time Analysis (RTA)o Wait Events / Wait Time
• 12 Steps To Follow
• Several Case Studies
AGENDA
3© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
• SQL tuning is hard. o Who should tune – DBA or Developer?o Which SQL to tune?
• Requires expertise in many areas. o Technical – Plan, Data Access, SQL Design.o Business – What is the Purpose of SQL?
• Tuning takes timeo Large Number of SQL Statements.o Each statement is different.
• Low priority in some companieso Vendor applications.o Focus on hardware or system issues.
• Never ending.
CHALLENGES OF TUNING
4© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
5
Image courtesy of Gentle-Stress-Relief.com
© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
1. FIND WHICH SQL TO TUNE
• Methods for Identifying. • User / batch job complaints.
o Known poorly performing SQL.o Trace session/process.
• Queries performing most I/O (rows examined vs rows sent).o Table or index scans.
• Queries consuming CPU.• Highest wait times – DPA.
o Performance schema.o Information schema.
6© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
WAIT TIME ANALYSIS
7
Understand the total time a Query spends in Database.Measure time while Query executes.MySQL helps by providing wait events & thread states.
Focus on Wait Time
© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
MYSQL 5.6+ PERFORMANCE_SCHEMA
• Greatly improved.• More information / more instrumentation.• Current / historical events at all levels.
o Statements.o Stages.o Waits.
• Performance_Schema consumers now ON by Default.• Storage Engine now defaults to INNODB.• 5.7 Performance_Schema.
o 35 new tables – 5.7 has 87, 5.6 had 52, 5.5 had 17.o Reduced overhead and footprint.o 1005 Instruments (5.7.10).
• New ones for transactions, metadata locks, memory usage.o SYS Schema – now provided by default.
• 100 views – User, Host, IO, Innodb, Memory Summaries.• Wait analysis and statement level. 8© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
MYSQL – WAIT TIME DATA
9Information_Schema© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
BASE QUERY
INSERT INTO wta_data SELECT current_timestamp, t.processlist_id AS conn_id, t.processlist_user AS user, t.processlist_host AS host, t.processlist_db AS db,LOWER(t.processlist_state) AS state, t.processlist_info AS current_sql, MD5(t.processlist_info) AS current_sql_md5,t.processlist_command AS command, w.event_id AS wait_event_id, w.end_event_id AS wait_event_end_id,w.event_name AS wait_event_name, w.operation AS wait_operation, w.object_schema AS wait_schema, w.object_name AS wait_object_name,w.object_type AS wait_type, w.index_name AS wait_index, s.event_name AS statement_event, s.sql_text AS statement_sql,MD5(s.sql_text) AS statement_sql_md5, CASE WHEN RIGHT(s.digest_text, 3) = '...' THEN 1 ELSE 0 END AS statement_digest_truncated,s.digest AS statement_digest, s.event_id AS statement_event_id, s.end_event_id AS statement_event_end_id,conn.attr_value AS program_name, trx.trx_state, trx.trx_operation_state, trx.blocking_idFROM performance_schema.threads AS t
LEFT JOIN performance_schema.events_waits_current AS w ON w.thread_id = t.thread_id AND w.end_event_id IS NULLAND w.event_name <> 'idle'
LEFT JOIN performance_schema.events_statements_current AS s ON s.thread_id = t.thread_id AND s.digest IS NOT NULLLEFT JOIN (SELECT processlist_id, GROUP_CONCAT(attr_value ORDER BY attr_name DESC SEPARATOR '~^~') attr_value
FROM performance_schema.session_connect_attrsWHERE attr_name IN ('program_name','_client_name') GROUP BY processlist_id ) conn
ON t.processlist_id = conn.processlist_idLEFT JOIN (SELECT trx.trx_mysql_thread_id processlist_id, blocking_trx.trx_mysql_thread_id blocking_id, trx.trx_state,
trx.trx_operation_stateFROM information_schema.innodb_trx trxLEFT JOIN (SELECT requesting_trx_id, MIN(blocking_trx_id) blocking_trx_id FROM information_schema.innodb_lock_waitsGROUP BY requesting_trx_id) lock_waitsON trx.trx_id=lock_waits.requesting_trx_idLEFT OUTER JOIN information_schema.innodb_trx blocking_trx ON lock_waits.blocking_trx_id=blocking_trx.trx_id) trx
ON trx.processlist_id = t.processlist_idWHERE t.processlist_id IS NOT NULLAND t.instrumented = 'YES'AND t.processlist_id <> connection_id()AND (t.processlist_info IS NOT NULL OR s.sql_text IS NOT NULL)AND (t.PROCESSLIST_COMMAND <> 'Sleep') 10© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
QUERY - NOT ROCKET SCIENCE
11
select w.sql_text, w.wait_operation, w.time_in_seconds, tot.tot_time from (select substr(current_sql, 1, 60) sql_text, wait_operation, count(*) time_in_secondsfrom wta_data w group by substr(current_sql, 1, 60), wait_operation) w,
(select substr(current_sql, 1,60) sql_text, count(*) tot_time from wta_datagroup by substr(current_sql, 1,60)) tot
where w.wait_operation is not nulland w.sql_text = tot.sql_textorder by tot.tot_time, time_in_seconds;
RTA - WAIT TIME & EVENTS
12
Almost 16 hours spent on ‘waiting for table metadata lock’
100% of time on ‘sending data’ Over 20 hours spent!
© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
RTA BENEFITS
12© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
RTA – BLOCKING ISSUE
14© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
2. GATHER RUN-TIME DETAILS
• Get baseline metrics.o How long does it take now?o What is acceptable? (Ten seconds? Two minutes? One hour?)o Record ‘rows examined’ and ‘rows sent’ for tuning comparison.
• Collect wait or thread states.o Locking / blocking (system lock).o I/O problem (sending data).o Calculating statistics (statistics).o Network slowdown (writing to net).o May be multiple issues.o All have different resolutions.
15© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
BASELINE METRICS
16© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
3. GET EXECUTION PLAN
17
• Explain
• Explain Extended • Explain FORMAT=JSON • Optimizer Trace 5.6.3+• MySQL Workbench
© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
EXPLAIN CHEAT SHEET
• http://d1kgm347lql2s5.cloudfront.net/blog/wp-content/uploads/explain-diagram1.pdf
18© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
EXPLAIN EXTENDED EXAMPLE
• Shows optimizer transformation of query.o It can help you write better SQL. 18
© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
EXPLAIN FORMAT=JSON
• JSON format shows extended and partition information. 19© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
OPTIMIZER TRACING
• Stored in information_schema.optimizer_trace• Default is 16k of memory.• set optimizer_trace_max_mem_size=1000000;• set optimizer_trace_features="greedy_search=OFF";
• Not as verbose.
20© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
MYSQL WORKBENCH
22© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
4. EXAMINE THE EXECUTION PLAN
22
• Find expensive operators.o Examine IO costs, rows examined vs. rows sent.o Look for full table or clustered index scans.
• Review the predicate information.o Know how parameters are being interpreted.
• Review the data types.• Implicit conversions?
o Know which step filtering predicate is applied.
• Review temporary tables and filesort activities in extra column.o Try to fix them.
5. REVIEW TABLE & COLUMN INFO
24
• Table Definitiono Is it really a table or is it a view?o Get size of tables.
• Quick way: mysqlshow –status database {table} {column}.
• Examine Columns in Where Clauseo Cardinality of columns / data skew.
• Review the ‘Selected’ Columns o Use of ‘*’ or scalar functions.
23© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
• Know existing indexes. o Columns those indexes contain.o If multi-column, know the left leading column.o Cardinality.
• Make sure the optimizer can use the index.o Functions on indexed columns can turn off index.
• Consider rewriting.o Look for implicit conversions.
• Get sample parameter values.• Review existing keys and constraints.
o Know Multi-Table Relationships (ERD).• Primary and foreign key definitions.
o Check and not null constraints.• Tip: Keys & constraints help create better execution plans.
6. REVIEW INDEXES, KEYS & CONSTRAINTS
25© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
REVIEW INDEXES, KEYS & CONSTRAINTS – CONT.
26• No relationships?
© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
• Need to know the size of the actual data sets of each step.o In joins (right, left, outer).o What are the filtering predicates?o When is each filtering predicate applied?
• Try to filter earlier rather than later.• Compare size of final result set with data examined.
o Find the driving table. • To reduce rows examined.
7. UNDERSTAND SELECTIVITY
27© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
UNDERSTAND SELECTIVITY – CONT.
• Who registered yesterday for SQL Tuning?
SELECT s.fname, s.lname, r.signup_dateFROM student sINNER JOIN registration r
ON s.student_id = r.student_idINNER JOIN class c
ON r.class_id = c.class_idWHERE c.name = 'SQL TUNING'AND r.signup_date BETWEEN
DATE_SUB('@BillDate', INTERVAL 1 DAY) and '@BillDate'AND r.cancelled = 'N';
28 27© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
SELECTIVITY – USE SQL DIAGRAMMING
• Great Book “SQL Tuning” by Dan Towo Great book that teaches SQL Diagramming.o http://www.singingsql.com
29
registration
student class
5
1
30
1
5%
.2%
select count(*) from registration rwhere r.signup_date BETWEEN DATE_SUB('@BillDate', INTERVAL 1 DAY) and '@BillDate' AND r.cancelled = 'N';
4,228 / 79,981 * 100 = 5.2%
select count(*) from class where name = 'SQL TUNING'
2 / 1,000 * 100 = .2%
© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
8. TUNE THE QUERY
• Focus on most expensive operations first.o Try to reduce high-cost steps.o Read less rows.
• Filter earlier than later.• Find the driving table .• Consider a more selective or covering index.
• Seeks vs scans. Which is more expensive?o Implicit conversions?
• Review filesorts / temporary tables.o Group by.o Order by.
30© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
REVIEW INDEXES, KEYS & CONSTRAINTS – CONT.
31
• No relationships? © 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
ADD & VIEW FOREIGN KEYS
ALTER TABLE registration ADD FOREIGN KEY (student_id) REFERENCES student(student_id);ALTER TABLE registration ADD FOREIGN KEY (class_id) REFERENCES class(class_id);
SELECT table_name,column_name,constraint_name,referenced_table_name,referenced_column_name
FROM information_schema.key_column_usageWHERE table_schema = 'csu' AND table_name = 'registration'AND referenced_column_name IS NOT NULL;
32© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
ANY CHANGE?
33
F Keys added
© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
PLAN DIDN’T CHANGE – MORE DATA
34
DRIVE THE QUERY BY CLASS
• CREATE INDEX class_nm ON class(name);
35© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
DID WE IMPROVE IT?
36© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
• Try to reduce Logical IO (rows examined).• Adding indexes is not always the right thing to do!
o Consider insert, update & delete activity.• Covering index.
o CREATE INDEX cidx_registration ON registration(class_id, signup_date, cancelled);• Include all columns in query.• Reads only the index. It doesn’t go back to table.
• Partial index.o ALTER TABLE class ADD INDEX (description(3));
• description varchar(200)o SELECT * FROM class WHERE description LIKE 'SQL%‘;
9. ADJUST THE INDEX
37© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
ADJUST THE INDEX – CONT.
• Create an index on a Virtual Column in MySQL 5.7.7.o Kind of like a function index. o Persists the data in an index, NOT the table.
SELECT signup_dayofweek, count(*)FROM vregistration r WHERE r.cancelled = 'N‘GROUP BY signup_dayofweek HAVING count(*) < 10000;
• Consider other specialized index types.o Fulltext.o Spatial. o Hash for memory engine.o All beyond scope of this presentation.
38© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
• Make small changes.
o Re-run & check run-time details.
o Compare results with baseline metrics.
o Use ‘Rows Examined vs Rows Sent’ as a key measurement.
o Did you improve it? No? Rinse and repeat.
10. RE-RUN THE QUERY
39
• CREATE INDEX cidx_registration ON registration(class_id, signup_date, cancelled);
ADD COVERING INDEX
40© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
• Can’t cover it because of range.
AND r.signup_date BETWEEN DATE_SUB('@BillDate', INTERVAL 1 DAY) and '@BillDate'
IS THE COVERING INDEX WORKING?
41© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
• Monitor the improvement.o Be able to prove that tuning made a difference.o Take new metric measurements.o Compare them to initial readings.o Brag about the improvements. No one else will.
• Monitor for next tuning opportunity.o Tuning is iterative.o There is always room for improvement.o Make sure you tune things that make a difference.
• Shameless product pitch: DPA.
11. MONITOR YOUR TUNING RESULTS
42© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
DID WE IMPROVE IT?
43© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
Covered Index added
No improvement?
12. ENGINEER OUT THE STUPID
• Look for performance inhibitors.o Cursor or row by row processing. o Parallel query processing.
• Not always bad but…• Percona_Alex_Rubin_increasing-slow-query-performance-with-parallel-query-execution
o Hard-coded hints.• logicalread.solarwinds.mysql-query-hints-improve-performance
o Nested views.• Also, sub-queries.
o Abuse of Wild Cards (*) or No Where Clause.o Code-based SQL Generators (e.g. PHP generator, LINQ; nHibernate).o Implicit data conversions.o Non-SARG-able / scalar functions.
• Select… where upper(first_name) = ‘JANIS’44© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
DATA TYPES
• Signup_Date is a DATE not TIMESTAMP.• Change between clause to ‘=‘AND r.signup_date BETWEEN
DATE_SUB('@BillDate', INTERVAL 1 DAY) and '@BillDate‘----------------AND r. signup_date = DATE_SUB ( ‘@BillDate’ , INTERVAL ? DAY )
45© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
FIX THE QUERY
46© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
NOW DID WE IMPROVE PERFORMANCE?
47© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
ANOTHER STUPID EXAMPLE
48© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
SLOPPY CODING – NEED TO FIX QUERY
49© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
CHANGED THE CODE TO last_name LIKE ‘GRIFFIN%’
50© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
SUMMARY OF THE 12 STEP PROGRAM
44
Download Poster at: SQL Server - 12_Step_Tuning1. Find Which SQL to tune.
2. Gather run-time details.
3. Get execution plan.
4. Examine the execution plan.5. Review table and column info.6. Review indexes.7. Understand selectivity.8. Tune the query.9. Adjust the index.10. Re-run the query.11. Monitor tuning results.
12. Engineer out the stupid.
A 12 Step Program for Cats
MySQL - 12_Step_Tuning
© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
www.solarwinds.com/dpa-download/
RESOLVE PERFORMANCE ISSUES QUICKLY
• Try Database Performance Analyzer FREE for 14 days.• Improve root cause of slow performance.
o Quickly identify root cause of issues that impact end-user response time.o See historical trends over days, months, and years.o Understand impact of VMware® performance.o Agentless, installs in minutes.
© 2016 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
53© 2017 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.