the key to effective analytics: fast-returning queries
TRANSCRIPT
THELINEUP
ANALYST:
DezBlanchfieldDataScientist,TheBloorGroup
ANALYST:
RobinBloorChiefAnalyst,TheBloorGroup
GUEST:
BullettManaleDirectorofSalesEngineering,IDERA
The DBA
For sophisticated databases (Oracle, SQL Server, DB2, etc.) database tuning is a tough job,
but a SECURE job
file://localhost/Users/robinbloor2/Dropbox/~Desktop/bottlenecks.jpg
The Database Landscape
u Relational databases still dominate in popularity (> 95%)
u MS SQL Server is second in popularity to Oracle
u Used both for OLTP and query workloads
u Alternatives (NoSQL & Graph) used mainly for analytics
u Most companies standardize and use alternatives only in exceptional circumstances
u Databases are “strategic” and they evolve
u Neither Hadoop nor Spark is a database
file://localhost/Users/robinbloor2/Dropbox/~Desktop/bottlenecks.jpg
DBA Activities
u Install, config, upgrade
u License management
u ETL & Replication
u Storage & Caplan
u Troubleshooting
u Performance monitoring & tuning
u Security
u BU & Recovery
u S/W Test systems
u Data lifecycle
file://localhost/Users/robinbloor2/Dropbox/~Desktop/bottlenecks.jpg
Database Performance Bottlenecks
u CPU: CPU overload, context switches, I/O waits, CPU cache, Backplane throughput
u Memory: Thrashing, fragmentation
u Disk: Caching, I/O saturation, fragmentation, SSD performance
u Network Overhead (SANs, NAS, local disk)
u Locking
file://localhost/Users/robinbloor2/Dropbox/~Desktop/bottlenecks.jpg
Hardware Factors
u CPUs, GPUs & FPGAs
u Cross breeding
u 3D Xpoint and PCM
u SSDs & parallel access
u Parallel hardware architectures
At the hardware level PERFORMANCE is accelerating and COSTS continue to fall.
file://localhost/Users/robinbloor2/Dropbox/~Desktop/bottlenecks.jpg
Database Issues
u Database entropy (proliferating instances) is very common – it’s the norm, with databases being used as “cupboards”
u Few databases are self-tuning or properly tuned (especially the ones that perform)
u SQL needs to be tuned (particularly for JOINS)
u Most companies employ few DBAs (they’re expensive)
@dez_blanchfield
Keychallengesimpac/ngDatabaseAnaly/csperformance
§ InfrastructurePerformanceMonitoring
§ KeepingontopofQueryMonitoring
§ Monitoringresponse6mestopla8orm&user
§ Addressingpoorlystructuredqueries
§ Itera6vequery/review&finetuningworkflows
§ Keepingontopofdeadlocksandblocks
§ ShiFfromHistoricaltoPredic6vealer6ng
Typicaldaytodayperformancehits..
@dez_blanchfield
Performanceissueswhichregularuserscan’teasilyresolve
§ Performanceofcomplexcodeyoudidn’twrite
§ Pre-stagingofdataforperformancewherepossible
§ Dele6ngindexesbeforebulkimportsthenre-index
§ Op6mizeindexesregularlyoraFerabulkimport
§ Limitqueriestojustnumberofcolumnsyouneed
§ Knowinghow&whentousetemptables
§ Batchupdelete&updateworkloadsefficiently
§ Detec6ng&addressingforeignkeyconstraints
Issuesinvisibletomostmeremortals..
@dez_blanchfield
Itbehoovesustofindbe@er
waystoperformreal-%me
PlaJormPerformance
Diagnos%csintelligently
© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.
IDERA - SQL DIAGNOSTIC MANAGER Hot Technology, The Key to Effective Analytics: Fast-Returning Queries Bullett Manale Director, Sales Engineering
© 2015 IDERA, Inc. All rights reserved. Proprietary and confidential. 3
§ FOCUS: SQL DIAGNOSTIC MANAGER
© 2015 IDERA, Inc. All rights reserved. Proprietary and confidential. 4
§ Data Manager § DBA § Acting DBA (or most technical person in the room)
TYPICAL USER OF SQL DIAGNOSTIC MANAGER
© 2015 IDERA, Inc. All rights reserved. Proprietary and confidential. 5
RESPONSIBILITIES OF THE DBA (ACCORDING TO WIKIPEDIA)
6 © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.
USUALLY MORE TO A QUESTION THAN MEETS THE EYE
‘Loaded’ Questions: What is my worst performing query? | (Reads, Writes, CPU, Frequency, Waits, Duration)
Show me blocking incidents within my production environment? | (Blocking by count, by blocking time) When will I run out of space? | (Drive space, Database space, VM Host Space, Partition) What is my worst instance of SQL Server running? | (CPU, Waits, I/O, Memory, Alerts, etc..) What is my worst performing application? | (CPU, I/O, by user, by client, by database) What caused my problem that happened last night? | (What time last night, CPU, I/O, Non-responsive) What can I do proactively to improve my performance? (Totally Open Ended)
7 © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.
TYPICAL REQUIREMENTS OF A GOOD ANALYTICS TOOL
BE ABLE TO…..
…collect the information at little to no cost
…collect the appropriate data and leave out the unnecessary data
…once collected, access the data quickly and efficiently
…easily understand the data or provide additional resources that help educate
...easily share the information with others using reports, exporting, and integration
…easily drill even deeper into the data if it is called for
8 © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.
SQL DIAGNOSTIC MANAGER CORE DELIVERABLES
§ Manage at the Enterprise Level – DBAs typically manage a lot of database instances. Having a view that represents the health of each instance in a single window is vital. This provides peace of mind for the DBA in that they can quickly glance at a screen to know if everything is running as it should be; if it’s not, then to have the ability to drill down and identify and troubleshoot. The tool also needs to be presented in a way that is familiar to the day to day operations of the DBA (DBA Centric View).
9 © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.
SQL DIAGNOSTIC MANAGER CORE DELIVERABLES
§ Ability to Look Into the Past – A DBA can’t continuously stare at a console and wait for problems to happen. Problems happen during meetings, in the middle of the night and off hours. Having a tool to help see the state of a server when a problem happened in the past is vital. Additionally, the historical data collected can be used for other purposes than just troubleshooting. For example: What is my fastest growing database over the last year?
10 © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.
SQL DIAGNOSTIC MANAGER CORE DELIVERABLES
§ Query Performance and Accountability – The DBA is often left holding the bag whenever there is a performance related issue. The perception is that if the database is the slowdown (or calls to the database) then it must be the DBAs fault. Having the ability to identify performance issues down to the command level is a vital part of what SQL Diagnostic Manager does.
11 © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.
SQL DIAGNOSTIC MANAGER CORE DELIVERABLES
§ Ability to be Proactive – Proactive means different things to different people. For DBAs, in most cases, it means taking corrective action before the user experience is diminished; or even going so far as to automate the remediation of known issues automatically. SQL Diagnostic Manager can help with all of these.
12 © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.
SQL DIAGNOSTIC MANAGER CORE DELIVERABLES
§ Overall Flexibility – Every environment is different. Out-of-the-box SQL Diagnostic Manager has canned metrics that are monitored and collected historically for review. The same holds true for reports. Having the ability to extend this monitoring footprint is paramount, especially for larger enterprise shops. Obscure monitoring requirements will be necessary. Having a tool that adapts to the environment is crucial.
13 © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.
ARCHITECTURE OF SQL DIAGNOSTIC MANAGER
SQLdm Thick
Consoles
SQLdm Services
SQLdm Repository (SQL server
2005+)
Management Service Collection Service
Active
Android iOS
Windows
• Nothing installed on the monitored SQL instance • No agents, stored procedures, etc. • Monitor physical and virtual databases
IDERA core services
SQL Diagnostic Manager SQLSafe Backup* SQL Enterprise Job Manager*
SQLdm & Other Web
Consoles * Sold separately * Sold separately
SQLdm Mobile
Licensed per instance
Now Supports
2016!
14 © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.
INTRODUCING SQL WORKLOAD ANALYSIS (PART OF SQL DIAGNOSTIC PRO BUNDLE)
15 © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.
QUICK DEMONSTRATION
SQL DIAGNOSTIC MANAGER