sql tuning for everyone jai maa durga
TRANSCRIPT
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
1/38
1
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
2/38
SQL Tuning for Smarties, Dummies and Everyone in Between
Novices
Jagan Athreya
Director, Database Manageability, Oracle
Arup Nanda
2
Senior Director, Database Architecture, Starwood Hotels and Resorts
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
3/38
The following is intended to outline our general
product direction. It is intended for information
purposes only, and may not be incorporated into any
material, code, or functionality, and should not be
relied upon in making purchasing decisions. , ,
features or functionality described for Oracles
products remains at the sole discretion of Oracle.
3
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
4/38
Outline
SQL Tuning Challenges
SQL Tuning Solutions New Feature Overview
Problem Root Causes and their Solutions
Preventing SQL Problems
Q & A
4
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
5/38
SQL Tuning Challenges
-
DBA team
Mostly average, some superstars
Superstars take most of the burden over-stretched
Development staff
Mostl non-Oracle skills Java, C++
Usually considers the DB as a black box
Writing efficient queries, troubleshooting performance issues
5
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
6/38
SQL Tuning Challenges
Situation:
Query from hell pops up
Brings the database to its knees
Response
DBA: Developer should be taking care of this.
Developer: Why is the DBA not aware of this problem?
Manager: DBA will review all queries and approve them.
What is the most efficient way to manage this process?
6
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
7/38
SQL Tuning Challenges
Situation
New SQL statements added as part of application patch
deployment
Database u rades
Database patching
Response Users: How will the application perform after the changes?
DBA: How do I ensure that our SLA remains intact after the
changes are rolled out? Challenge
How to reduce business risk while absorbing new
7
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
8/38
SQL Tuning Challenges
Situation
Data in Production has evolved over time. Have the optimizer
statistics stayed current?
DBA:
Will statistics refresh break something?
What will happen if we dont collect?
How often should I collect the statistics ?
What happens when you collect a new set? Challenge
What is the recommended strategy for managing optimizer
8
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
9/38
SQL Tuning Challenges
No time to find the root cause. How to prevent this
from recurring?
Bind variables: How do you prevent bad plans based
How to diagnose a bad plan
10053 trace, endless ourin over traces
Wrongly constructed predicates
How to fix a bad plan
n s c ange o co e
Baselines vs. SQL Profiles
Pick out a single SQL or a bunch from the shared pool
9
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
10/38
Outline
SQL Tuning Challenges
SQL Tuning Solutions New Feature Overview
Problem Root Causes and their Solutions
Preventing SQL Problems
Q & A
10
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
11/38
Real-Time SQL Monitoring
Lookin Inside SQL Execution
Automatically monitors longrunning SQL
Enabled out-of-the-box with noperformance overhead
Monitors each SQL execution
Exposes monitoring statistics
Global execution level
Plan operation level
Parallel Execution level
Guides tuning efforts
11
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
12/38
New capabilities in SQL Monitoring
New in Oracle Database 11 Release 2
PL/SQL monitoring including associated high load SQL monitored recursively
Exadata aware I/O erformance monitorin and associated metric data
Capture rich metadata such as bind values, session details e.g. user,program, client_id and error codes and error messages
Save as Active Report for rich interactive offline analysis
12
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
13/38
DEMO
13
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
14/38
Application Tuning
High-Load
Packaged Apps +SQL Profi le
Well-Tuned SQL
us om za e pps +SQL Advice
Customizable Apps +Indexes & MVs +Partitions
Appl ications
Automatic Tuning Opt imizer
Automatic SQL Tuning
Identifies high-load SQL from AWR
Tunes SQL usin SQL Profiles Implements greatly improved SQL plans (optional)
Performance benefit of advice provided
SQL Profiling tunes execution plan without changing SQL text
14
Enables transparent tuning for packaged applications
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
15/38
Automatic SQL Tuning
Gather Missing or Stale
Statistics
SQL Profiling
Statistics Analysis
Access Path Analysis
SQL Restructure Analysis
rea e a ro e
Add Missing Access
Structures
Modify SQL Constructs
Al ternative Plan Analysis
Parallel Query AnalysisAdministrator
Comprehensive
op erna ve
Execution Plan
Create Parallel SQL
Profile
SQL TuningAdvisor
SQL Tuning Advisor
-
OptimizerSQL Tuning
Recommendations
performance data
NEW: Recommends parallel profile if it will improve SQL performance
significantly (2x or more)
15
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
16/38
SQL Tuning for DevelopersIntegration with Visual Studio
Introduced in Oracle Developer Tools for Visual Studio Release 11.1.0.7.20 Oracle Performance Analyzer
Tune running applications with the help of ADDM
Query Window
Tune individual SQL statements with STA Server Explorer
16
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
17/38
Agenda
SQL Tuning Challenges
SQL Tuning Solutions New Feature Overview
Problem Root Causes and their Solutions
Preventing SQL Problems
Q & A
17
What makes SQL go bad?
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
18/38
What makes SQL go bad?Root Causes of Poor SQL Performance
1. Optimizer statistics issues
a. Stale/Missing statistics
b. Incomplete statistics
c. Improper optimizer configuration
d. U raded database: new o timizer
e. Changing statistics
f. Rapidly changing data2. Application Issues
a. ss ng access s ruc ures
b. Poorly written SQL statements
3. Cursor sharing issuesa. Bind-sensitive SQL with bind eekinb. Literal usage
4. Resource and contention issues
a. Hardware resource crunch
. on en on row oc con en on, oc up a e con en on
c. Data fragmentation
5. Parallelism issues
a. Not parallelized (no scaling to large data)
18
b. Improperly parallelized (partially parallelized, skews)
Wh t k SQL b d?
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
19/38
What makes SQL go bad?Root Causes of Poor SQL Performance
1. Optimizer statistics issuesa. Stale/Missing statistics
b. Incomplete statistics
c. Improper optimizer configuration
d. Upgraded database: new optimizer
e. Changing statistics
f. Rapidly changing data
2. Application Issues
3. Cursor sharing issues
4. Resource and contention issues
5. Parallelism issues
19
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
20/38
Oracle Optimizer Statistics
Inaccurate statistics Suboptimal Plans
Table Statistics
Column Statistics
Index Statistics
20
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
21/38
Oracle Optimizer StatisticsPreventing SQL Regressions
NoviceMode
Automatic Statistics Collection Job (stale ormissin
Out-of-the box, runs in maintenance window
Configuration can be changed (at tablelevel)
a ers s a s cs on user an c onaryobjects
Uses new collection algorithm with Nightly
sampling of 10%
Incrementally maintains statistics for
Set DBMS_STATS.SET_GLOBAL_PREFS
21
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
22/38
Oracle Optimizer StatisticsPreventing SQL Regressions
ExpertMode
Extended Statistics Extended Optimizer Statistics provides a mechanism
to collect statistics on a group of related columns:
Function-Based Statistics Multi-Column Statistics
Automatically maintained with column statisticsDBMS_STATS. CREATE_EXTENDED_STATS
Allows validation of statistics before publishing
Disabled by default
To enable set table/schema PUBLISH settin to FALSEDBMS_STATS. SET_TABLE_PREFS( ' SH' , ' CUSTOMERS' , ' PUBLI SH' , ' f al se' )
To use for validation
ALTER SESSI ON SET opt i mi zer _pendi ng_st at i st i cs = TRUE;
22
Publish after successful verification
What makes SQL go bad?
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
23/38
What makes SQL go bad?Root Causes of Poor SQL Performance
1. Optimizer statistics issues
2. Application Issues
.
b. Poorly written SQL statements
3. Cursor sharing issues
.
5. Parallelism issues
23
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
24/38
Identify performance problems using ADDMAutomatic Database Diagnostic Monitor
Novice
Provides database and
o e
diagnostic
Throughput centric - Focus
on reducing time DB time
Identifies top SQL:
Shows SQL impact
Frequency of
occurrence
Pinpoints root cause:
SQL stmts waiting for
SQL stmts not shared
24
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
25/38
Identify High Load SQL Using Top ActivityNovice
Performance Page
Identify Top SQL by DB Time:
CPU
o e
I/O
Non-idle waits
Different Levels of Analysis
Historical analysis
AWR data
Performance Page
Top Activity
Real-time analysis
ASH data
More ranular anal sis Enables identification of
transient problem SQL
Top Activity Page
25
Tune using SQL Tuning Advisor
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
26/38
Advanced SQL TuningUniverse of Access Structures
Novice+Mode
Indexes: B-tree indexes, B-tree cluster indexes, Hash cluster indexes, Globaland local indexes, Reverse key indexes, Bitmap indexes, Function-basedindexes, Domain indexes
Materialized Views:Primary Key materialized views,
ROWID materialized views
Complex materialized views
Partitioned Tables:
Range partitioning, Hash partitioning,
List partitioning, Composite partitioning,
n erva ar on ng, par on ng,
Virtual Column Based partitioning
B-tree index
26
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
27/38
SQL Access Advisor: Partition AdvisorNovice+
Mode
Indexes
Materialized
viewsSQL AccessAdvisor
RepresentativeWorkload
Materialized
views logs
Automatic Tuning
Optimizer
Partitioned
ob ects
Analysis
27
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
28/38
SQL Access AdvisorAdvanced Options
ExpertMode
Workload filtering Limited vs. advanced mode
Hypothetical workload tuning
Factoring in the cost of creation
28
What makes SQL go bad?
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
29/38
What makes SQL go bad?Root Causes of Poor SQL Performance
. p m zer s a s cs ssues
2. Application Issues
3. Cursor sharing issues
.
b. Bind-sensitive SQL with bind peeking
4. Resource and contention issues
29
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
30/38
What makes SQL go bad?a. Literal Usage Issue
ExpertMode
SELECT * FROM jobs WHERE min_salary > 12000;
SELECT * FROM jobs WHERE min_salary > 15000;
SELECT * FROM jobs WHERE min_salary > 10000;
SELECT * FROM
SELECT * FROM *
cursor_sharing ={exact, force, similar}
ar ngCursorsis good!
30
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
31/38
What makes SQL go bad?b. Bind Peeking Issue
NMode
Processed_FlagY
Full Table Scan
Y
Y CBO10gFTS
99
Two different optimal plansfor different bind values
Index Range Scan
N
IRS
1
31
Problem: Binds will affect optimality in any subsequent uses of the stored plan
Fi i bl ith Ad ti C Sh i
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
32/38
Fixing problems with Adaptive Cursor SharingAdaptive Cursor Sharing Expert
SELECT * FROM emp WHERE wage := wage_value
1
2
20 25
SamePlan
3
22 24
DifferentPlan
4
30 35
SamePlan,
ExpandInterval
32
34 43
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
33/38
Agenda
SQL Tuning Challenges
SQL Tuning Solutions New Feature Overview
Problem Root Causes and their Solutions
Preventing SQL Problems
Q & A
33
P ti bl ith SQL Pl M t
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
34/38
Preventing problems with SQL Plan Management
Problem: changes in the
environment cause plans to changeNL
GB
Parse
Plan baseline is establishedNL
Statement log
Plan history
SQL statement is parsed again and
a different plan is generated
HJ
GB
Plan baselineGB
NL
NL
New plan is not executed butHJmar e or ver ca on
34
SQL Plan Management
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
35/38
SQL Plan ManagementMigration of Stored Outlines to Plan Baselines
Oracle Database 11g
GB 5. Migrate Stored OutlinesPlan History
GB
OH Schema
No plan
regressions
HJ
HJ
HJ
HJHJ
HJ
4. Upgrade
to 11g
Oracle Database 11g
OH Schema
HJ
GB
CREATE_STORED_OUTLINES=true. eg n
with
2. Run all SQL in the
Application and auto
Oracle Database 9 or 10g
HJ
CREATE_STORED_OUTLINES=false
crea e a ore u ne
for each one
3. After Store
Outlines are
ca tured
35
SQL Performance Analyzer (SPA)
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
36/38
SQL Performance Analyzer (SPA)Validate statistics refresh with SPA
Steps:
1. Capture SQL workload in STS
Validating upgrade with SPASQL Workload
capture capability
2. Execute SPA pre-change trial3. Refresh statistics using
SQL plans + stats SQL plans + stats
opt on
4. Execute SPA post-change trial
5. Run SPA report comparing SQL Compare
- -
Before PUBLISHing stats:
Remediate individual few SQLAnalysis Report
SQL Performance
for plan regressions: SPM, STA
Revert to old statistics if toomany regressions observed
36
C l i
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
37/38
ConclusionIdentify, Resolve, Prevent
PreventSPASPM
1. Production Performance
2. Change Causing Problems
3. Optimizer Statistics Management
. a p ans agnos s an eso u on
ADDM, Top Activity, SQL Monitoring
Identify
,
Access Advisor,
Auto Stat Collect ion
Top Activity,
ADDM,
SQL Monitoring
37
-
8/11/2019 SQL Tuning for Everyone JAI MAA DURGA
38/38