an introduction to oracle sql tuning

8/3/2019 An Introduction to Oracle SQL Tuning

1/97

31/01/2012 1

An IntroductionTo Oracle SQL

TuningChris Adkin30th May 2008


2/97

2

Some Inspirational Thoughts

Before We Begin . . . The following Ask Tom excerpt comes in response to a You Asked Can

u give a methodology of tuning the sql statements. question.

The link to the full answer is at:

http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:8764517459743

Despite being five years old, in the intervening time, artificial intelligencehas not been built into the RDBMS where by it can re-write your SQLsuch that it will run in the most efficient manner possible.

Advisors take some of the leg work out of tuning and the tools such as

DBMS_XPLAN, v$ views etc constantly change, evolve and improve,however SQL tuning and writing efficient SQL is not a prescriptiveprocess that can be captured in a process.

I will however try to present useful techniques and good practise to de-mystify some of the art behind this.


3/97

3


Before We Begin . . . 1.1 Efficient SQL This was probably the hardest

part of the book to write - this chapter. That is

not because the material is all that complex,

rather because I know what people want - and I

know what can be delivered. What people want:

The 10 step process by which you can tune any

query. What can be delivered: Knowledge about

how queries are processed, knowledge you can

use and apply day to day as you develop them.

Think about it for a moment.


4/97

4


Before We Begin . . . If there were a 10 step or even 1,000,000 step

process by which any query can be tuned (or

even X% of queries for that matter), we would

write a program to do it. Oh don't get me wrong,there are many programs that actually try to do

this - Oracle Enterprise Manager with its tuning

pack, SQL Navigator and others. What they do

is primarily recommend indexing schemes to

tune a query, suggest materialized views, offer

to add hints to the query to try other access

plans.


5/97

5


Before We Begin . . . They show you different query plans for the same

statement and allow you to pick one. They offer "rules

of thumb" (what I generally call ROT since the acronymand the word is maps to are so appropriate for each

other) SQL optimizations - which if they were

universally applicable - the optimizer would do it as a

matter of fact. In fact, the cost based optimizer doesthat already - it rewrites our queries all of the time.

These tuning tools use a very limited set of rules that

sometimes can suggest that index or set of indexes you

really should have thought of during your design.


6/97

6


Before We Begin . . . I'll close this idea out with this thought - if there

were an N step process to tuning a query, to

writing efficient SQL - the optimizer wouldincorporate it all and we would not be having a

discussion about this topic at all. It is like the

search for the holy grail - maybe someday the

software will be sophisticated enough to beperfect in this regards, it will be able to take our

SQL, understand the question being asked and

process the question - rather then syntax.


7/97

31/01/2012 7

Section 1Mechanics OfThe Cost BasedOptimizer


8/97

8

What Is The Optimizer I will focus on the Cost Based Optimizer.

Around since Oracle 7.

Devises the best plan for executing a query basedon cost.

Transparent to application and users, except when

the wrong plan is selected.

The stages of optimisation will be covered on thenext set of slides.


9/97

9

Stages Of Optimisation

On 10g parsed SQL statements are assigned uniqueidentifiers called sql ids, prior releases use a combination ofSQL address and hash value.

If a query does not exist in parsed form in the shared pool,a hard parse or optimisation takes place (a gross oversimplification, but basically what happens).

Oracle can also perform optimisations at run time.

A parsed SQL statement (cursor) can have multiple childcursors, these are affectively different plans for the sameSQL text when different variables are supplied (an oversimplification, but basically what happens).


10/97

10


Establish the environment

Ascertain what parameters are set

Ascertain what bug fixes are in place based on thesetting of the compatible parameter

Query transformation Sub query un-nesting

Complex view merging Set Join Conversion

Predicate move around


11/97

11


Establish the base statistics of all the relevanttables and indexes.

Single table access cardinality estimation.

Join order consideration.

Record will be kept of the best join order maintainedso far.


12/97

12

What Influences The Cost Based

Optimizer ? Object statistics

Object data types

Oracle initialisation parameters, refer to:-

v$sys_optimizer_env

v$ses_optimizer_env

V$sql_optimizer_env


13/97

13


Optimizer ? System statistics:-1* select * from sys.aux_stats$

SQL> /

SYSSTATS_INFO STATUS COMPLETED

SYSSTATS_INFO DSTART 11-26-2006 19:53

SYSSTATS_INFO DSTOP 11-26-2006 19:53

SYSSTATS_INFO FLAGS 1

SYSSTATS_MAIN CPUSPEEDNW 1081.76881

SYSSTATS_MAIN IOSEEKTIM 10

SYSSTATS_MAIN IOTFRSPEED 4096

SYSSTATS_MAIN SREADTIM

SYSSTATS_MAIN MREADTIMSYSSTATS_MAIN CPUSPEED

SYSSTATS_MAIN MBRC

SYSSTATS_MAIN MAXTHR

SYSSTATS_MAIN SLAVETHR


14/97

14


Optimizer ? System statistics facilitate something called the

CPU costing model introduced in Oracle 9i.

Until this came along the optimizer did not take into account CPU

performance nor the difference in performance between single andmulti block reads.

In Oracle 9i, no system statistics are present.

In 10g out of the box statistics called no workload statistics areprovided.

Stats that are not populated in aux_stats$ only appear after systemstatistics have been gathered.

The optimizer cost model can be set via _optim_cost_model, set this toIO or CPU.


15/97

15


Optimizer ? !!! Warning !!!

You have to be very careful when gathering system statistics.

I

f you have anyI

/O

sub system with a cache you can end up gettingsingle block and multi block I/O times that reflect the speed of thecache rather than the under lying disks.

This can seriously skew a plans costings.

Oracle supply an I/O calibration tool called Orion which might helphere:http://www.oracle.com/technology/software/tech/orion/index.html


16/97

16


Optimizer SQL Usage

Unless the NO_MONITOR hint is used, col_usage$

in the data dictionary will be updated whenever astatement is parsed.

DBMS_STATS uses this in order to determinewhether a histogram should be created on a columnwhen size = AUTO is specified in the method_opt.

Oracle can get confused with what to do forcolumns where the LIKE and col_usage$ and hencenot create histograms when size auto is specified.


17/97

17


Optimizer Hints and profiles

SQL hints

Stored out lines (essentially hints)

SQL profiles as created by the Oracle 10g tuningadvisor

Objects, presence of indexes, partitioning,constraints etc ...


18/97

31/01/2012 18

Section 2A Plan Of Attack ForInvestigating A Bad Plan


19/97

19

Identify The SQL To Be Tuned

Many ways of doing this:-

Automatic Database Diagnostic Manager

$ORACLE_HOME/rdbms/admin/addmrptAutomatic Workload Repository reports

$ORACLE_HOME/rdbms/admin/awrrpt

Toad

SQL Tuning Advisor$ORACLE_HOME/rdbms/admin/sqltrpt


20/97

20

A Simple Script For Identifyng

SQL To Be Tuned SELECT *

FROM (SELECT sql_id,

elapsed_time,

cpu_time,

user_io_wait_time,

concurrency_wait_time

FROM v$sql

ORDER BY elapsed_time DESC)

WHERE rownum


21/97

21

Causes Of Poor Performance

Stale or missing statistics (includes missing histograms).

Lack of appropriate indexes to support your query

Bugs in the optimizer, there are the best part of 100 bugfixes related to the optimizer in 10.2.0.4.0.

Optimizer flaws, such as the predicate independenceassumption, more on this later.

SQL that is inefficient by design. SQL that is optimizer unfriendly, more on this later.


22/97

22

Causes Of Poor Performance

Index, table type or partitioning schemes that arenot conducive to good performance with the query

mix you are running. Misuse or poor use ofOracle features.

Abuse of the Oracle optimizer environment.

Poor schema design.


23/97

23

The Tuning Ethos

Ask the following questions:-

Has what I need to tune ever ran fast, if so what

has changed in the environment.Do not confuse something that is running slow

because it has an abnormally high workload forsomething that is inefficient at what it does.

Is the part of the application running slowly becauseit is not designed to deal with the shape and / orsize of the data it is processing ?.


24/97

24

The Tuning Ethos

When tuning Only make one change at a time so as to be able to

accurately gauge what affects your query.

Try to make the scope of any changes match the scope of theproblem, e.g. if you have a problem with one query, a global

change such as a change to an initialisation parameter isunlikely to address the root cause of the issue and it maycause performance regression for queries that are currentlyperforming well.

Use realistic test data. Try to use a clean environment without factors that may skew

your results, e.g. for accurate and consistent response timesavoid database and servers which are being used by otherusers if possible.


25/97

25

The Tuning Ethos

Tuning a piece of SQL may result in:-

Indexes being added or modified.

Statistic gathering regimes being modified.

SQL being re-written.

Object types being changed, e.g. heap to indexorganized tables.

Schema design changing, e.g. extra columns added

to tables in order to reduce numbers of joins. The use of hints.


26/97

26

Tuning Is Dead We Have The

Tuning Advisor !!! Based on my experience this:-

Recommends indexes

Identifies stale statisticsNotifies you of Cartesian joins

Spots predicates not conducive to the efficient useof indexes, e.g. col , col NOTIN (1, 2, ...

Identifies parts of statements where view merginingcannot take place.

Recommends SQL profiles.


27/97

27

Tuning Is Dead We Have The

Tuning Advisor !!! If the tuning advisor does anything else, I am yet to see it.

SQL profiles are enhanced hints that provide the optimizerwith extra information with which to get cardinalityestimates correct with.

The use of SQL profiles can cause a problem if your basedata changes such that they are no longer relevant.

It is the best tool of its type that Oracle have produced to

date, it is not artificially intelligent and it smacks of theROT tools that Tom Kyte mentioned.

This is good for a first pass at identifying more obviouscauses of a query running sub optimally.


28/97

28

A Simple MethodologyFor Query Tuning

Despite what the Oracle marketing machine maysay, tuning is not a prescriptive process.

The tuning advisor in its 10g incarnation is no

substitute for a knowledgeable DBA at best it semiautomates the task of tuning.

Some people instantly assume that because aplan contains a full table scan that the execution

plan is therefore bad. This is an out dated method of tuning with origins

based around the rule based optimizer.


29/97

29

Good And Bad Plans

Do not automatically jump to the conclusion that a bad plancontains full table scans and a good plan only uses indexrange scans.

Reasons for full table scans might include:-

Queries that use un-selective predicates, I.e. result in all or alarge proportion of a tables data being returned.

Queries return most of the columns in a table.

The table is small in size.

The indexes you think the query should use have poorclustering factors.


30/97

30


What I will outline is a plan of attack for tacklingquery issues.

There is no such thing as a definitive tuningmethodology as alluded to by Tom Kyte.

Even when Oracle support is engaged with atuning related service request, you will not alwaysbe asked for the same information or to use thesame diagnostic tools depending on who the

service request is assigned to. This may change with the DBMS_SQLDIAG

package in 11g, aka the SQL test case builder.


31/97

31


My plan of attack will not cover the writing ofefficient SQL, this would require a differentpresentation entirely.

I will assume:- All reasonable indexes, partitioning schemes etc ... are in

place.

Nothing daft has been done to abuse the optimizerenvironment.

Oracle has produced sub optimal plans due todiscrepancies between predicated cardinalities and actualcardinalities.


32/97

32


1. Obtain the execution plan for your query

If you run an explain plan from the SQL prompt this will giveyou the plan that Oracle predicts the statement will use, I.e.:-

SQL> EXPLAIN PLAN FOR SELECT * FROM dual

SQL> SELECT * FROM table(dbms_xplan.display)

A better way of explaining the execution plan is by taking it straightfrom the shared pool:-

SQL> SELECT * FROM

table(dbms_xplan.display_cursor(sqlid))

Sql ids are new to Oracle 10g and uniquely identify parsed

SQL statements


33/97

33


1. Obtain the execution plan for your query

The DBMS_XPLAN package is the best tool here.

Where possible obtain the plan after running the query:SELECT * FROMTABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL, NULL, ALLSTATS LAST)

Obtain the plan from the shared poolSELECT * FROMTABLE(DBMS_XPLAN.DISPLAY_CURSOR()

Obtain the plan from the workload repositorySELECT * FROMTABLE(DBMS_XPLAN.DISPLAY_AWR()

If you use explain plan, this will give you what the optimizerpredicts the plan will be, this may not necessarily but what the planis when the query runs due to such things as bind variablepeeking.


35/97

35


2. Visually sanity check your execution plan

Does the plan contain any Cartesian joins ?.

Is the most restricted table the driving table, this isthe first table in the join order and usually the tablethat is furthest indented into the plan, I.e. the tablefor which the filter predicates will reduce the most.

Is the access for the driving table via a full tablescan, not always an issue, but could indicate that

the join order is incorrect

Continued ...


36/97

36



Are you battling against the schema design, an example:-SELECT DISTINCT A.ID_JOB

FROM TR_JOB A,

TR_OFFCYCLE_JOB_DETAILS F,

TR_CYCLIC_JOB_DETAILS C,

TR_CONSUMPTION_HISTORY B

WHERE A.ID_JOB = B.ID_JOB

AND A.ID_SERVICEPOINT =:1

AND A.SD_JOB_STATUS = :2

AND B.SD_READING_TYPE = :3

AND (( F.APPMT_START_TIME < to_date(:4,'yyyy-MM-dd hh24:MI:SS')

AND F.APPMT_END_TIME > to_date(:5,'yyyy-MM-dd hh24:MI:SS')

AND F.ID_JOB=A.ID_JOB )

OR ( C.VISIT_DATE_FROM < to_date(:6,'yyyy-MM-dd hh24:MI:SS')

AND C.VISIT_DATE_TO > to_date(:7,'yyyy-MM-ddhh24:MI:SS')

AND C.ID_JOB=A.ID_JOB ))

----------------------------------------------------------------------------------------------

| Id | Operation | Name | Rows | Bytes | Cost |

----------------------------------------------------------------------------------------------

| 0 | SELECT STATEMENT | | 1 | 151 | 1187 |

| 1 | HASH UNIQUE | | 1 | 151 | 1187 |

| 2 | NESTED LOOPS | | 1 | 151 | 1176 |

| 3 | MERGE JOIN CARTESIAN | | 1 | 104 | 684 |

| 4 | NESTED LOOPS | | 1 | 51 | 665 |

|* 5 | INDEX FAST FULL SCAN | TR_CONSUMPTION_HISTORY_IX2 | 1 | 5 | 663 |

|* 6 | TABLE ACCESS BY INDEX ROWID| TR_JOB | 1 | 46 | 2 |

|* 7 | INDEX UNIQUE SCAN | TR_JOB_ID_PK | 1 | | 1 |

| 8 | BUFFER SORT | | 32474 | 1680K| 682 |

| 9 | TABLE ACCESS FULL | TR_OFFCYCLE_JOB_DETAILS | 32474 | 1680K| 19 |

|* 10 | TABLE ACCESS FULL | TR_CYCLIC_JOB_DETAILS | 1 | 47 | 492 |

----------------------------------------------------------------------------------------------

Continued ...


37/97

37



In the previous example there are two issues:-

There is a Cartesian join due to the OR condition,specifically if one of the branches evaluates to

TRUE the join predicates do not get performed fortwo tables.

We have an index fast full scan (index equivalent ofa full table scan) on

TR_CONSUMPTION_HISTORY_IX2, note that theonly reason we are going to theTR_CONSUMPTION_HISTORY table is to get thecontents of the sd_reading_type column.


38/97

38

A Simple Methodology

For Query Tuning3. Check the environment is correct for the query:-

Are statistics up to date for all the objects used by the query.

Do appropriate indexes exists, you can run the sqltrpt scriptfrom $ORACLE_HOME/rdbms/admin, supply the sql id foryour query and the tuning advisor will go to work for you, thiswill also highlight any tables with stale statistics.

Continued . . .


39/97

39

A Simple Methodology For Query Tuning

3. Checking the environment is correct for the query

From Greg Rahn of the Oracle Real World performance group:-

I think its important to understand what variables influence the Optimizer in order to

focus the debugging effort. There are quite a number of variables, but frequently the

cause of the problem ones are: (1) non default optimizer parameters and (2) non-

Representative object/system statistics. Based on my observations Iwould say that

the most abused Optimizer parameters are:

OPTIMIZER_INDEX_CACHING

OPTIMIZER_INDEX_COST_ADJ

DB_FILE_MULTIBLOCK_READ_COUNT


40/97

40

A Simple Methodology For Query Tuning

3. Checking the environment is correct for the query

Many see setting these as a solution to get the Optimizer to choose an index plan

over a table scan plan,but this is problematic in several ways: This is a global change to a local problem Although it appears to solve one

problem, it is unknown how many bad execution plans resulted from this

change.

The root cause of why the index plan was not chosen is unknown, just that

tweaking parameters gave the desired result

Using non-default parameters makes it almost impossible to correctly andeffectively troubleshoot the root cause

A Si l M h d l


41/97

41


4. Understand your data, how Selective is it ?

Are the predicates and variables used in your statementgoing to retrieve a significant or small proportion of rowsfrom the queries base tables ?.

The following predicates come from a query I haveinvestigated the performance of:-

UPPER (a.walkroute_reviewed_flag) = 'Y'

( ( a.sd_service_type = '10000'

AND a.sd_servicepoint_status = '10001' )

OR ( a.sd_service_type = '10001'

AND a.sd_occupied_status = '10001'

AND a.sd_servicepoint_status = '10001' )

OR a.sd_servicepoint_status = '10000'OR a.sd_servicepoint_status = '10003

It returns a third of the data in the AD_SERVICE_POINTtable, therefore if the clustering factors of the appropriateindexes are not particularly good, a full table scan is

probably the most efficient way to retrieve the relevant data.

A Si l M th d l


42/97

42


5. Are the predicted cardinalities for for the execution

plan accurate

If the predicted cardinalities for the row sources in the planare accurate and the appropriate access paths (indexes)

exist, the optimizer will usually come up with a good plan. Bad cardinality estimates, when bad, will be out by an order

of magnitude and ripple throughout the rest of the place.

If predicted cardinalities are accurate, detailed analysis ofaccess path costing may need to be performed.

This method of trouble shooting is sometimes referred to asTuning by cardinality feedback.


43/97

43

Tuning By Cardinality Feedback

Generally speaking if the predicated cardinalitiesand actual cardinalities for a row source are close,

the optimizer will pick the best plan for the query. To start down this route of tuning we need to obtain

the SQL text for the query in question along withany bind variable values it uses.

The best way of doing this is via one of the mostuseful tools in our tuning toolbox, theDBMS_XPLAN package.


44/97

44

Tuning By Cardinality Feedback

Tuning by cardinality feedback is borrowed from one of the reviewersof Jonathan Lewiss Cost Based Fundamentals book:www.centrexcc.com/Tuning by Cardinality Feedback.pdf.

O

therO

racle professionals may call this something different.

This is also endorsed by the Oracle Real World Performance Grouphttp://structureddata.org/2007/11/21/troubleshooting-bad-execution-plans/

With the GATHER_PLAN_STATISTICS hint and ALLSTATS LASTformat string in DBMS_XPLAN (Oracle 10g), this is a really easy

method to use. I would not advocate playing around with density settings as this may

fix queries suffering from the Predicate Independence Assumptionbut cause bad plans for other queries.


45/97

45

Obtaining Your SQL Text + Binds A trivial example to whet the appetiteSQL> select * from table(dbms_xplan.display_cursor('ft33c3agapy0k',0,'TYPICAL +PEEKED_BINDS'))

2 /

SQL_ID ft33c3agapy0k, child number 0

-------------------------------------

UPDATE TR_CYCLIC_WORK_PACK SET PACK_JOB_COUNTER =PACK_JOB_COUNTER +1

WHERE ID_PACK=:1

Plan hash value: 115273857

--------------------------------------------------------------------------

| Id | Operation | Name | Rows | Bytes | Cost |

--------------------------------------------------------------------------

| 0 | UPDATE STATEMENT | | | | 2 |

| 1 | UPDATE | TR_CYCLIC_WORK_PACK | | | |

|* 2 | INDEX UNIQUE SCAN| TR_CYC_WORKPACK_PK | 1 | 9 | 1 |

--------------------------------------------------------------------------

Peeked Binds (identified by position):

--------------------------------------

1 - :1 (NUMBER): 80310011


---------------------------------------------------

2 - access("ID_PACK"=:1)

Note

-----

- cpu costing is off (consider enabling it)


46/97

46

Checking Predicated

Versus Estimated CardinalitiesSQL> var b1 number

SQL> exec :b1 := 803100113

SQL> exec dbms_stats.gather_table_stats(user,'TR_CYCLIC_WORK_PACK',ESTIMATE_PERCENT=>NULL,CASCADE=>TRUE);

SQL> UPDATE /*+ GATHER_PLAN_STATISTICS */ TR_CYCLIC_WORK_PACK SET PACK_JOB_COUNTER =PACK_JOB_COUNTER +1

2 WHERE ID_PACK=:b1

3 /

SQL> select * from table(dbms_xplan.display_cursor(NULL,NULL,'ALLSTATS LAST'));

SQL_ID cqnxyqmp08rtu, child number 0

-------------------------------------

UPDATE /*+ GATHER_PLAN_STATISTICS */ TR_CYCLIC_WORK_PACK SET PACK_JOB_COUNTER

=PACK_JOB_COUNTER +1 WHERE ID_PACK=:b1


----------------------------------------------------------------------------------------------------

| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers |

----------------------------------------------------------------------------------------------------

| 1 | UPDATE | TR_CYCLIC_WORK_PACK | 1 | | 1 |00:00:00.01 | 4 |

|* 2 | INDEX UNIQUE SCAN| TR_CYC_WORKPACK_PK | 1 | 1 | 1 |00:00:00.01 | 2 |

----------------------------------------------------------------------------------------------------


---------------------------------------------------

2 - access("ID_PACK"=:B1)

Note

-----



47/97

47

Checking Predicated

Versus Estimated Cardinalities The example on the previous slide was the simplest example that can be provided.

Line 2 in the execution plan is what is known as a row source.

Note that for line 2 the values in the E-Rows and A-Rows columns matched.

If nested loop joins are used:-starts x E-Rows = A-Rows

otherwiseE-Rows = A-Rows

Usually for a bad plan the estimated cardinalities will differ from the predicated cardinalitiesby orders of magnitude, this inaccuracy in will ripple throughout the rest of the plan andlead to poor performance.

On the next slide a more complex example will be provided.


48/97

48

Reasons For Estimated

Cardinalities Being Inaccurate Statistics gathering limitations:-

Columns with skewed containing more than 254 distinctvalues => can be rectified with the DBMS_STATS API.

DBMS_STATS in 10g can produce statistically incorrectestimates of distinct values when auto sampling / sampling isused.

Not possible until 11g to gather statistics on correlatedcolumns, this is due to what is sometimes called data

correlation or the predicate independence assumption, moreon this later.In 10g dynamic sampling or hints can help here.

Not possible until 11g to gather statistics on functions unlessfunction based indexes are used.


49/97

49


Cardinalities Being Inaccurate None representative statistics:-

Statistics are missing or stale.

D

uring the course of processing data changes such that it nolonger reflects the statistics in the data dictionary, a particularproblem with scratch data, dynamic sampling can help here.

Lack of histograms or histograms with too few buckets oncolumns with skewed data.

Sampled statistics taken with too small a sample size, in 11gauto sampling gives 100% statistically accurate statistics.


50/97

50


Cardinalities Being Inaccurate Optimizer unfriendly SQL

Using values in the WHERE clauses with data typeswhich are different to those used by the columnscomparisons are being made against.


51/97

51

Reasons For EstimatedCardinalities Being Inaccurate

Other factors:-

Optimizer bugs.


52/97

52

Fixing Bad Cardinality Estimates

Hints may work fine for the data in your database at the timeof testing, however, as soon as the data changes, the planyou have forced via hints may no longer be the best plan.

#1 In the first place your stats should be correct.

#2 write SQL that gives the optimizer a fighting chance ofgets cardinality estimates correct, I.e. watch out for the useof functions and expressions.

#3 If you run into the Predicate independence assumption

issue this is a tough nut to crack, look at usingdynamic_sampling, read this article first:-http://structureddata.org/2008/03/05/there-is-no-time-like-now-to-use-dynamic-sampling/

#4 Use hints and profiles when all else fails.


53/97

53

Fixing Bad Cardinality Estimates

Histograms should be present for columns withskewed data with the correct number of buckets.

Only create histograms where they are required,they can have the side affect of increased hard

parsing through bind peeking.

!!! A histogram with only two end points storesno information on data distribution !!!.

Unless a hint is forDYNAMIC_SAMPLING only useaccess path hints as a last resort.


54/97

31/01/2012 54

Section 3Worked Examples OfTuning BYCardinality Feedback

A Si l E l Of B d C di li


55/97

55

A Simple Example Of Bad CardinalityEstimate Trouble Shooting

SELECT *

FROM (SELECT a.*, ROWNUM rnum

FROM (SELECT DISTINCT a.id_servicepoint, 'WALKROUTEREFID' AS walkrouterefid,

a.target_read_date, a.do_not_visit_flag, a.servicepoint_refid, c.customer_ref,

c.sd_customer_type, c.id_personal_details, c.business_name, c.trading_name,

d.surname, d.middlename, d.forename, b.out_post_code,

b.in_post_code, b.incomp_postcode, b.better_address_flag, b.address_line1,

b.address_line2, b.address_line3, b.address_line4, b.address_line5,

b.address_line6, b.address_line7, b.address_line8, b.address_line9,

b.address_line10, b.address_line11, b.address_line12,

g.out_post_code AS best_out_post_code, g.in_post_code AS best_in_post_code,

g.address_line1 AS best_address_line1, g.address_line2 AS best_address_line2,


g.address_line5 AS best_address_line5, g.address_line6 AS best_address_line6,g.address_line7 AS best_address_line7, g.address_line8 AS best_address_line8,



f.description, a.id_customer, f.ID AS customerid,

CASE WHEN b.better_address_flag = 'N'

THEN b.out_post_code || ' ' || b.in_post_code

ELSE g.out_post_code || ' ' || g.in_post_code

END AS out_in_postcode

FROM ad_service_point a, ad_portfolio_address b, ad_customer c, ad_personal_details d,

sd_customer_type f, ad_best_address g

WHERE a.id_customer = c.id_customer

AND a.id_portfolio_address = b.id_portfolio_address

AND a.id_best_address = g.id_best_address(+)

AND c.id_personal_details = d.id_personal_details

AND c.sd_customer_type = f.ID

AND b.incomp_postcode IN ('n', 'N')

AND a.id_servicepoint NOT IN (

SELECT id_servicepoint

FROM tr_job

WHERE sd_job_status IN (10003, 10011)

AND sd_job_type IN (10002, 10003))

AND UPPER (b.out_post_code) = 'FY1'

ORDER BY out_in_postcode) a

WHERE ROWNUM 0

A Simple Example Of Bad Cardinality


56/97

56


----------------------------------------------------------------------------------------------------

| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads | Writ

----------------------------------------------------------------------------------------------------

|* 1 | VIEW | | 1 | 1 | 26 |00:00:38.90 | 385K| 139K| 637 ||

|* 2 | COUNT STOPKEY | | 1 | | 26 |00:00:38.90 | 385K| 139K| 637 ||

| 3 | VIEW | | 1 | 1 | 26 |00:00:38.90 | 385K| 139K| 637 ||

|* 4 | SORT ORDER BY STOPKEY | | 1 | 1 | 26 |00:00:38.90 | 385K| 139K| 637 |

| 5 | HASH UNIQUE | | 1 | 1 | 43983 |00:00:38.75 | 385K| 139K| 637 | 12M

| 6 | NESTED LOOPS | | 1 | 1 | 43983 |00:00:38.36 | 385K| 138K| 0 |

| 7 | NESTED LOOPS | | 1 | 1 | 43983 |00:00:37.92 | 297K| 138K| 0 |

| 8 | NESTED LOOPS | | 1 | 1 | 43983 |00:00:37.66 | 253K| 138K| 0 ||

|* 9 | HASH JOIN ANTI | | 1 | 1 | 43983 |00:00:36.60 | 165K| 124K| 0 | 9005K

| 10 | NESTED LOOPS OUTER | | 1 | 26626 | 43983 |00:00:23.58 | 148K| 107K| 0 ||* 11 | HASH JOIN | | 1 | 26626 | 43983 |00:00:23.40 | 147K| 107K| 0 | 6444K

|* 12 | INDEX FAST FULL SCAN | AD_PORTADD_PK | 1 | 26626 | 43983 |00:00:13.04 | 564

| 13 | TABLE ACCESS FULL | AD_SERVICE_POINT | 1 | 6762K| 6762K|00:00:20.30 | 90832

| 14 | TABLE ACCESS BY INDEX ROWID| AD_BEST_ADDRESS | 43983 | 1 | 825 |00:00:00.13 | 1

|* 15 | INDEX UNIQUE SCAN | AD_BESTADD_PK | 43983 | 1 | 825 |00:00:00.06 | 826 | 1

|* 16 | INDEX FAST FULL SCAN | TR_JOB_IX13 | 1 | 266K| 3 |00:00:04.43 | 16759 | 16

|* 17 | INDEX UNIQUE SCAN | AD_CUST_ID_PK | 43983 | 1 | 43983 |00:00:01.11 | 87968 |

| 18 | TABLE ACCESS BY INDEX ROWID | SD_CUSTOMER_TYPE | 43983 | 1 | 43983 |00:00:00.22 |

|* 19 | INDEX UNIQUE SCAN | SD_CUSTTYPE_ID_PK | 43983 | 1 | 43983 |00:00:00.09 | 2 |

|* 20 | INDEX UNIQUE SCAN | AD_PERDET_ID_PK | 43983 | 1 | 43983 |00:00:00.39 |

----------------------------------------------------------------------------------------------------


---------------------------------------------------

1 - filter("RNUM">0)

2 - filter(ROWNUM


57/97

57


Estimate Trouble Shooting Lets create a histogram on sd_job_type

Lets look at the distinct values in the sd_job_type andsd_job_status columns:-

SQL> select count(distinct sd_job_type), count(distinct sd_job_status) from tr_job;COUNT(DISTINCTSD_JOB_TYPE) COUNT(DISTINCTSD_JOB_STATUS)

-------------------------- ----------------------------

3 5

Now lets look at the histograms on these columns:-1 select column_name, count(*) from user_tab_histograms where column_name in

('SD_JOB_TYPE', 'SD_JOB_STATUS)

2* group by column_name

SQL> /

COLUMN_NAME COUNT(*)

---------------- --------

SD_JOB_STATUS 5

SD_JOB_TYPE 2


58/97

58


Estimate Trouble Shooting The estimated cardinality is going badly wrong on line 16 of

the plan.SQL> exec dbms_stats.gather_table_stats(user,'TR_JOB',method_opt=>'FOR COLUMNS

SD_JOB_TYPE SIZE 3',estimate_percent=>NULL);

Well flush the shared pool, re run the query and get theestimated and actual cardinalities from the shared pool withDBMS_STATS.

You will see that on line 13 of the plan that the estimatedand actual cardinalities are closer.

We now need to look at line 12 on the plan, but thisillustrates the general principle of tuning by this approach.



59/97

59


----------------------------------------------------------------------------------------------------

| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads | W

----------------------------------------------------------------------------------------------------

|* 1 | VIEW | | 1 | 1 | 26 |00:03:24.00 | 13M| 148K| 637 ||

|* 2 | COUNT STOPKEY | | 1 | | 26 |00:03:24.00 | 13M| 148K| 637 |

| 3 | VIEW | | 1 | 1 | 26 |00:03:24.00 | 13M| 148K| 637 ||

|* 4 | SORT ORDER BY STOPKEY | | 1 | 1 | 26 |00:03:24.00 | 13M| 148K|

| 5 | HASH UNIQUE | | 1 | 1 | 43983 |00:03:23.86 | 13M| 148K| 637 |

| 6 | NESTED LOOPS | | 1 | 1 | 43983 |00:02:36.66 | 13M| 147K| 0 |

| 7 | NESTED LOOPS | | 1 | 1 | 43983 |00:02:36.14 | 13M| 147K| 0

| 8 | NESTED LOOPS | | 1 | 1 | 43983 |00:02:35.78 | 13M| 147K| 0 |

| 9 | NESTED LOOPS | | 1 | 1 | 43983 |00:02:29.76 | 13M| 131K| 0 |

|* 10 | HASH JOIN RIGHT ANTI | | 1 | 1 | 6762K|00:00:35.11 | 91941 | 91844 |

| 11 | INLIST ITERATOR | | 1 | | 3 |00:00:00.06 | 409 | 376 | 0 |

|* 12 | TABLE ACCESS BY INDEX ROWID| TR_JOB | 2 | 8237 | 3 |00:00:00.06 | 409 | 37

|* 13 | INDEX RANGE SCAN | TR_JOB_IX3 | 2 | 45251 | 36201 |00:00:00.01 | 34 |

|* 14 | HASH JOIN RIGHT OUTER | | 1 | 6762K| 6762K|00:00:28.35 | 91532 | 91468 |

| 15 | TABLE ACCESS FULL | AD_BEST_ADDRESS | 1 | 102K| 102K|00:00:00.01 | 700 |

| 16 | TABLE ACCESS FULL | AD_SERVICE_POINT | 1 | 6762K| 6762K|00:00:14.18 | 9083

|* 17 | INDEX UNIQUE SCAN | AD_PORTADD_PK | 6762K| 1 | 43983 |00:02:32.81 | 13M|

|* 18 | INDEX UNIQUE SCAN | AD_CUST_ID_PK | 43983 | 1 | 43983 |00:00:05.50 | 87968

| 19 | TABLE ACCESS BY INDEX ROWID | SD_CUSTOMER_TYPE | 43983 | 1 | 43983 |00:00:00.30 |

|* 20 | INDEX UNIQUE SCAN | SD_CUSTTYPE_ID_PK | 43983 | 1 | 43983 |00:00:00.11 | 2 |

|* 21 | INDEX UNIQUE SCAN | AD_PERDET_ID_PK | 43983 | 1 | 43983 |00:00:00.50 |

----------------------------------------------------------------------------------------------------


---------------------------------------------------1 - filter("RNUM">0)

2 - filter(ROWNUM


60/97

60

A More Complex Example

Lets turn the complexity setting up a notch.

We will use a divide and conquer strategy to work

out where the predicted cardinality is going wrong. In cases where queries may take hours to run,

running statements with theGATHER_PLAN_STATISTICS hint may not be

practical without taking the query apart.


61/97

61

I said I would provide a more involved example of plan for working through bad

cardinality estimate trouble shooting ... :-----------------------------------------------------------------------------------------------------

| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads

----------------------------------------------------------------------------------------------------

| 1 | VIEW | | 1 | 27 | 52 |00:01:33.74 | 199K| 304K| 62937

| 2 | SORT UNIQUE | | 1 | 27 | 52 |00:01:33.74 | 199K| 304K|

| 3 | UNION-ALL | | 1 | | 52 |00:00:02.11 | 199K| 304K| 62937

|* 4 | VIEW | | 1 | 1 | 26 |00:00:02.11 | 4301 | 3703 | 0 |

|* 5 | COUNT STOPKEY | | 1 | | 26 |00:00:02.11 | 4301 | 3703 | 0

| 6 | VIEW | | 1 | 1 | 26 |00:00:02.11 | 4301 | 3703 | 0

|* 7 | SORT UNIQUE STOPKEY | | 1 | 1 | 26 |00:00:02.11 | 4301

| 8 | NESTED LOOPS | | 1 | 1 | 107 |00:00:02.93 | 4290 | 3692 |

| 9 | NESTED LOOPS | | 1 | 1 | 107 |00:00:02.14 | 3967 | 3674 || 10 | NESTED LOOPS | | 1 | 1 | 107 |00:00:01.55 | 3751 | 3547

| 11 | NESTED LOOPS | | 1 | 1 | 107 |00:00:01.54 | 3749 | 3546

|* 12 | TABLE ACCESS BY INDEX ROWID| AD_SERVICE_POINT | 1 | 1 | 107 |00:00:00.

|* 13 | INDEX FULL SCAN | AD_SERVICE_POINT_IX6 | 1 | 1 | 1109 |00:00:00.28

| 14 | TABLE ACCESS BY INDEX ROWID| AD_WALKROUTE | 107 | 1 | 107 |00:00:00.04

|* 15 | INDEX UNIQUE SCAN | AD_WALKROUTE_PK | 107 | 1 | 107 |00:00:00.0

|* 16 | INDEX UNIQUE SCAN | SD_FIELDREG_ID_PK | 107 | 1 | 107 |00:00:00.01

|* 17 | INDEX UNIQUE SCAN | AD_CUST_ID_PK | 107 | 1 | 107 |00:00:00.52 |

|* 18 | TABLE ACCESS BY INDEX ROWID | AD_BEST_ADDRESS | 107 | 1 | 107 |00:00:00.

|* 19 | INDEX UNIQUE SCAN | AD_BESTADD_PK | 107 | 1 | 107 |00:00:00.01 |

|* 20 | VIEW | | 1 | 26 | 26 |00:01:31.63 | 195K| 300K| 629

|* 21 | COUNT STOPKEY | | 1 | | 26 |00:01:31.63 | 195K| 300K| 6

| 22 | VIEW | | 1 | 5945 | 26 |00:01:31.63 | 195K| 300K| 6

|* 23 | SORT UNIQUE STOPKEY | | 1 | 5945 | 26 |00:01:31.63 | 195

| 24 | NESTED LOOPS | | 1 | 5945 | 1799K|00:01:30.02 | 159K| 300K|

|* 25 | HASH JOIN | | 1 | 5945 | 1799K|00:01:24.62 | 159K| 300K| 6

| 26 | TABLE ACCESS FULL | AD_WALKROUTE | 1 | 10061 | 10061 |00:00:00.01 |

|* 27 | HASH JOIN | | 1 | 5945 | 1799K|00:01:20.99 | 159K| 300K|

|* 28 | HASH JOIN | | 1 | 5945 | 1800K|00:00:34.85 | 102K| 137K|

|* 29 | TABLE ACCESS FULL | AD_SERVICE_POINT | 1 | 5945 | 2043K|00:00:16.37

| 30 | INDEX FAST FULL SCAN | AD_CUSTOMER_IX1 | 1 | 6296K| 6296K|00:00:00.

|* 31 | INDEX FAST FULL SCAN | AD_PORTADD_PK | 1 | 67137 | 6668K|00:00:06.6

|* 32 | INDEX UNIQUE SCAN | SD_FIELDREG_ID_PK | 1799K| 1 | 1799K|00:00:03.20 |

----------------------------------------------------------------------------------------------------

continued ...

A Complex Example Of BadCardinality Estimate Trouble Shooting


62/97

62


---------------------------------------------------

4 - filter("RNUM">0)

5 - filter(ROWNUM


63/97

63

Start by looking for the row source which is the furthest into theexecution plan for where the cardinality is out by an order of magnitude.

This is line 28 in our plan:-|* 28 | HASH JOIN | | 1 | 5945 | 1800K|00:00:34.85 | 102K| 137K|

We have an estimated cardinality of 5945 versus an actual cardinality of1,800,000.

Notice the * at the beginning of the line, this means that there arepredicates associated with this table, lets look at these

29 - filter(("A"."ID_WALKROUTE" IS NOT NULL AND "A"."ID_BEST_ADDRESS" IS NULL AND UPPER("A"."WALKR

AND "A"."SD_SERVICE_TYPE"=10000) OR ("A"."SD_SERVICEPOINT_STATUS"=10001 AND "A"."SD_OCCUPIED_

INTERNAL_FUNCTION("A"."SD_SERVICEPOINT_STATUS"))))

Due to a possible bug, Oracle thinks that the predicates associated with

line 28 are for line 29.

Now lets use a Divide and conquer strategy for working out whereexactly in this list of predicates that are OR-ed and AND-ed together theestimate is going wrong.

A Complex Example Of BadCardinality Estimate Trouble Shooting

Trouble Shooting 1st Pass


64/97

64

1 SELECT /*+ GATHER_PLAN_STATISTICS */

2 COUNT(*)

3 FROM ad_service_point a

4 WHERE a.id_best_address IS NULL

5 AND UPPER (a.walkroute_reviewed_flag) = 'Y'

6 AND ( ( a.sd_service_type = '10000'

7 AND a.sd_servicepoint_status = '10001' )

8 OR ( a.sd_service_type = '10001'

9 AND a.sd_occupied_status = '10001'


11 OR a.sd_servicepoint_status = '10000'


13* )

14 /

COUNT(*)

----------

2043175


PLAN_TABLE_OUTPUT

----------------------------------------------------------------------------------------------------

SQL_ID 9qvfwxgnv1scj, child number 0

-------------------------------------

SELECT /*+ GATHER_PLAN_STATISTICS */ COUNT(*) FROM ad_service_point a WHERE

a.id_best_address IS NULL AND UPPER (a.walkroute_reviewed_flag) = 'Y' AND ( (

a.sd_service_type = '10000' AND a.sd_servicepoint_status = '10001' )

OR ( a.sd_service_type = '10001' AND a.sd_occupied_status = '10001'

AND a.sd_servicepoint_status = '10001' ) OR a.sd_servicepoint_status = '10000'

OR a.sd_servicepoint_status = '10003' )


PLAN_TABLE_OUTPUT

----------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------

| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads |

----------------------------------------------------------------------------------------------------

| 1 | SORT AGGREGATE | | 1 | 1 | 1 |00:00:17.77 | 92450 | 92426 |

|* 2 | TABLE ACCESS FULL| AD_SERVICE_POINT | 1 | 19524 | 2043K|00:00:18.42 | 92450 | 9

----------------------------------------------------------------------------------------------------. . . .

Trouble Shooting 1st Pass


65/97

65

We have two sets of predicates the id_best_address IS NULL andUPPER(a.walkroute_reviewed_flag) = Y and a load of stuff relating tostanding data columns within brackets.


2 COUNT(*)



5 AND UPPER (a.walkroute_reviewed_flag) = 'Y'6 AND ( ( a.sd_service_type = '10000'







13* )

14 /

For our second pass lets work which one of these sections is causingthe cardinality estimate to go wrong.

Analysis Of 1st Pass


66/97

66

We have two sets of predicates the id_best_address IS NULL andUPPER(a.walkroute_reviewed_flag) = Y and a load of stuff relating tostanding data columns within brackets.


2 COUNT(*)



5 AND UPPER (a.walkroute_reviewed_flag) = 'Y'6 AND ( ( a.sd_service_type = '10000'







13* )

14 /

For our second pass lets work which one of these sections is causingthe cardinality estimate to go wrong.

Analysis Of 1st Pass

T bl Sh ti 2nd P


67/97

67

It appears that the cardinality estimate is going wrong in the section in brackets.

1* select * from table(dbms_xplan.display_cursor(NULL,NULL,'ALLSTATS LAST'))

SQL> SELECT /*+ GATHER_PLAN_STATISTICS */2 COUNT(*)



5 /

COUNT(*)

----------

6712561


PLAN_TABLE_OUTPUT

----------------------------------------------------------------------------------------------------SQL_ID 2545fjyq0m5nm, child number 0

-------------------------------------


a.id_best_address IS NULL


----------------------------------------------------------------------------------------------------


----------------------------------------------------------------------------------------------------

| 1 | SORT AGGREGATE | | 1 | 1 | 1 |00:00:16.37 | 92450 | 92426 |

PLAN_TABLE_OUTPUT

----------------------------------------------------------------------------------------------------

|* 2 | TABLE ACCESS FULL| AD_SERVICE_POINT | 1 | 6712K| 6712K|00:00:13.46 | 92450 | 9

----------------------------------------------------------------------------------------------------


---------------------------------------------------

2 - filter("A"."ID_BEST_ADDRESS" IS NULL)

Note

-----


Trouble Shooting - 2nd Pass

T bl Sh ti 3rd P


68/97

68

Lets break down the section in brackets.


2 COUNT(*)3 FROM ad_service_point a

4 WHERE a.sd_servicepoint_status = '10000'

5* OR a.sd_servicepoint_status = '10003'

SQL> /

COUNT(*)

----------

282


PLAN_TABLE_OUTPUT

----------------------------------------------------------------------------------------------------SQL_ID 4rjg28wg48zv7, child number 0

-------------------------------------


a.sd_servicepoint_status = '10000' OR a.sd_servicepoint_status = '10003'


----------------------------------------------------------------------------------------------------


----------------------------------------------------------------------------------------------------

| 1 | SORT AGGREGATE | | 1 | 1 | 1 |00:00:00.03 | 6 | 5 |

PLAN_TABLE_OUTPUT

----------------------------------------------------------------------------------------------------

| 2 | INLIST ITERATOR | | 1 | | 282 |00:00:00.02 | 6 | 5 |

|* 3 | INDEX RANGE SCAN| AD_SERVICE_POINT_IX5 | 2 | 282 | 282 |00:00:00.03 | 6 | 5 |

----------------------------------------------------------------------------------------------------


---------------------------------------------------

3 - access(("A"."SD_SERVICEPOINT_STATUS"=10000 OR "A"."SD_SERVICEPOINT_STATUS"=10003))

Note

-----

Trouble Shooting 3rd Pass

T bl Sh ti 4th P


69/97

69

Lets break down the section in brackets.


2 COUNT(*)3 FROM ad_service_point a

4 WHERE a.sd_servicepoint_status = '10000'

5* OR a.sd_servicepoint_status = '10003'

SQL> /

COUNT(*)

----------

282


PLAN_TABLE_OUTPUT

----------------------------------------------------------------------------------------------------SQL_ID 4rjg28wg48zv7, child number 0

-------------------------------------


a.sd_servicepoint_status = '10000' OR a.sd_servicepoint_status = '10003'


----------------------------------------------------------------------------------------------------


----------------------------------------------------------------------------------------------------

| 1 | SORT AGGREGATE | | 1 | 1 | 1 |00:00:00.03 | 6 | 5 |

PLAN_TABLE_OUTPUT

----------------------------------------------------------------------------------------------------

| 2 | INLIST ITERATOR | | 1 | | 282 |00:00:00.02 | 6 | 5 |

|* 3 | INDEX RANGE SCAN| AD_SERVICE_POINT_IX5 | 2 | 282 | 282 |00:00:00.03 | 6 | 5 |

----------------------------------------------------------------------------------------------------


---------------------------------------------------

3 - access(("A"."SD_SERVICEPOINT_STATUS"=10000 OR "A"."SD_SERVICEPOINT_STATUS"=10003))

Note

-----

Trouble Shooting 4th Pass

T bl Sh ti 5th P


70/97

70


2 COUNT(*)


4 WHERE ( ( a.sd_service_type = '10000'




8* AND a.sd_servicepoint_status = '10001' ) )

SQL> /

COUNT(*)

----------

2070646


PLAN_TABLE_OUTPUT

----------------------------------------------------------------------------------------------------SQL_ID 17vy5ysvdqgxd, child number 0

-------------------------------------

SELECT /*+ GATHER_PLAN_STATISTICS */ COUNT(*) FROM ad_service_point a WHERE ( (

a.sd_service_type = '10000' AND a.sd_servicepoint_status = '10001' )

OR ( a.sd_service_type = '10001' AND a.sd_occupied_status = '10001'

AND a.sd_servicepoint_status = '10001' ) )


----------------------------------------------------------------------------------------------------


PLAN_TABLE_OUTPUT

----------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------

| 1 | SORT AGGREGATE | | 1 | 1 | 1 |00:00:17.90 | 92450 | 92426 |


----------------------------------------------------------------------------------------------------


---------------------------------------------------

2 - filter(("A"."SD_SERVICEPOINT_STATUS"=10001 AND ("A"."SD_SERVICE_TYPE"=10000 OR

("A"."SD_OCCUPIED_STATUS"=10001 AND "A"."SD_SERVICE_TYPE"=10001))))


T bl Sh ti 6th P


71/97

71


2 COUNT(*)




SQL> /

COUNT(*)

----------

1679148


PLAN_TABLE_OUTPUT

----------------------------------------------------------------------------------------------------

SQL_ID 2qd1zzbj3qgf7, child number 0

-------------------------------------


a.sd_service_type = '10000' AND a.sd_servicepoint_status = '10001' ) )


----------------------------------------------------------------------------------------------------


----------------------------------------------------------------------------------------------------| 1 | SORT AGGREGATE | | 1 | 1 | 1 |00:00:04.74 | 18921 | 18859 |

|* 2 | INDEX FAST FULL SCAN| AD_SERVICE_POINT_IX11 | 1 | 1679K| 1679K|00:00:01.71 | 18

----------------------------------------------------------------------------------------------------




72/97

72


2 COUNT(*)





SQL> /

COUNT(*)

----------

391498


PLAN_TABLE_OUTPUT

----------------------------------------------------------------------------------------------------

SQL_ID 186k84m0t9rmx, child number 0

-------------------------------------


a.sd_service_type = '10001' AND a.sd_occupied_status = '10001'

AND a.sd_servicepoint_status = '10001' ) )


----------------------------------------------------------------------------------------------------| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads |

----------------------------------------------------------------------------------------------------

| 1 | SORT AGGREGATE | | 1 | 1 | 1 |00:00:16.47 | 92450 | 92426 |


----------------------------------------------------------------------------------------------------



73/97

73

It appears that the cardinality estimate goes awry when Oracle is combining the cardinalities of:-


2 COUNT(*)





and:-


2 COUNT(*)

3 FROM ad_service_point a4 WHERE ( ( a.sd_service_type = '10000'


SQL> /

Analysis Of 7th Pass


74/97

74

To rule out statistics:- We could re-compute statistics on the AD_SERVICE_POINT table, this may ok for a test /

development environment but it may not be practical to do this on a production environmenton ad-hoc basis.

If table monitoring is enabled, which it should be by default you flush the monitoringinformation:-

SQL> execute DBMS_STATS.FLUSH_DATABASE_MONITORING_INFO

Then run:-

SELECT num_rows,

last_analyzed,

inserts,

updates,

deletes

FROM user_tables t,

user_tab_modifications m

WHERE t.table_name = m.table_name

AND table_name = < your table name >




75/97

75

To rule out statistics:- We could re-compute statistics on the AD_SERVICE_POINT table, this may ok for a test /

development environment but it may not be practical to do this on a production environmenton ad-hoc basis.

If table monitoring is enabled, which it should be by default you flush the monitoringinformation:-

SQL> execute DBMS_STATS.FLUSH_DATABASE_MONITORING_INFO

Then run:-

SELECT t.table_name,

last_analyzed,

SUM(inserts),

SUM(updates),

SUM(deletes)

FROM user_tables t,

user_tab_modifications m

WHERE t.table_name = m.table_name

AND timestamp > last_analyzedAND t.table_name =

GROUP BY t.table_name,

last_analyzed

/

You can see from this that it wouldnt be that difficult to produce something similar that

works for indexes.



76/97

76

This particular issue is down to what:- people from Oracle call data correlation

and what some people from outside ofOracle coin the predicate independenceassumption.

The exact nature of the problem is that if there are predicates on the same table

for which the data in the relevant columns is related, the optimizer alwaysassumes that these are independent.

This culminates in:-

=> incorrect selectivitys=> incorrect cardinality estimates

=> sub-optimal plans



77/97

77

This issue has been covered in great depth by the Oracle community that focuson the CBO, Jonathan Lewis describes this as:-

...Assume everyone in the audience knows which star sign they were bornunder . . . IfIask all the people born under Aries to raise their hands, Iexpect tosee 100 hands, there are 12 star signs, and we assume even distribution of data

selectivity is 1/12, cardinality is 1,200 / 12 = 100.How many people will raise their hands ifIask for all the people born underAries and in December to raise their hands? What about all the people bornunder Aries in March? What about the people born under Aries in April ?According to Oracle the answer will be the same for all three questions:

Selectivity (month AND star sign) = selectivity (month) * selectivity (star sign) =

1/12 * 1/12 = 1 /144 Cardinality = 1,200 * 1/144 = 8.5 (rounded to 8 or 9 depending on version of

Oracle). . .



78/97

78

Solutions to the data correlation / predicate independence assumptionissue include:-

dynamic sampling (9i onwards), this causes Oracle to sample thedata being queried, the level of sampling performed depends on thesampling level.

Hints to force the optimizer down the appropriate execution path.

SQL profiles (Oracle 10g onwards), Oracle uses what is known asoffline optimization to sample data and partially execute the query inorder to create a profile containing cardinality scaling information.

Extended statistics (Oracle 11g onwards) allows you to gatherstatistics on columns containing related data.

Oracle 11g automatic tuning ?.



79/97

31/01/2012 79

Section 4Summary & Wrap Up


80/97

80

Summary

Having good technical knowledge will only get youso far.

To be affective at tuning, your technical knowledgeneeds to be augmented with good practise andknowledge of common pit falls.

This will be covered on the remaining slides.


81/97

81

Tuning Objectives

Set clear and well defined performance goals, interms of data volume, concurrent users, timings,

CPU /IO

loading.

How do you know you have met your goals withouttargets ?.

Remember that timings are ultimately what is

important to the business.


82/97

82

Tuning By Magic Bullets

Do not tune using magic bullets, when faced with aproblem, be aware of likely causes, but appreciate that onesize does not fit all.

Do not pick up text book or article from the internet

describing a performance enhancing feature and apply itblindly across an application.

Do not assume you can use the tuning advisor to solve allyour ills because Oracle have mentioned this.

Do not use dynamic_sampling all over the place because I

have mentioned this. Etc ...

!!!ONE SIZE DOES NOT ALWAYS FIT ALL !!!


83/97

83

Tuning By Hit Ratios

Hit ratios can hide a multitude of sins.

Take the buffer cache hit ratio (bchr) for example.

A piece of SQL performing meaningless work canperform lots of logical reads and hence lead to agood BCHR.

Refer to custom hit ratio from

www.oracledba.co.uk.

Hit Ratios According To Tom Kyte


84/97

84


The following was posted on Ask Tom 4 December 2002 ....

You Asked

IfLIOs are bad,

1). is it possible to reduce the LIOs programatically ?

2). If yes, what steps/guidelines are to be followed, while coding, inorder to

reduce/avoidLIOs ?

3). what is a "OK" number of the LIOs (some percentage of SGA or PIOetc ?)

If possible, can you please demonstrate with an example ? (Iwill be using

PL/SQL forcoding)

Thanks,

Uday



85/97

85


And we said . . .1) by rewriting your SQL, tuning your SQL, setting environmental things like

sort_area_size or db_file_multi_block_read_count or the optimize_index_* parameters.

2) the piece of advice Ihave for everyone: totally and forever forget the concept that

"if my query ain't using an index, it must be broken". Massive nested loop joins are the

main culprit of this. If you are joining a 1,000,000 row table to a 2,000,000 row table

and trying to get 2,000,000 rows back -- it is very doubtful that an index should be used.

3) ratios STINK ( i have stronger words but this is a public forum after all ). There is

one ratio Iuse -- soft parse ratio (the ratio of soft to hard parses). It should be near 100 for

most systems. All other ratios -- forget about them. There is no magic "good number".

It is like the stupidest ratio of them all - cache hit. Ihave a theory that systems with high

cache hit ratios, over 95, 96% -- are among the most poorly tuned systems. They areExperiencing excessive LIO's due to massive nested loop joins. Yet, theirDBA's sit there

and say "well, my cache hit is 99% so all is well in the world".

Ihave no ratios for you. If you have a query that you need to execute and the best we

can do is 1,000,000LIO's -- then SO BEIT, that is that. However, if that query could

be executing without doing 1,000,000LI

O's then we need to fix it.


86/97

86

The Fan Hit Ratio

This is a meaningful hit ratio that I first came across in aJonathan Lewis presentationhttp://www.jlcomp.demon.co.uk/hit_ratio.pdf

100 * (

1 - least (1,

desiredresponse time /

actual response time)

)


87/97

87

Tuning By Oracle Marketing

Oracle 11g is self tuning => tuning is dead !!!

So why has Oracle produced an SQL test case packager in11g => http://optimizermagic.blogspot.com/2008/03/oracle-

support-keeps-closing-my-tar.html. Every Oracle release has bugs and flaws, 11g will be no

exception.

When 11.1.0.7 comes out, digest and the list of bugs fixed inthe optimizer....

New optimizer features are constantly introduced, everyfeature has its quirks and boundary cases under whichthings can break.


88/97

88

Tuning By The Internet

Some organisations publish on the internet aremore interested in dominating search engine

searches and advertising their books thandisseminating advice grounded on welldocumented test cases and evidence.

Prefer web sites and experts that provide worked

examples, e.g.Tom Kyte and Jonathan Lewis.


89/97

89

Tuning By Problem Scope

The scope of a solution to a performance problem shouldmatch the scope of the problem.

For example, an issue is caused by one particular SQL

statement is more likely to be resolved by something suchas a new index, index change, histogram than a systemwide parameter change.

Yes, there are DBAs out there who when faced with a

performance issue will look at adjusting obscureparameters which most DBAs in their professional careerswill never have any need to touch.


90/97

90

Tuning What You Know

Always tune what you know, find where the time is going tofirst and then apply tuning techniques accordingly.

Understand the flight envelope of you application, how it

behaves under load.

Refer to Cary Milsaps presentation on performance andskew from Oracle Open World 2008.

Do not blindly pick an Oracle feature conducive to good

performance, pick the bottle necks off one by one.


91/97

91

Tuning Is A Moving Target

The RDBMS is constantly evolving, in 11g release1 alone we have:-

Extended statisticsNull aware joins

Adaptive cursors

Plan base lining

A new method of calculating density

Protect yourself from this by thorough testing


92/97

92

Understand Response Time

Response time = wait time + service time

Service time = CPU time => joining, parsing etc

Wait time = waiting on an event I/O, contention etc

Understand this basic equation and its context with yourapplication.

Avoid blind scatter gun techniques such as looking atyour software with a monitoring tool and saying hey there is

a lot of latching taking place, therefore that must be myproblem, work off the top waits events and drill down.


93/97

93

Tuning Via Flat Statistics

You get a mechanic to look at your car, you saythere is a performance issue with the engine, it isdoing 3000 Rpm at 60 Mph.

The mechanic will probably ask:- When did first start happening

Under what driving conditions does this happen

Does this happen consistently

Have there been any changes to your car, whenwas it last serviced.


94/97

94

Tuning Via Flat Statistics

The moral of this story is that a single statistic inisolation is not that useful.

In order to put statistics into context you need toalso understand things such as:-

where is most of the time going in your application ?

What is the normal performance base line or Flight

envelope for your application Is you think you are seeing strange and anomalous

behaviour, what conditions does it occur under ?


95/97

95

Know Your Tool Kit

Performance views

10g tuning infra structure, ASH, advisors, ADDMand the time model.

SQL trace, SQL extended trace, tkprof, trcsess.

O/s tools, iostat, sar etc ...

DBMS_XPLAN

Your tuning arsenal should consist of morethan explain plan and the ability to spot fulltable scans.


96/97

96

Understand Key Object Statistics

Histograms, frequency and height balanced,MetaLink note 72539.1.

Index quality => clustering factors, many DBAs do not

understand this, this gets a whole chapter in JonathanLewiss Cost Based Fundamentals book.MetaLink notes 39836.1 and 223117.1

Predicate selectivity, MetaLink note 68992.1

System statistics aux_stats$, MetaLink note 153761.1

Sample sizes

Numbers of distinct values.

Etc ...


97/97

Useful Internet Resources

The Oracle Real World performance group bloghttp:/www.structureddata.org

Jonathan Lewiss blog http://jonathanlewis.wordpress.com/

The Oracle Optimizer Group Blog http://optimizermagic.blogspot.com/ Wolfgang Breitlings web site (a reviewer of Jonathan Lewiss CBO book)

www.centrexcc.com

General Oracle papers, look out for those in particular from ChritianAntognini

http://www.trivadis

.com/en/know-how-community/download-area

.html

The Search ForIntelligent Life In The Cost-Based Optimizer by AnjoKolk: http://www.evdbt.com/SearchIntelligenceCBO.doc

an introduction to oracle sql tuning

Documents