Download - Optimizer in oracle 11g by wwf from ebay COC
Some new features of Optimizer in Oracle 11G
--------------- Wang Weifeng/COC
2 eBay Inc. confidential
1 AUTO_SAMPLE_SIZE enhancement
In 11g, the auto_sample_size parameter was enhanced greatly: it will enhance the accuracy while accelerate the statistics gathering process compared with 10g.
3 eBay Inc. confidential
Time elapsed compare:
4 eBay Inc. confidential
Accuracy compare
5 eBay Inc. confidential
Accuracy compare
6 eBay Inc. confidential
2 Selectivity of multi-column index enhancement
• SQL> create table wangweifeng(a number, b number, c varchar2(200));
• Table created.
• SQL> insert into wangweifeng select mod(rownum,100), mod(rownum, 50), object_name from all_objects where rownum <= 5000;
• 5000 rows created.
• SQL> insert into wangweifeng select mod(rownum,100), mod(rownum, 50), object_name from all_objects where rownum <= 5000;
• 5000 rows created.
• SQL> commit;
• Commit complete.
• SQL> select count(*), count(distinct a), count(distinct b), count(distinct a||b) from wangweifeng;
• COUNT(*) COUNT(DISTINCTA) COUNT(DISTINCTB) COUNT(DISTINCTA||B)
• ---------- ---------------- ---------------- -------------------
• 10000 100 50 100
• SQL> create index ind_wangweifeng on wangweifeng(a, b);
• Index created.
7 eBay Inc. confidential
In 10g:
SQL> exec dbms_stats.gather_table_stats(user, 'wangweifeng');PL/SQL procedure successfully completed.
SQL> explain plan for 2 select * from wangweifeng where a = 10 and b = 10;Explained.
SQL> select * from table(dbms_xplan.display);
PLAN_TABLE_OUTPUT-----------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |-----------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 2 | 50 | 2 (0)| 00:00:01 || 1 | TABLE ACCESS BY INDEX ROWID| WANGWEIFENG | 2 | 50 | 2 (0)| 00:00:01 ||* 2 | INDEX RANGE SCAN | IND_WANGWEIFENG | 2 | | 1 (0)| 00:00:01 |-----------------------------------------------------------------------------------------------Predicate Information (identified by operation id):------------------------------------------------------------------------ 2 - access("A"=10 AND "B"=10)
2 is got by using “10000/num_distinct_of_column_A/number_distinct_of_column_B”, that is: 10000/100/50, which equal to 2.It won’t use the 1/distinct_keys of the index.
8 eBay Inc. confidential
In 11g:
• SQL> explain plan for
• 2 select * from wangweifeng where a = 10 and b = 10;
• Explained.
• SQL> select * from table(dbms_xplan.display);
• PLAN_TABLE_OUTPUT
• --------------------------------------------------------------------------------
• Plan hash value: 3853876490
• -----------------------------------------------------------------------------------------------
• | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
• -----------------------------------------------------------------------------------------------
• | 0 | SELECT STATEMENT | | 100 | 2400 | 42 (0)| 00:00:01 |
• | 1 | TABLE ACCESS BY INDEX ROWID| WANGWEIFENG | 100 | 2400 | 42 (0)| 00:00:01 |
• |* 2 | INDEX RANGE SCAN | IND_WANGWEIFENG | 100 | | 1 (0)| 00:00:01 |
• -----------------------------------------------------------------------------------------------
• 100 is get by using distinct_keys of index
9 eBay Inc. confidential
3 Multi-Column Correlation and Extended Stats in Oracle 11g
• create table t_vc asselect mod(n, 100) n1, mod(n, 100) n2 ,mod(n, 50) n3 , mod(n, 20) n4from (select level n from dual connect by level <= 10001); The first two columns, n1 and n2, have a strong correlation: n1 is always equals to n2.
• Let’s collect statistics with histograms on all columns.
• dbms_stats.gather_Table_stats( user, 'T_VC', estimate_percent => null, method_opt => 'for all columns size 254');
• The first two columns, n1 and n2, have a strong correlation: n1 is always equals to n2.
10 eBay Inc. confidential
Example:
• select * from table(dbms_xplan.display);
• ----------------------------------------------------
• | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
• ----------------------------------------------------
• | 0 | SELECT STATEMENT | | 1 | 6 | 9 (0)| 00:00:01 |
• | 1 | SORT AGGREGATE | | 1 | 6
• |* 2 | TABLE ACCESS FULL| T_VC | 1 | 6 | 9 (0)| 00:00:01 |
• ---------------------------------------------------------------------------------------------------------------
• The optimizer cardinality is far off and 100 times lower than the correct cardinality.
• # of rows~= total # of rows * (1/NDV for n1) * (1/NDV for n2)= 10000 * (1/100) * (1/100) =1 row.
11 eBay Inc. confidential
Extended Stats
• SELECT dbms_stats.create_extended_stats(ownname=>user, tabname => 'T_VC',extension => '(n1, n2)' ) as n1_n2_correlation FROM dual;
• n1_n2_correlation
• ---------------------------------------------------
• SYS_STUBZH0IHA7K$KEBJVXO5LOHAS
• Let’s collect stats again on this table and check the SQL plan.
• exec dbms_stats.gather_Table_stats( user, 'T_VC', estimate_percent => null, method_opt => 'for all columns size 254');
12 eBay Inc. confidential
Optimizer understand the correlation:
explain plan for select count(*) from t_vc where n1=10 and n2=10;
----------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |----------------------------------------------------| 0 | SELECT STATEMENT | | 100 | 1200 | 9 (0)| 00:00:01 ||* 1 | TABLE ACCESS FULL| T_VC | 100 | 1200 | 9 (0)| 00:00:01--------------------------------------------------
in 11g, the optimizer has truly understood the correlation
13 eBay Inc. confidential
Under the Hood
• 1. Adding an extended stats adds a new virtual column to the table. Here is the line from sqltrace.
• alter table "CBQT"."T_VC" add (SYS_STUBZH0IHA7K$KEBJVXO5LOHAS as (sys_op_combined_hash(n1, n2)) virtual BY USER for statistics);
• The virtual column name is cryptic — it seems to have been derived from table_name, column_name combinations. This is why we reanalyzed the table.
• 2. A new deterministic hash function, sys_op_combined_hash, is called by optimizer to populate this virtual column values. This deterministic function returns same value for unique combination of arguments passed.
• col h1 format 99999999999999999999999999
• select sys_op_combined_hash (1,1) h1 from dual;
• H1----------------------------
• 7026129190895635777
• select sys_op_combined_hash (1,2) h1 from dual;
• H1----------------------------298332787864732998
• Collecting histograms on all columns collects histograms on this virtual column also.
14 eBay Inc. confidential
Under the hood
• Using the above histogram, the CBO is able to find that there is a strong correlation between these two columns. This is visible in the 10053 output.
• SINGLE TABLE ACCESS PATH
• Single Table Cardinality Estimation for T_VC[T_VC]
• Column (#1): NewDensity:0.005000, OldDensity:0.000050 BktCnt:10001, PopBktCnt:10001, PopValCnt:100, NDV:100
• Column (#2): NewDensity:0.005000, OldDensity:0.000050 BktCnt:10001, PopBktCnt:10001, PopValCnt:100, NDV:100
• Column (#5): NewDensity:0.005000, OldDensity:0.000050 BktCnt:10001, PopBktCnt:10001, PopValCnt:100, NDV:100
• ColGroup (#1, VC) SYS_STUBZH0IHA7K$KEBJVXO5LOHAS Col#: 1 2 CorStregth: 100.00
• ColGroup Usage:: PredCnt: 2 Matches Full: #0 Partial: Sel: 0.0100
• Table: T_VC Alias: T_VC Card: Original: 10001.000000 Rounded: 100 Computed: 100.00 Non Adjusted: 100.00
• Access Path: TableScan
• Cost: 9.11 Resp: 9.11 Degree: 0
• Cost_io: 9.00 Cost_cpu: 2404620
• Resp_io: 9.00 Resp_cpu: 2404620
• Best:: AccessPath: TableScan
• Cost: 9.11 Degree: 1 Resp: 9.11 Card: 100.00 Bytes: 0 Notice the ColGroup line and CorStrength field above. It is set to 100.
• CorStrength is calculated using histograms for the virtual column and final cardinality estimates are multiplied by CorStrength.
• # of rows~= total # of rows * (1/NDV for n1) * (1/NDV for n2)*corStrength= 10000 * (1/100) * (1/100)*100 =100 rows.
15 eBay Inc. confidential
views
• We can see the information for multi-column statistics in the view dba_stat_extensions
• SQL> column EXTENSION_NAME format a40
• SQL> select EXTENSION_NAME, EXTENSION, CREATOR, DROPPABLE from user_stat_extensions where table_name = 'T_VC';
• EXTENSION_NAME EXTENSION CREATOR DROPPABLE
• ------------------------- -------------------- ---------- ---------
• SYS_STUBZH0IHA7K$KEBJVXO5LOHAS ("N1","N2") USER YES
16 eBay Inc. confidential
Histogram
– SQL> select column_name, ENDPOINT_NUMBER, ENDPOINT_VALUE from user_tab_histograms where table_name = 'T_VC';– COLUMN_NAME ENDPOINT_NUMBER ENDPOINT_VALUE– -------------------------------------------------- --------------- --------------– N1 100 0– N1 200 1– N1 300 2– ……………………………………………………………….– N2 100 0– N2 200 1– N2 300 2– ……………………………………………………………….– N3 200 0– N3 400 1– N3 600 2– ……………………………………………………………….– N4 500 0– N4 1000 1– N4 1500 2– ……………………………………………………………….– SYS_STUBZH0IHA7K$KEBJVXO5LOHAS 100 132792849– SYS_STUBZH0IHA7K$KEBJVXO5LOHAS 200 164854493– SYS_STUBZH0IHA7K$KEBJVXO5LOHAS 300 186027136– SYS_STUBZH0IHA7K$KEBJVXO5LOHAS 400 256223149– SYS_STUBZH0IHA7K$KEBJVXO5LOHAS 500 305884769– SYS_STUBZH0IHA7K$KEBJVXO5LOHAS 600 331414609– SYS_STUBZH0IHA7K$KEBJVXO5LOHAS 700 521035387– SYS_STUBZH0IHA7K$KEBJVXO5LOHAS 800 577086672– SYS_STUBZH0IHA7K$KEBJVXO5LOHAS 900 606199386– SYS_STUBZH0IHA7K$KEBJVXO5LOHAS 1000 852695214
17 eBay Inc. confidential
Some functions for extended statistics
• We can also use the following way to create multi-column statistics:
• SQL> exec dbms_stats.gather_table_stats(user, 'T_VC', method_opt=>'for all columns for columns (n1,n2) size 254')
• PL/SQL procedure successfully completed.
• We can use show_extended_stats_name procedure to should the extension name:
• SQL> select dbms_stats.show_extended_stats_name(user, 'T_VC', '(n1,n2)') from dual;
• DBMS_STATS.SHOW_EXTENDED_STATS_NAME(USER,'T_VC','(N1,N2)')
• ----------------------------------------------------------------
• SYS_STUBZH0IHA7K$KEBJVXO5LOHAS
• To drop multi-column statistics, we can user DROP_EXTENDED_STATS procedure:
• SQL> exec dbms_stats.DROP_EXTENDED_STATS(user, 't_vc', '(n1,n2)')
• PL/SQL procedure successfully completed.
18 eBay Inc. confidential
4 Expression statistics
• Create test table
• SQL> create table test(id number, name varchar2(40), country varchar2(50));
• SQL> insert into test select rownum, 'name' || rownum,
• 2 case
• 3 when rownum <= 10 then 'China'
• 4 when rownum <= 20 then 'USA'
• 5 when rownum <= 30 then 'India'
• 6 when rownum <= 100 then 'usa'
• 7 when rownum <= 120 then 'France'
• 8 when rownum <= 135 then 'UK'
• 9 when rownum <= 140 then 'uk'
• 10 when rownum <= 180 then 'india'
• 11 when rownum <= 230 then 'china'
• 12 when rownum <= 280 then 'france'
• 13 when rownum <= 310 then 'Canada'
• 14 when rownum <= 350 then 'Mexico'
• 15 when rownum <= 390 then 'canada'
• 16 when rownum <= 450 then 'mexico'
• 17 when rownum <= 1000 then 'INDIA'
• 18 else 'Tailand'
• 19 end
• 20 from dual connect by rownum <= 1200;
• 1200 rows created.
19 eBay Inc. confidential
Data distribution
• SQL> select country, count(*) from test group by country;
• COUNTRY COUNT(*)
• -------------------- ----------
• Canada 30
• China 10
• France 20
• INDIA 550
• India 10
• Mexico 40
• Tailand 200
• UK 15
• USA 10
• canada 40
• china 50
• france 50
• india 40
• mexico 60
• uk 5
• usa 70
20 eBay Inc. confidential
Bad estimate result
• SQL> exec dbms_stats.gather_table_stats(user, 'test', cascade=>true);
• PL/SQL procedure successfully completed.
• SQL> explain plan for
• 2 select * from test where lower(country) = 'india';
• Explained.
• SQL> select * from table(dbms_xplan.display);
• PLAN_TABLE_OUTPUT
• ---------------------------------------------------------------------
• Plan hash value: 217508114
• --------------------------------------------------------------------------
• | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
• --------------------------------------------------------------------------
• | 0 | SELECT STATEMENT | | 12 | 216 | 3 (0)| 00:00:01 |
• |* 1 | TABLE ACCESS FULL| TEST | 12 | 216 | 3 (0)| 00:00:01 |
• --------------------------------------------------------------------------
• The optimizer estimates that only 12 rows filtered, which has much difference to the true
21 eBay Inc. confidential
gather expression statistics
• SQL> exec dbms_stats.gather_table_stats(user, 'test', method_opt => 'for all columns for columns (lower(country))');
• PL/SQL procedure successfully completed.
• SQL> explain plan for
• 2 select * from test where lower(country) = 'india';
• Explained.
• SQL> select * from table(dbms_xplan.display);
• --------------------------------------------------------------------------
• | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
• --------------------------------------------------------------------------
• | 0 | SELECT STATEMENT | | 600 | 16200 | 3 (0)| 00:00:01 |
• |* 1 | TABLE ACCESS FULL| TEST | 600 | 16200 | 3 (0)| 00:00:01 |
• --------------------------------------------------------------------------
• Predicate Information (identified by operation id):
• ---------------------------------------------------
• 1 - filter("TEST"."SYS_STUEJ3KN8Q$YG7Z8LBEZ7A6P3G"='india')
We can see: the optimizer is right this time! It estimates 600 rows will filtered.
22 eBay Inc. confidential
views
• SQL> select EXTENSION_NAME, EXTENSION from user_stat_extensions where table_name = 'TEST';
• EXTENSION_NAME EXTENSION
• -------------------------------------- ---------------------------------------
• SYS_STUEJ3KN8Q$YG7Z8LBEZ7A6P3G (LOWER("COUNTRY"))
• We can see that, Oracle create statistics for the expression lower(country):
• SQL> select column_name, num_distinct, histogram from user_tab_col_statistics where table_name = 'TEST';
• COLUMN_NAME NUM_DISTINCT HISTOGRAM
• ------------------------------ ------------ ---------------------------------------------
• ID 1200 HEIGHT BALANCED
• NAME 1200 HEIGHT BALANCED
• COUNTRY 16 FREQUENCY
• SYS_STUEJ3KN8Q$YG7Z8LBEZ7A6P3G 8 FREQUENCY
23 eBay Inc. confidential
6 Pending statistics
• In previous versions, new optimizer statistics were automatically published immediately after they are gathered.
• In 11G, this is still the default behavior. But we now have the option of keeping the newly gathered statistics in a pending state until we choose to publish them.
• We can use set_table_prefs procedure in dbms_stats in 11G to let optimizer choose to not publish statistics automatically for the table. And we can use get_prefs in dbms_stats
• to check whether the optimizer automatically publish the statistics automatically or not.
24 eBay Inc. confidential
Create test table
• SQL> create table tt(id number, status varchar2(20));
• Table created.
• SQL> select dbms_stats.get_prefs('publish', user, 'TT') from dual;
• DBMS_STATS.GET_PREFS('PUBLISH',USER,'TT')
• ---------------------------------------------
• TRUE
• SQL> insert into tt values(1, 'VALID');
• 1 row created.
• SQL> commit;
• Commit complete.
• SQL> exec dbms_stats.gather_table_stats(user, 'tt')
• PL/SQL procedure successfully completed.
• SQL> select num_rows, blocks, last_analyzed from user_tables where table_name = 'TT';
• NUM_ROWS BLOCKS LAST_ANALYZED
• ---------- ---------- -------------------
• 1 5 2009-10-11 22:02:12
25 eBay Inc. confidential
Disable immediate publish attribute:
• SQL> exec dbms_stats.set_table_prefs(user, 'TT', 'publish', 'false');
• PL/SQL procedure successfully completed.
• SQL> select dbms_stats.get_prefs('publish', user, 'TT') from dual;
• DBMS_STATS.GET_PREFS('PUBLISH',USER,'TT')
• -----------------------------------------
• FALSE
• SQL> insert into tt select rownum + 1, 'INVALID' from dual connect by rownum <= 10000;
• 10000 rows created.
• SQL> commit;
26 eBay Inc. confidential
Do a test:
• SQL> exec dbms_stats.gather_table_stats(user, 'tt')
• PL/SQL procedure successfully completed.
• We can see that, the new statistics was not populated:
• SQL> select num_rows, blocks, last_analyzed from user_tables where table_name = 'TT';
• NUM_ROWS BLOCKS LAST_ANALYZED
• ---------- ---------- -------------------
• 1 5 2009-10-11 22:02:12
27 eBay Inc. confidential
User_tab_pending_stats
• The statistics gathered just now is stored in view: user_tab_pending_stats
• SQL> desc user_tab_pending_stats
• Name Null? Type
• ------------------------------------------------------------------------ -------- ------------------- TABLE_NAME VARCHAR2(30)
• PARTITION_NAME VARCHAR2(30)
• SUBPARTITION_NAME VARCHAR2(30)
• NUM_ROWS NUMBER
• BLOCKS NUMBER
• AVG_ROW_LEN NUMBER
• SAMPLE_SIZE NUMBER
• LAST_ANALYZED DATE
• SQL> select num_rows, blocks, last_analyzed from user_tab_pending_stats where table_name = 'TT';
• NUM_ROWS BLOCKS LAST_ANALYZED
• ---------- ---------- -------------------
• 10001 28 2009-10-11 22:04:53
28 eBay Inc. confidential
Let’s publish the statistics:
• SQL> exec dbms_stats.publish_pending_stats(user, 'TT')
• PL/SQL procedure successfully completed.
• SQL> select num_rows, blocks, last_analyzed from user_tables where table_name = 'TT';
• NUM_ROWS BLOCKS LAST_ANALYZED
• ---------- ---------- -------------------
• 10001 28 2009-10-11 22:04:53
• We can see that there are no rows in user_tab_pending_stats after publishing:
• SQL> select num_rows, blocks, last_analyzed from user_tab_pending_stats where table_name = 'TT';
• no rows selected
29 eBay Inc. confidential
The usage of pending statistics
• Optimizer won’t use the pending statistics which is not published.
• Then, what we can do by using pending statistics? Let us continue to see example.
30 eBay Inc. confidential
Example
• SQL> update tt set status = 'EXPIRED' where status = 'INVALID' and rownum <= 9990;
• 9990 rows updated.
• SQL> commit;
• Commit completed
• SQL> select status, count(*) from tt group by status;
• STATUS COUNT(*)
• -------------------- ----------
• INVALID 10
• VALID 1
• EXPIRED 9990
31 eBay Inc. confidential
Example
• SQL> exec dbms_stats.gather_table_stats(user, 'TT', cascade=>true, method_opt=>'for all columns size 254')
• PL/SQL procedure successfully completed.
• SQL> select num_rows, blocks, last_analyzed from user_tab_pending_stats where table_name = 'TT';
• NUM_ROWS BLOCKS LAST_ANALYZED
• ---------- ---------- -------------------
• 10001 28 2009-10-13 09:09:12
32 eBay Inc. confidential
Example
• SQL> select num_rows, distinct_keys, last_analyzed from user_ind_pending_stats
• 2 where index_name = 'IND_TT_STATUS';
• NUM_ROWS DISTINCT_KEYS LAST_ANALYZED
• ---------- ------------- -------------------
• 10001 3 2009-10-13 09:09:12
• SQL> desc USER_TAB_HISTGRM_PENDING_STATS
• name null? type
• ------------------------------------------------------------------------ -------- -------------------
• TABLE_NAME VARCHAR2(30)
• PARTITION_NAME VARCHAR2(30)
• SUBPARTITION_NAME VARCHAR2(30)
• COLUMN_NAME VARCHAR2(30)
• ENDPOINT_NUMBER NUMBER
• ENDPOINT_VALUE NUMBER
• ENDPOINT_ACTUAL_VALUE VARCHAR2(1000)
• SQL> select ENDPOINT_ACTUAL_VALUE, ENDPOINT_NUMBER from USER_TAB_HISTGRM_PENDING_STATS where table_name = 'TT' and column_name = 'STATUS
• ENDPOINT_ACTUAL_VALUE ENDPOINT_NUMBER
• ------------------------------ ---------------
• EXPIRED 9990
• INVALID 10000
• VALID 10001
33 eBay Inc. confidential
Example:
• SQL> select ENDPOINT_ACTUAL_VALUE, ENDPOINT_NUMBER from user_tab_histograms where table_name = 'TT' and column_name = 'STATUS';
• ENDPOINT_ACTUAL_VALUE ENDPOINT_NUMBER
• ------------------------------ ---------------
• INVALID 10000
• VALID 10001
• We can also check the column level statistics:
• SQL> select column_name, num_distinct, density, last_analyzed from user_tab_col_statistics where table_name = 'TT';
• COLUMN_NAME NUM_DISTINCT DENSITY LAST_ANALYZED
• -------------------- ------------ ---------- -------------------
• ID 10001 .00009999 2009-10-13 09:05:33
• STATUS 2 .000049995 2009-10-13 09:05:33
34 eBay Inc. confidential
Example:
• SQL> desc USER_COL_PENDING_STATS
• name null? type
• ------------------------------------------------------------------------ -------- -------------------
• TABLE_NAME VARCHAR2(30)
• PARTITION_NAME VARCHAR2(30)
• SUBPARTITION_NAME VARCHAR2(30)
• COLUMN_NAME VARCHAR2(30)
• NUM_DISTINCT NUMBER
• LOW_VALUE RAW(32)
• HIGH_VALUE RAW(32)
• DENSITY NUMBER
• NUM_NULLS NUMBER
• AVG_COL_LEN NUMBER
• SAMPLE_SIZE NUMBER
• LAST_ANALYZED DATE
• SQL> select column_name, num_distinct, density, last_analyzed from USER_COL_PENDING_STATS where table_name = 'TT';
• COLUMN_NAME NUM_DISTINCT DENSITY LAST_ANALYZED
• -------------------- ------------ ---------- -------------------
• ID 10001 .00009999 2009-10-13 09:09:12
• STATUS 3 .000049995 2009-10-13 09:09:12
35 eBay Inc. confidential
Example
• SQL> explain plan for select * from tt where status = 'INVALID';
• Explained.
• SQL> select * from table(dbms_xplan.display);
• --------------------------------------------------------------------------
• | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
• --------------------------------------------------------------------------
• | 0 | SELECT STATEMENT | | 10000 | 107K| 8 (0)| 00:00:01 |
• |* 1 | TABLE ACCESS FULL| TT | 10000 | 107K| 8 (0)| 00:00:01 |
• --------------------------------------------------------------------------
• SQL> explain plan for
• 2 select * from tt where status = 'EXPIRED';
• Explained.
• SQL> select * from table(dbms_xplan.display);
• ---------------------------------------------------------------------------------------------
• | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
• ---------------------------------------------------------------------------------------------
• | 0 | SELECT STATEMENT | | 1 | 11 | 2 (0)| 00:00:01 |
• | 1 | TABLE ACCESS BY INDEX ROWID| TT | 1 | 11 | 2 (0)| 00:00:01 |
• |* 2 | INDEX RANGE SCAN | IND_TT_STATUS | 1 | | 1 (0)| 00:00:01 |
• ---------------------------------------------------------------------------------------------
36 eBay Inc. confidential
Parameter: optimizer_use_pending_statistics
• We can set the parameter optimizer_use_pending_statistics to check the new statistics work fine or not.
• The default value is false:
• SQL> show parameter optimizer_use_pending_statistics
• NAME TYPE VALUE
• ------------------------------------ --------------------------------- ------------
• optimizer_use_pending_statistics boolean FALSE
• Let us enable it in session level to check if the new statistics can make the execution plan better or not.
37 eBay Inc. confidential
Pending statistics works!
• SQL> alter session set optimizer_use_pending_statistics = true;
• Session altered.
• SQL> explain plan for select * from tt where status = 'INVALID';
• Explained.
• SQL> select * from table(dbms_xplan.display);
• ---------------------------------------------------------------------------------------------
• | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
• ---------------------------------------------------------------------------------------------
• | 0 | SELECT STATEMENT | | 10 | 110 | 2 (0)| 00:00:01 |
• | 1 | TABLE ACCESS BY INDEX ROWID| TT | 10 | 110 | 2 (0)| 00:00:01 |
• |* 2 | INDEX RANGE SCAN | IND_TT_STATUS | 10 | | 1 (0)| 00:00:01 |
• ---------------------------------------------------------------------------------------------
• SQL> explain plan for select * from tt where status = 'EXPIRED';
• Explained.
• SQL> select * from table(dbms_xplan.display);
• --------------------------------------------------------------------------
• | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
• --------------------------------------------------------------------------
• | 0 | SELECT STATEMENT | | 9990 | 107K| 8 (0)| 00:00:01 |
• |* 1 | TABLE ACCESS FULL| TT | 9990 | 107K| 8 (0)| 00:00:01 |
• --------------------------------------------------------------------------
38 eBay Inc. confidential
Delete pending statistics:
• We can also use delete_pending_stats to delete pending statistics:
• SQL> select num_rows, blocks, last_analyzed from user_tab_pending_stats where table_name = 'TT';
• NUM_ROWS BLOCKS LAST_ANALYZED
• ---------- ---------- -------------------
• 10001 28 2009-10-11 22:30:15
• SQL> exec dbms_stats.delete_pending_stats(user, 'TT');
• PL/SQL procedure successfully completed.
• SQL> select num_rows, blocks, last_analyzed from user_tab_pending_stats where table_name = 'TT';
• no rows selected
39 eBay Inc. confidential
Export pending statistics
• We can use export_pending_stats procedure to export the pending statistics to a stat table:
• PROCEDURE EXPORT_PENDING_STATS
• Argument Name Type In/Out Default?
• ------------------------------ ----------------------- ------ --------
• OWNNAME VARCHAR2 IN DEFAULT
• TABNAME VARCHAR2 IN
• STATTAB VARCHAR2 IN
• STATID VARCHAR2 IN DEFAULT
• STATOWN VARCHAR2 IN DEFAULT
40 eBay Inc. confidential
7 The enhancement of delete_column_stats procedure
• Before 11G, if we want to remove histogram information for a column in table (while keeping other statistics), we have to use the following method:
• Exec dbms_stats.gather_table_stats(user, ‘T’, cascade=>true, method=>’for column colA size 1’)
• Such operation is always resource intensive and time consuming.
• From 11G, we can we can only delete the statistics for a certain column while keeping other information by using the option: col_stat_type=>'HISTOGRAM':
41 eBay Inc. confidential
Example
• SQL> select COLUMN_NAME, num_distinct, low_value, high_value, density, histogram from user_tab_col_statistics where table_name = 'TT';
• COLUMN_NAME NUM_DISTINCT LOW_VALUE HIGH_VALUE DENSITY HISTOGRAM
• ------------ ------------ ------------------------------ ------------------------------ ---------- ---------------
• ID 10001 C102 C3020102 .00009999 HEIGHT BALANCED
• STATUS 2 494E56414C4944 56414C4944 .000049995 FREQUENCY
• SQL> exec dbms_stats.delete_column_stats(user, 'TT', 'status', col_stat_type=>'histogram');
• PL/SQL procedure successfully completed.
• SQL> select COLUMN_NAME, num_distinct, low_value, high_value, density, histogram from user_tab_col_statistics where table_name = 'TT';
• COLUMN_NAME NUM_DISTINCT LOW_VALUE HIGH_VALUE DENSITY HISTOGRAM
• ------------ ------------ ------------------------------ ------------------------------ ---------- ---------------
• ID 10001 C102 C3020102 .00009999 HEIGHT BALANCED
• STATUS 2 494E56414C4944 56414C4944 .5 NONE
42 eBay Inc. confidential
Example
• SQL> exec dbms_stats.delete_column_stats(user, 'TT', 'id', col_stat_type=>'histogram');
• PL/SQL procedure successfully completed.
• SQL> select COLUMN_NAME, num_distinct, low_value, high_value, density, histogram from user_tab_col_statistics where table_name = 'TT';
• COLUMN_NAME NUM_DISTINCT LOW_VALUE HIGH_VALUE DENSITY HISTOGRAM
• ------------ ------------ ------------------------------ ------------------------------ ---------- ---------------
• ID 10001 C102 C3020102 .00009999 NONE
• STATUS 2 494E56414C4944 56414C4944 .5 NONE
43 eBay Inc. confidential
8 set_****_prefs and get_prefs
• Although get_param and set_param are first introduced in 10G, it will be deprecated in 11G.
• In 11g, package dbms_stats introduced 2 new procedures to replace them. That is set_****_prefs and get_prefs:
44 eBay Inc. confidential
New procedures:
• FUNCTION GET_PREFS RETURNS VARCHAR2
• Argument Name Type In/Out Default?
• ------------------------------ ----------------------- ------ --------
• PNAME VARCHAR2 IN
• OWNNAME VARCHAR2 IN DEFAULT
• TABNAME VARCHAR2 IN DEFAULT
• PROCEDURE SET_DATABASE_PREFS
• Argument Name Type In/Out Default?
• ------------------------------ ----------------------- ------ --------
• PNAME VARCHAR2 IN
• PVALUE VARCHAR2 IN
• ADD_SYS BOOLEAN IN DEFAULT
• PROCEDURE SET_GLOBAL_PREFS
• Argument Name Type In/Out Default?
• ------------------------------ ----------------------- ------ --------
• PNAME VARCHAR2 IN
• PVALUE VARCHAR2 IN
• PROCEDURE SET_SCHEMA_PREFS
• Argument Name Type In/Out Default?
• ------------------------------ ----------------------- ------ --------
• OWNNAME VARCHAR2 IN
• PNAME VARCHAR2 IN
• PVALUE VARCHAR2 IN
• PROCEDURE SET_TABLE_PREFS
• Argument Name Type In/Out Default?
• ------------------------------ ----------------------- ------ --------
• OWNNAME VARCHAR2 IN
• TABNAME VARCHAR2 IN
• PNAME VARCHAR2 IN
• PVALUE VARCHAR2 IN
• PROCEDURE DELETE_DATABASE_PREFS
• Argument Name Type In/Out Default?
• ------------------------------ ----------------------- ------ --------
• PNAME VARCHAR2 IN
• ADD_SYS BOOLEAN IN DEFAULT
• FORCE BOOLEAN IN DEFAULT
• PROCEDURE DELETE_SCHEMA_PREFS
• Argument Name Type In/Out Default?
• ------------------------------ ----------------------- ------ --------
• OWNNAME VARCHAR2 IN
• PNAME VARCHAR2 IN
• PROCEDURE DELETE_TABLE_PREFS
• Argument Name Type In/Out Default?
• ------------------------------ ----------------------- ------ --------
• OWNNAME VARCHAR2 IN
• TABNAME VARCHAR2 IN
• PNAME VARCHAR2 IN
45 eBay Inc. confidential
Example
• As we have already know, if we keep a table monitoring and when the change of the table exceeds 10%, then the statistics will be gathered automatically by using “gather stale” option.
• The default value is hard coded and can not be modified.
• From 11G, we can set it manually:
• SQL> select dbms_stats.get_prefs('stale_percent', user, 'tt') from dual;
• DBMS_STATS.GET_PREFS('STALE_PERCENT',USER,'TT')
• ------------------------------
• 10
• SQL> exec dbms_stats.set_table_prefs(user, 'tt', 'stale_percent', 15);
• PL/SQL procedure successfully completed.
• SQL> select dbms_stats.get_prefs('stale_percent', user, 'tt') from dual;
• DBMS_STATS.GET_PREFS('STALE_PERCENT',USER,'TT')
• -----------------------------
• 15
46 eBay Inc. confidential
Example:
• To restore it to the default value, we can simple do:
• SQL> exec dbms_stats.delete_table_prefs(user, 'tt', 'stale_percent')
• PL/SQL procedure successfully completed.
• SQL> select dbms_stats.get_prefs('stale_percent', user, 'tt') from dual;
• DBMS_STATS.GET_PREFS('STALE_PERCENT',USER,'TT')
• ---------------------------------------------------
• 10
• We can also use the following way to set default value for all database:
• SQL> exec dbms_stats.set_global_prefs('stale_percent', 20)
• PL/SQL procedure successfully completed.
47 eBay Inc. confidential
Totally the following several preferences can be set:
• AUTOSTATS_TARGET (SET_GLOBAL_PREFS only)
• CASCADE
• DEGREE
• ESTIMATE_PERCENT
• GRANULARITY
• INCREMENTAL
• METHOD_OPT
• NO_INVALIDATE
• PUBLISH
• STALE_PERCENT
48 eBay Inc. confidential
The preference of one particular table
• We can also check the view dba_tab_stat_prefs to check the preference of a table:
• SQL> select * from user_tab_stat_prefs where table_name = 'TT';
• TABLE_NAME PREFERENCE_NAME PREFERENCE_VALUE
• ------------------------------ ------------------------------ ----------------------
• TT PUBLISH TRUE
• TT STALE_PERCENT 10
49 eBay Inc. confidential
Export/import peference
• PROCEDURE EXPORT_DATABASE_PREFS
• Argument Name Type In/Out Default?
• ------------------------------ ----------------------- ------ --------
• STATTAB VARCHAR2 IN
• STATID VARCHAR2 IN DEFAULT
• STATOWN VARCHAR2 IN DEFAULT
• ADD_SYS BOOLEAN IN DEFAULT
• PROCEDURE EXPORT_SCHEMA_PREFS
• Argument Name Type In/Out Default?
• ------------------------------ ----------------------- ------ --------
• OWNNAME VARCHAR2 IN
• STATTAB VARCHAR2 IN
• STATID VARCHAR2 IN DEFAULT
• STATOWN VARCHAR2 IN DEFAULT
• PROCEDURE EXPORT_TABLE_PREFS
• Argument Name Type In/Out Default?
• ------------------------------ ----------------------- ------ --------
• OWNNAME VARCHAR2 IN
• TABNAME VARCHAR2 IN
• STATTAB VARCHAR2 IN
• STATID VARCHAR2 IN DEFAULT
• STATOWN VARCHAR2 IN DEFAULT
• PROCEDURE IMPORT_DATABASE_PREFS
• Argument Name Type In/Out Default?
• ------------------------------ ----------------------- ------ --------
• STATTAB VARCHAR2 IN
• STATID VARCHAR2 IN DEFAULT
• STATOWN VARCHAR2 IN DEFAULT
• ADD_SYS BOOLEAN IN DEFAULT
• PROCEDURE IMPORT_SCHEMA_PREFS
• Argument Name Type In/Out Default?
• ------------------------------ ----------------------- ------ --------
• OWNNAME VARCHAR2 IN
• STATTAB VARCHAR2 IN
• STATID VARCHAR2 IN DEFAULT
• STATOWN VARCHAR2 IN DEFAULT
• PROCEDURE IMPORT_TABLE_PREFS
• Argument Name Type In/Out Default?
• ------------------------------ ----------------------- ------ --------
• OWNNAME VARCHAR2 IN
• TABNAME VARCHAR2 IN
• STATTAB VARCHAR2 IN
• STATID VARCHAR2 IN DEFAULT
• STATOWN VARCHAR2 IN DEFAULT
50 eBay Inc. confidential
9 copy_table_stats procedure
• From 10G, there is a new procedure named copy_table_stats introduced. It is used to copy statistics from one partition to another partition.
• PROCEDURE COPY_TABLE_STATS
• Argument Name Type In/Out Default?
• ------------------------------ ----------------------- ------ --------
• OWNNAME VARCHAR2 IN
• TABNAME VARCHAR2 IN
• SRCPARTNAME VARCHAR2 IN
• DSTPARTNAME VARCHAR2 IN
• SCALE_FACTOR NUMBER IN DEFAULT
• FLAGS NUMBER IN DEFAULT
• FORCE BOOLEAN IN DEFAULT
SCALE_FACTOR is newly introduced in 11G
51 eBay Inc. confidential
Example
• Let us see the following example to see its usage:
• SQL> create table a1(id number, name varchar2(64), partition_key number)
• 2 partition by range(partition_key) (
• 3 partition p0 values less than (1),
• 4 partition p1 values less than (2),
• 5 partition p2 values less than (3),
• 6 partition pmax values less than (maxvalue)
• 7 );
• SQL> create index ind_a1_id on a1(id) local;
• Index created.
• SQL> insert into a1 select rownum, 'name' || rownum, 0 from dual connect by rownum <= 10000;
• 10000 rows created.
52 eBay Inc. confidential
Example
• SQL> exec dbms_stats.gather_table_stats(user, 'a1', cascade=>true) ;
• PL/SQL procedure successfully completed.
• SQL> select partition_name, num_rows, blocks, avg_row_len, last_analyzed from user_tab_partitions where table_name = 'A1';
• PARTITION_NAME NUM_ROWS BLOCKS AVG_ROW_LEN LAST_ANALYZED
• -------------------- ---------- ---------- ----------- -------------------
• P0 10000 28 14 2009-10-13 21:55:16
• P1 0 0 0 2009-10-13 21:55:16
• P2 0 0 0 2009-10-13 21:55:16
• PMAX 0 0 0 2009-10-13 21:55:16
• SQL> select partition_name, num_rows, distinct_keys, BLEVEL, last_analyzed from user_ind_partitions where index_name = 'IND_A1_ID';
• PARTITION_NAME NUM_ROWS DISTINCT_KEYS BLEVEL LAST_ANALYZED
• -------------------- ---------- ------------- ---------- -------------------
• P0 10000 10000 1 2009-10-13 21:55:17
• P1 0 0 0 2009-10-13 21:55:17
• P2 0 0 0 2009-10-13 21:55:17
• PMAX 0 0 0 2009-10-13 21:55:17
53 eBay Inc. confidential
Example
• SQL> select num_rows, blocks, avg_row_len, last_analyzed from user_tables where table_name = 'A1';
• NUM_ROWS BLOCKS AVG_ROW_LEN LAST_ANALYZED
• ---------- ---------- ----------- -------------------
• 10000 28 14 2009-10-13 21:55:16
• SQL> select num_rows, distinct_keys, BLEVEL, last_analyzed from user_indexes where index_name = 'IND_A1_ID';
• NUM_ROWS DISTINCT_KEYS BLEVEL LAST_ANALYZED
• ---------- ------------- ---------- -------------------
• 10000 10000 1 2009-10-13 21:55:17
54 eBay Inc. confidential
Copy statistics
• SQL> exec dbms_stats.copy_table_stats(user, 'a1', 'p0', 'p1')
• PL/SQL procedure successfully completed.
• SQL> select partition_name, num_rows, blocks, avg_row_len, last_analyzed from user_tab_partitions where table_name = 'A1';
• PARTITION_NAME NUM_ROWS BLOCKS AVG_ROW_LEN LAST_ANALYZED
• -------------------- ---------- ---------- ----------- -------------------
• P0 10000 28 14 2009-10-13 21:55:16
• P1 10000 28 14 2009-10-13 21:55:16
• P2 0 0 0 2009-10-13 21:55:16
• PMAX 0 0 0 2009-10-13 21:55:16
• SQL> select partition_name, num_rows, distinct_keys, BLEVEL, last_analyzed from user_ind_partitions where index_name = 'IND_A1_ID';
• PARTITION_NAME NUM_ROWS DISTINCT_KEYS BLEVEL LAST_ANALYZED
• -------------------- ---------- ------------- ---------- -------------------
• P0 10000 10000 1 2009-10-13 21:55:17
• P1 10000 10000 1 2009-10-13 21:55:17
• P2 0 0 0 2009-10-13 21:55:17
• PMAX 0 0 0 2009-10-13 21:55:17
55 eBay Inc. confidential
Example
• The global level statistics does not change correspondingly:
• SQL> select num_rows, blocks, avg_row_len, last_analyzed from user_tables where table_name = 'A1';
• NUM_ROWS BLOCKS AVG_ROW_LEN LAST_ANALYZED
• ---------- ---------- ----------- -------------------
• 10000 28 14 2009-10-13 21:55:16
• SQL> select num_rows, distinct_keys, BLEVEL, last_analyzed from user_indexes where index_name = 'IND_A1_ID';
• NUM_ROWS DISTINCT_KEYS BLEVEL LAST_ANALYZED
• ---------- ------------- ---------- -------------------
• 10000 10000 1 2009-10-13 21:55:17
56 eBay Inc. confidential
Example
• SQL> exec dbms_stats.copy_table_stats(user, 'a1', 'p0', 'p2', 3);
• We set scale_factor to 3:
• SQL> select partition_name, num_rows, blocks, avg_row_len, last_analyzed from user_tab_partitions where table_name = 'A1';
• PARTITION_NAME NUM_ROWS BLOCKS AVG_ROW_LEN LAST_ANALYZED
• -------------------- ---------- ---------- ----------- -------------------
• P0 10000 28 14 2009-10-13 21:55:16
• P1 10000 28 14 2009-10-13 21:55:16
• P2 30000 84 14 2009-10-13 21:55:16
• PMAX 0 0 0 2009-10-13 21:55:16
• SQL> select partition_name, num_rows, distinct_keys, BLEVEL, last_analyzed from user_ind_partitions where index_name = 'IND_A1_ID';
• PARTITION_NAME NUM_ROWS DISTINCT_KEYS BLEVEL LAST_ANALYZED
• -------------------- ---------- ------------- ---------- -------------------
• P0 10000 10000 1 2009-10-13 21:55:17
• P1 10000 10000 1 2009-10-13 21:55:17
• P2 30000 10000 1 2009-10-13 21:55:17
• PMAX 0 0 0 2009-10-13 21:55:17
57 eBay Inc. confidential
Example
• After using copy_table_stats to copy statistics from one partition to another partition, the previous sql won’t be invalidated and reparsed due to the last_analyze_time does not change
58 eBay Inc. confidential
Example
• SQL> select count(*) from a1;
• COUNT(*)
• ----------
• 10000
• SQL> column first_laod_time format a30
• SQL> select hash_value, executions, INVALIDATIONS, first_load_time from v$sql where sql_text = 'select count(*) from a1';
• HASH_VALUE EXECUTIONS INVALIDATIONS FIRST_LOAD_TIME
• ---------- ---------- ------------- ----------------------
• 3010923763 1 0 2009-10-13/22:31:35
59 eBay Inc. confidential
Example
• SQL> exec dbms_stats.copy_table_stats(user, 'a1', 'p0', 'p2', 3);
• PL/SQL procedure successfully completed.
• SQL> select count(*) from a1;
• COUNT(*)
• ----------
• 10000
• SQL> select count(*) from a1;
• COUNT(*)
• ----------
• 10000
• SQL> select hash_value, executions, INVALIDATIONS, first_load_time from v$sql where sql_text = 'select count(*) from a1';
• HASH_VALUE EXECUTIONS INVALIDATIONS FIRST_LOAD_TIME
• ---------- ---------- ------------- ----------------
• 3010923763 2 0 2009-10-13/22:31:35
60 eBay Inc. confidential
Example
•
• We have to manually do some DDL to the table and make the sql reparsed:
• SQL> grant select on a1 to dba;
• Grant succeeded.
• SQL> select count(*) from a1;
• COUNT(*)
• ----------
• 10000
• SQL> select hash_value, executions, INVALIDATIONS, first_load_time from v$sql where sql_text = 'select count(*) from a1';
• HASH_VALUE EXECUTIONS INVALIDATIONS FIRST_LOAD_TIME
• ---------- ---------- ------------- -------------------
• 3010923763 1 1 2009-10-13/22:31:35