partitioning on oracle 12c - what changed on the most important oracle feature
TRANSCRIPT
Partitioning on Oracle 12cWHAT CHANGED ON THE MOST IMPORTANT ORACLE FEATURE
Luís Marques • Oracle ACE• Founder of Redglue • OUGPT Founder and Leader• Blogger at http://lcmarques.com• @drune / [email protected]
Once upon a time in partitioning
Range, Hash and Composite Partitioning
List Partitioning,Composite Range-
List Partitioning
Global Hash Indexes,
1M partitions per table limit
Interval, Virtual Column, Extended Composite and
System Partitioning
Interval Reference PartitioningCascading
TRUNCATE&EXCHANGE,Online Moving
Maintaning Multiple PartitionGlobal Async Indexes
Partial Indexes
Partial Indexes & async global index maintenance
Partial Indexes “Create local or global indexes on subset of partitions enabling more flexibility”
• Default table indexing property (INDEXING) for table or partitions
SQL> create table doc ( doc_id number(10), doc_date DATE, doc_status VARCHAR2(2), doc_owner VARCHAR2(24)) INDEXING OFF
PARTITION BY RANGE (doc_date ) (
PARTITION doc_old_p1 VALUES LESS THAN (TO_DATE('01-JAN-2016','DD-MON-YYYY')) INDEXING OFF, PARTITION doc_new_p2 VALUES LESS THAN (TO_DATE('01-APR-2016','DD-MON-YYYY')) INDEXING ON,PARTITION doc_new_p3 VALUES LESS THAN (TO_DATE('01-JUL-2016','DD-MON-YYYY'))INDEXING ON,PARTITION doc_new_p4 VALUES LESS THAN (TO_DATE('01-OCT-2016','DD-MON-YYYY'))INDEXING ON,PARTITION doc_new_p5 VALUES LESS THAN (TO_DATE('01-DEC-2016','DD-MON-YYYY'))INDEXING ON);
Partial Indexes “Partial Indexes only spans for partitions with INDEXING ON attribute. They work with local and global indexes”
SQL>@t doc
TABLE_NAME PARTITION PARTITION_COUNT STATUS DEF_INDEXING---------- --------- --------------- -------- ------------DOC RANGE 5 VALID OFF
TABLE_NAME PARTITION_NAME INTERVAL INDEXING---------- --------------- -------- --------DOC DOC_NEW_P2 NO ONDOC DOC_NEW_P3 NO ONDOC DOC_NEW_P4 NO ONDOC DOC_NEW_P5 NO ONDOC DOC_OLD_P1 NO OFF
Partial Indexes “Create local or global indexes on subset of partitions enabling more flexibility”
SQL> create index doc_owner_idx on doc(doc_owner) global indexing partial;
doc_old_p1 (OFF)
doc_old_p3
doc_old_p2
doc_old_p4
doc_old_p5
global partial index
SQL> create index doc_status_local_idx on doc(doc_status) local indexing partial;
doc_old_p1 (OFF)
doc_old_p3
doc_old_p2
doc_old_p4
doc_old_p4
local idx 1 Local idx 2 Local idx 3 Local idx 4
Partial Indexes “Create local or global indexes on subset of partitions enabling more flexibility”
SQL> @i doc
INDEX_NAME INDEX_TYPE TABLE_NAME UNIQUENES PARTITIONED INDEXING ORPHANED_ENTRIES -------------------- --------------------------- ---------- --------- ----------- -------- ----------------- DOC_OWNER_IDX NORMAL DOC NONUNIQUE NO PARTIAL NO DOC_STATUS_LOCAL_IDX NORMAL DOC NONUNIQUE YES PARTIAL NO
INDEX_NAME TABLE_NAME PARTITION PARTITION_COUNT LOCALITY -------------------- ---------- --------- --------------- -------- DOC_OWNER_LOCAL_IDX DOC RANGE 5 LOCAL
INDEX_NAME PARTITION_NAME STATUS INTERVAL ORPHANED_ENTRIES -------------------- --------------- -------- -------- ----------------- DOC_STATUS_LOCAL_IDX DOC_OLD_P1 UNUSABLE NO NO DOC_STATUS_LOCAL_IDX DOC_NEW_P5 USABLE NO NO DOC_STATUS_LOCAL_IDX DOC_NEW_P4 USABLE NO NO DOC_STATUS_LOCAL_IDX DOC_NEW_P3 USABLE NO NO DOC_STATYS_LOCAL_IDX DOC_NEW_P2 USABLE NO NO
Partial Indexes - Global “CBO awareness – Table expansion”
SQL> create index doc_owner_idx on doc(doc_owner) global indexing partial;SQL> select * from doc where doc_owner = 'LNLAZMALQMMQQVTEVGCZ’
Global partial index
Full partition single scan
Partial Indexes – Local “CBO awareness”
SQL> create index doc_status_local_idx on doc(doc_status) local indexing partial;SQL> select * from doc where doc_status = 'XX'
Local partial index (2,5)
Full partition single scan
Partial Indexes – orphaned entries “when global index is subject to deferred index maintenance or INDEXING was turned off for partition”
SQL> alter table t.doc modify partition DOC_NEW_P3 indexing off;
doc_old_p1 (OFF)
doc_old_p3
doc_old_p2
doc_old_p4
doc_old_p4
global partial index
doc_old_p1 (OFF)
doc_old_p3
doc_old_p2
doc_old_p4
doc_old_p4
local idx 1 Local idx 2 Local idx 3 Local idx 4
Partial Indexes – orphaned entries “when global index is subject to deferred index maintenance or INDEXING was turned off for partition”
SQL> alter table t.doc modify partition DOC_NEW_P3 indexing off;
INDEX_NAME TABLE_NAME UNIQUENES PARTITIONED INDEXING STATUS ORPHANED_ENTRIES -------------------- ---------- --------- ----------- -------- -------- ----------------- DOC_STATUS_LOCAL_IDX DOC NONUNIQUE YES PARTIAL N/A NO DOC_OWNER_IDX DOC NONUNIQUE NO PARTIAL VALID YES
INDEX_NAME PARTITION_NAME STATUS INTERVAL ORPHANED_ENTRIES -------------------- --------------- -------- -------- ----------------- DOC_STATUS_LOCAL_IDX DOC_OLD_P1 UNUSABLE NO NO DOC_STATUS_LOCAL_IDX DOC_NEW_P5 USABLE NO NO DOC_STATUS_LOCAL_IDX DOC_NEW_P4 USABLE NO NO DOC_STATUS_LOCAL_IDX DOC_NEW_P3 UNUSABLE NO NO DOC_STATUS_LOCAL_IDX DOC_NEW_P2 USABLE NO NO
Partial Indexes – orphaned entries “Behind the scenes of indexing off/on is something fast and something slow ”
SQL> alter table t.doc modify partition DOC_NEW_P3 indexing off;•Lock Table in EXCLUSIVE & UPDATES dictionary internal tables
•insert into index_orphaned_entry$ (indexobj#, tabpartdobj#, hidden)
SQL> alter table t.doc modify partition DOC_NEW_P3 indexing on;
• Coalesce the global index: alter index "T"."DOC_OWNER_IDX" coalesce cleanup• Rebuild the local partitioned index: ALTER INDEX "T"."DOC_STATUS_LOCAL_IDX" REBUILD PARTITION "DOC_NEW_P3” •Delete from index_orphaned_entry$
Partial Indexes – orphaned entries “CBO awareness”
SQL> alter table t.doc modify partition DOC_NEW_P3 indexing off;SQL> select * from doc where doc_owner = 'LNLAZMALQMMQQVTEVGCZ’
Performance penalty – scan the entire global partial index, even the orphaned entries - maintenance is always required!
Global partial index
Partition Range – understands that you have orphaned entries
Partial Indexes – orphaned entries “CBO awareness”
SQL> alter table t.doc modify partition DOC_NEW_P3 indexing off;SQL> select * from doc where doc_owner = 'LNLAZMALQMMQQVTEVGCZ’
Undocumented function - TBL$OR$IDX$PART$NUM(…..) - Returns the partition numbers that it needs to scan.
Partial Indexes – orphaned entries
No segment changes
Partial Indexes – orphaned entries “orphaned entries, no maintenance? ”
SQL> alter table t.doc modify partition DOC_NEW_P3 indexing off;
Why is this operation so fast? (You already know it!) but …
Is Oracle is leaving orphaned entries on the indexes for ever? What are my options?
Asyncronous / deferred index maintenance “Option 1 – Rebuild me”
SQL> alter index DOC_OWNER_IDX rebuild partition DOC_NEW_P3;
Entire index structure rebuild
Makes sense when the scale of orphaned entries is big enough
Asyncronous / deferred index maintenance “Option 2 – Coaslesce is dead, long life to new coalesce”
SQL> alter index DOC_OWNER_IDX coalesce cleanup;
Visits each index leaf block and remove the orphaned entries
Better that the standard coalesce as it understands what is an index orphaned entry
Maybe an inexpensive alternative to rebuilt partition
Asyncronous / deferred index maintenance “Option 3 - Postponed index maintenance to a better time”
Index maintenance can be deferred to a better time; Scheduled (2:00 AM by default) or manually : Scheduled deferred maintenance.
JOB_NAME JOB_CREATOR COMMENTS ------------------------------ ------------ ------------------------------------------------------------ PMO_DEFERRED_GIDX_MAINT_JOB SYS Oracle defined automatic index cleanup for partition mainten ance operations with deferred global index maintenance
Manually restore global indexes in need of a cleanup
SQL> exec DBMS_PART.CLEANUP_GIDX('T','DOC');
Partial Indexes – Limitations and defaults “Create local or global indexes on subset of partitions enabling more flexibility”
• Unique indexes and indexes enforcing unique constraints are not supported Non-partitioned tables are not supported Defaults to FULL indexes instead of PARTIAL Supports INDEXING clause at partition and subpartition levels
Multipe Partition Maintenance & async global index maintenance
multiple partition maintenance “Single DDL does it all – add, drop, merge, splitting and truncating”
SQL> create table t.new_doc ( doc_id, doc_date, doc_status, doc_owner, doc_value)INDEXING OFFPARTITION BY RANGE (doc_value ) interval (100)(PARTITION p0 values less than (99))as select …., rownum from dual connect by level <= 1000;TABLE_NAME PARTITION_NAME INTERVAL NUM_ROWS INDEXING---------- --------------- -------- ---------- --------NEW_DOC P0 NO OFFNEW_DOC SYS_P481 YES OFFNEW_DOC SYS_P482 YES OFFNEW_DOC SYS_P483 YES OFFNEW_DOC SYS_P484 YES OFFNEW_DOC SYS_P485 YES OFF
SQL> alter table NEW_DOC modify partition SYS_P481 indexing on;
SQL> alter table NEW_DOC modify partition SYS_P482 indexing on;
multiple partition maintenance “Single DDL does it all – add, drop, merge, splitting and truncating”
SQL> create index doc_value_idx on new_doc(doc_value) global indexing partial;
SQL>create index doc_value_idx2 on new_doc(doc_id);
INDEX_NAME TABLE_NAME UNIQUENES PARTITIONED INDEXING STATUS ORPHANED_ENTRIES-------------------- ---------- --------- ----------- -------- -------- -----------------DOC_VALUE_IDX2 NEW_DOC NONUNIQUE NO FULL VALID NODOC_VALUE_IDX NEW_DOC NONUNIQUE NO PARTIAL VALID NO
P0 (OFF) SYS_P482 SYS_P483SYS_P481 SYS_P490
global partial index
Global index (doc_value_idx2)
multiple partition maintenance – drop/truncate “Single DDL does it all – add, drop, merge, splitting and truncating”
SQL> ALTER TABLE new_doc DROP partitions for (700), for (500), for (100);
P0 (OFF) SYS_P482(200)
SYS_P483(300)
SYS_P481(100)
SYS_P490(400)
global partial index
SYS_Pn(500)
SYS_Pn(700)
multiple partition maintenance – drop/truncate “Single DDL does it all – add, drop, merge, splitting and truncating”
SQL> ALTER TABLE new_doc DROP partitions for (700), for (500), for (100) update global indexes;
10046 Trace:
1
multiple partition maintenance (black magic) “Single DDL does it all – add, drop, merge, splitting and truncating”
SQL> ALTER TABLE new_doc DROP partitions for (700), for (500), for (100) update global indexes;
SQL> alter system set "_fast_index_maintenance"=false scope=both;
multiple partition maintenance - merge “Single DDL does it all – add, drop, merge, splitting and truncating”
SQL> MERGE partitions SYS_P481 to SYS_P490 INTO partition MERGE_P1 update indexes;
TABLE_NAME PARTITION_NAME INTERVAL NUM_ROWS INDEXING---------- --------------- -------- ---------- --------NEW_DOC MERGE_P1 NO OFFNEW_DOC P0 NO OFFNEW_DOC SYS_P525 YES OFF
P0 (OFF) SYS_P482(200)
SYS_P483(300)
SYS_P481(100)
SYS_P490(400)
MERGE_P1
SYS_Pn(500)
SYS_Pn(700)
global partial index
multiple partition maintenance - merge “Single DDL does it all – add, drop, merge, splitting and truncating”
Range Partitions:
• Partitions need to be adjacent on Range partitioning
•Partitions need to be specified in ascending of their partition bound values
• Keyword “TO” allows to specify lowest and highest partitions for Range Partitioning
List Partitions:
• Partitions don’t need to be adjacent on List Partitioning
• MERGE partition is the union of list values of the partitions merged
•UPDATE INDEXES leaves no orphaned entries on MERGE, but some “unexpected” things may appear
multiple partition maintenance “Single DDL does it all – add, drop, merge, splitting and truncating”
before merge
after merge
Asyncronous / deferred index maintenance conclusions
“Postponed index maintenance to a better time”
• Asynchronous global index maintenance for DROP,TRUNCATE and MODIFY INDEXING OFF Partitions are performed by default: Only affect Oracle dictionary tables (metadata)
• Global index maintenance is be decoupled from the DROP and TRUNCATE partition maintenance;
• Move global index maintenance for off-peak times, or whenever you decide to.
• Maintenance of a list of data object numbers in metadata, where orphaned index entries corresponding to the drop and truncated objects that are invalid are ignored.
Move Partitions Online
Move partitions online “ALTER TABLE ... MOVE PARTITION operation functions is a non-blocking online DDL command”
SQL> @dml SQL> ALTER TABLE move_doc move partition for (3000) TABLESPACE move_ts online update indexes;
SQL> ALTER TABLE move_doc move partition for (3000) TABLESPACE move_ts update indexes; ORA-14020: this physical attribute may not be specified for a table partition ERROR at line 1:ORA-00054: resource busy and acquire with NOWAIT specified or timeout expired
Move partitions online “ALTER TABLE ... MOVE PARTITION operation functions is a non-blocking online DDL command”
SQL> @dml SQL> ALTER TABLE move_doc move partition for (3000) TABLESPACE move_ts online;
INDEX_NAME TABLE_NAME LEAF_BLOCKS UNIQUENES PARTITIONED INDEXING STATUS ORPHANED_ENTRIES -------------------- ---------- ----------- --------- ----------- -------- -------- ----------------- DOC_LOCAL_VALUE_IDX MOVE_DOC 31 NONUNIQUE YES FULL N/A NO DOC_GLOB_VALUE_IDX MOVE_DOC 52 NONUNIQUE NO FULL VALID YES
Move partitions online “ALTER TABLE ... MOVE PARTITION operation functions is a non-blocking online DDL command”
Requires additional disk space and resources (CPU and I/O) for journaling
Index maintenance for global and local indexes
Online, transparent operation!
IOTs are out!
Interval Reference Partitioning
Interval reference partitioning “reference-partitioned table to use interval partitioning as a top partitioning strategy - Child table can inherit the partition characteristics from a parent table ”
TABLE_NAME PARTITION PARTITION_COUNT STATUS DEF_INDEXING ---------- --------- --------------- -------- ------------ DOC RANGE 1048575 VALID ON TRX REFERENCE 1048575 VALID ON
TABLE_NAME PARTITION_NAME INTERVAL NUM_ROWS TABLESPA INDEXING ---------- --------------- -------- ---------- -------- -------- DOC P0 NO USERS ON DOC SYS_P8771 YES USERS ON DOC SYS_P8772 YES USERS ON DOC SYS_P8773 YES USERS ON TRX P0 NO USERS ON TRX SYS_P8771 YES USERS ON TRX SYS_P8772 YES USERS ON TRX SYS_P8773 YES USERS ON
DOCP8771
TRXP8771
DOCp0
DOCP8772
DOCP8773
TRXp0
TRXP877
2
TRXP877
3
Interval reference partitioning Modeling
Interval reference partitioning “reference-partitioned table to use interval partitioning as a top partitioning strategy “
New partitions created automatically only when new data arrives
Child tables maintained automatically
Partition name inherit from current partitions (direct parent relative name – same name as parent )
Interval reference partitioning – CASCADE TRUNCATE “CASCADE option for TRUNCATE and EXCHANGE PARTITION”
Single atomic transaction preserves data integrityEasier, simplier and less error prone CASCADE applies for whole reference tree Cascading TRUNCATE available for non-partitioned tablesON DELETE CASCADE for all foreign keys required
Interval reference partitioning – CASCADE TRUNCATE
Bottom up truncates – 6 truncate operations 1 truncate operation
Interval reference partitioning – CASCADE TRUNCATE “CASCADE option for TRUNCATE and EXCHANGE PARTITION”
Simplify partition maintenance with interval reference partitions CASCADE option to truncate of exchange partition from parent to child.
SQL> ALTER TABLE doc TRUNCATE PARTITION for (299) cascade update indexes; TABLE_NAME PARTITION_NAME INTERVAL NUM_ROWS TABLESPA INDEXING ---------- --------------- -------- ---------- -------- -------- DOC P0 NO 1 USERS ON DOC SYS_P8791 YES 1 USERS ON DOC SYS_P8792 YES 0 USERS ON DOC SYS_P8793 YES 1 USERS ON TRX P0 NO 1 USERS ON TRX SYS_P8791 YES 1 USERS ON TRX SYS_P8792 YES 0 USERS ON TRX SYS_P8793 YES 1 USERS ON
Interval reference partitioning – CASCADE EXCHANGE “CASCADE option for TRUNCATE and EXCHANGE PARTITION”
Single atomic transaction preserves data integrityEasier, simplier and less error prone CASCADE applies for whole reference tree Hierarchy must match target and table to be exchanged
Interval reference partitioning – CASCADE TRUNCATE
1 Exchange operation – complete hierarchy tree
