Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
NOTICE: Proprietary and Confidential. This material is proprietary to Rolta and contains trade secret and confidential information which is solely the property of Rolta. This material is solely for Client's internal use. This material shall not be used, reproduced, copied, disclosed, and transmitted, in whole or in part, without the express consent of Rolta.
Partitioning and Compression for
Performance and Manageability
1
Michael R. Messina, Senior Managing ConsultantAdvizeX, A Rolta Company Infrastructure Services
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Speaker Introduction
•Michael Messina
•Senior Managing Consultant AdvizeX, A RoltaCompany
•Working with Oracle Approximately 20 years
•Background includes Performance Tuning, High Availability and Disaster Recovery
•Oracle Database OCP
•Oracle RAC Certified Expert
•Oracle Exadata Implementation Specialist
•Oracle ACE
•www.rolta.com / www.advizex.com
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Audience Experience
•How many have utilized Partitioning?
•What have been your experiences?
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Audience Experience
•How many have
utilized Table
Compression and/or
index compression
(Prior to 11g)?
•What are your
thoughts/experiences?
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Audience Experience
•How many are using Advanced Compression
in Production? Thoughts?
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Audience Experience
•Anyone using Hybrid
Columnar Compression?
•What are your thoughts
Experiences?
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Goals•Touch on industry challenges
▪ Explosive Data Growth
▪ Performance Degrading
▪ Costs
•Look at Partitioning Options best for manageability
offering best consistent performance
•Examine 11g Advanced Compression
•Examine Hybrid Columnar Compression
•Show how Partitioning and Compression together
help address some of the industry challenges
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Industry Challenges
•Exploding Data Growth
▪ Got to keep up
•Performance
▪ Query Performance Degradation as data volumes increase
▪ Backup time increases as data volumes increase
•Costs - What are the True Costs?
▪ Disk Space Purchase / Backup / Space Management / Power / Cooling
•What can we do??
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Exploding Data Growth• If you think storing data is a challenge now, it's
nothing compared to what it could be in just a few years. Data Growth of 60% is common.
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Performance“Storage capacity grows at
over 60% per year while performance improves at less than 10% per year. This trend has existed for over 10 years and is expected to continue for the foreseeable future.”
BNET
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Releasing Database Performance
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Partitioning for Performance•Ref Partitioning
▪ Introduced with 11g
▪ Improves performance for parent child relationships
▪ Partitions the child with the parent
•Interval Partitioning
▪ Introduced with 11g
▪ Same performance
benefits as Range
partitioning
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Partitioning for
Manageability•Interval Partitioning
▪ 11g and above
▪ Defined using an interval
▪ Works much like Range
Partitioning
▪ Partitions are created as needed eliminates need to manually add partitions.
•Ref Partitioning ▪ 11g and above
▪ Simpler partition management, child partitions created automatically when parent partitions are created
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
True Disk CostsThe cost of Storage for the Enterprise is greatly influenced by
performance requirements and redundancy which = usable capacity costs. All costs are in a per year $
IOPS
Est.
Cost/GB
Useable
Est.
GB Util
Rate
Real Cost
per GB
SATA 7k
rpm RAID 6
80 $42 80% $50
SAS 10k
rpm RAID 5
120 $84 80% $100
SAS 15k
rpm RAID
10
170 $169 80% $202
** Remember power, cooling, machine room space, switches, shelves, cabinets, backup, maintenance (20%), RAID protection, Managementie. People (30%) and other factors for SANs you can not just look at drive cost as at least 50% of cost is management and Maintenance.
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Compression•Index Compression since 8i
▪ Compress Indexes
▪ Works best on indexes with repeating values
•Table Compression since 9i
▪ No Additional License Requirement
▪ Only for direct inserts
▪ Compression Not Maintained with updates and normal inserts
▪ Had to re-org table to re-compress over time.
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Compression
•Advanced Compression 11g
▪ Additional License Requirement
▪ Compression Maintained with all DML activity
▪ No re-orgs required after initial
compression
•Hybrid Columnar Compression
▪ Introduced with Exadata
▪ Query High, Query Low, Archive High, Archive
Low compression modes.
▪ Exadata, ZFS
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Compression
•Check if Row is CompressedSELECT
DECODE(DBMS_COMPRESSION.GET_COMPRESSION_TYPE(
ownname => ‘TABLEOWNERHERE',
tabname => ‘TABLENAMEHERE',
row_id => ‘ROWIDHERE'),
1, 'No Compression',
2, 'Basic or OLTP Compression',
4, 'Hybrid Columnar Compression for Query High',
8, 'Hybrid Columnar Compression for Query Low',
16, 'Hybrid Columnar Compression for Archive High',
32, 'Hybrid Columnar Compression for Archive Low',
'Unknown Compression Type') compression_type
FROM DUAL;
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Compression•What can compression
accomplish?
▪ Shrink size of tables?
▪ Shrink Size of indexes?
▪ Improve buffer cache
utilization?
▪ Improve I/O disk visits?
▪ Improve performance?
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
What can we do•Reduce Size of Existing?
▪ Can we get a 10%, 20%, 30% reduction or more?
•Reduce Size of Future Data?
▪ Can we impact growth by 10%, 20%, 30% or more?
•Minimize performance impact of larger data volumes?
▪ Disk Space, Backup/Recovery, Server Resources
•Can we do all this without adding significant management overhead to the DBA?
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Ref Partitioning
•Examine Space Impact of Partitioning
▪ Show disk space impact partitioned and un-
partitioned tables.
•Examine the true performance gain from Ref
Partitioning
▪ Demonstrate the partitioned and un-partitioned
performance impact for queries.
▪ Demonstrate the partitioned and compressed
performance impact on queries.
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Ref Partitioning –
Un-Partitioned Table Size
•ORDERS (78880 rows)SUM(BYTES)/1024
---------------
4096
•ORDER_ITEMS (499792 rows)
SUM(BYTES)/1024
---------------
16384
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Ref Partitioning Impact on Table Sizes
•ORDERS (78880 rows)SUM(BYTES)/1024
---------------
4736
•ORDER_ITEMS (499792 rows)SUM(BYTES)/1024
---------------
13950
* Surprisingly we see the
child table size reduced
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Ref –
Non Partitioned Table PerformanceSELECT o.order_date,
sum(oi.unit_price*oi.quantity) order_total
FROM oe.orders o, oe.order_items oi
WHERE o.order_date BETWEEN TO_DATE('01-APR-
1999','DD-MON-YYYY') AND TO_DATE('30-JUN-
1999','DD-MON-YYYY') AND o.order_id =
oi.order_id
GROUP BY order_date
ORDER BY order_date ;
..
16 rows selected.
Elapsed: 00:00:00.93
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Ref –
Non Partitioned Table PerformanceStatistics-----------------------------------------
1 recursive calls0 db block gets
1967 consistent gets1964 physical reads
0 redo size970 bytes sent via SQL*Net to client427 bytes received via SQL*Net from client3 SQL*Net roundtrips to/from client1 sorts (memory)0 sorts (disk)16 rows processed
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Ref Partitioning ImpactSELECT o.order_date,
sum(oi.unit_price*oi.quantity) order_total
FROM oe.orders o, oe.order_items oi
WHERE o.order_date BETWEEN TO_DATE('01-
APR-1999','DD-MON-YYYY') AND TO_DATE('30-JUN-
1999','DD-MON-YYYY') AND o.order_id =
oi.order_id
GROUP BY order_date
ORDER BY order_date ;
..
16 rows selected.
Elapsed: 00:00:00.57
* .93 to .57 / 38% Improvement
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Ref Partitioning ImpactStatistics-----------------------------------------44 recursive calls0 db block gets
1630 consistent gets1621 physical reads
0 redo size896 bytes sent via SQL*Net to client427 bytes received via SQL*Net from client
3 SQL*Net roundtrips to/from client1 sorts (memory)0 sorts (disk)
16 rows processed
* PIO - from 1967 to 1630 / 17% ImprovementLIO – from 1964 to 1621 / 17% Improvement
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Interval Partitioning•Examine Space Impact of Range-Interval
Partitioning
▪ Show disk space impact partitioned and un-partitioned.
•Examine the true performance gain from Interval
Partitioning
▪ Demonstrate the partitioned and un-partitioned
performance
▪ Demonstrate the partitioned and compressed
performance
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Interval -
Non-Partitioned Table Size
•Un-Partitioned Table 6,290,116 Rows
SUM(BYTES)/1024/1024
--------------------
320
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Interval –
Partitioning Impact on Table Size
•Partitioned Table 6,290,116 Rows, 20 partitions
SUM(BYTES)/1024/1024
--------------------
464
* 320M to 464M represents and
increase in size when table is
partitioned of 144MB .
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Interval - Non Partitioned Table
Performance
•Un-Partitioned Table 6,290,116 RowsSQL> select deptno, avg(sal)
from emp
where hiredate
between to_date('01-JAN-1982', 'DD-MON-YYYY') and to_date('01-JAN-1983', 'DD-MON-YYYY')
group by deptno ;
..
Elapsed: 00:00:02.85
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Interval –
Non Partitioned Table StatisticsStatistics--------------------------------
0 recursive calls 0 db block gets
40465 consistent gets 40462 physical reads
0 redo size 684 bytes sent via SQL*Net to client 524 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 3 rows processed
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Interval Partitioning Impact•Partitioned Table 6,290,116 Rows SQL> select deptno, avg(sal)
from emp_part
where hiredate
between to_date('01-JAN-1982',
'DD-MON-YYYY') and to_date('01-JAN-
1983', 'DD-MON-YYYY')
group by deptno ;
..
Elapsed: 00:00:00.32
* 2.85 to 0.32 / 88% Improvement
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Interval Partitioning ImpactStatistics--------------------------------
7 recursive calls 0 db block gets
4358 consistent gets 4354 physical reads
0 redo size 684 bytes sent via SQL*Net to client 524 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 3 rows processed
• LIO – 40465 to 4358 / 89% Improvement• PIO – 40462 to 4354 / 89% Improvement
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Accomplished With Partitioning
•Positive
▪ Reduced logical I/O
▪ Reduced Physical I/O
▪ Improved elapse time
•Negative
▪ Increased the size of the table
utilizing more disk space
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Impact of Compression on Size of
Ref-Partitioned Tables•ORDERS (78880 rows)
SUM(BYTES)/1024
---------------
4352
* 8% reduction over partitioned table 5% increase on Original table.
•ORDER_ITEMS (499792 rows)SUM(BYTES)/1024
---------------
11520
* 29% reduction over partitioned table
17% reduction over Original table
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Impact of Ref Partitioning and
Compression TogetherElapsed: 00:00:00.43Statistics-------------------------------------1 recursive calls0 db block gets
413 consistent gets407 physical reads 0 redo size
896 bytes sent via SQL*Net to client427 bytes received via SQL*Net from client3 SQL*Net roundtrips to/from client1 sorts (memory)0 sorts (disk)16 rows processed
* .43 seconds - 53% improvement to original / 24% improvement partitioned un-compressed
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Impact of Compression on Size of
Interval Partitioned Table
• Partitioned Table 6,290,116 Rows
SUM(BYTES)/1024/1024
--------------------
312
•33% reduction on partition and
uncompressed table
2.5% reduction from original
table
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Impact of Interval Partitioning and
Compression TogetherElapsed: 00:00:00.11
Statistics---------------------------------
1 recursive calls 0 db block gets
3030 consistent gets 3026 physical reads
0 redo size 546 bytes sent via SQL*Net to client 416 bytes received via SQL*Net from client 2 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 3 rows processed
• Time Over Original 2.85 to .11 a 96% Imprv.Over Partitioned .32 to .11 a 65% Improv.
• PIO Over Original 40462 to 3026 a 93% Imprv.Over Partitioned 4358 to 3026 a 31% Imprv.
• LIO Over Original 40465 to 3030 a 93% Imprv.Over Partitioned 4354 to 3030 a 30% Imprv.
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Oracle Database 12c
Improvements•Can have interval partitioned table with REF
Partitioned child table now.
▪ Further improves manageability by allowing parent table
partitions to have partitions auto created in addition to
auto partitions created for REF Partitioned table.
•Partial Indexes for Partitioned Tables
▪ Only index partitions that are used
▪ Saves Disk Space only maintaining indexes on
partitions that are utilized.
•ONLINE Move Partition
▪ Reduce outages for Partition Maintenance.
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Hybrid Columnar Compression
•Select Storage Systems
▪ Exadata
▪ ZFS
•Offers Greater Levels of Compression
•Must use Insert Append to be able to
compress like much like Traditional Table
Compression prior to 11g
•Compression not maintained during updates.
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
HCC Compression
•Query High (Sample)No
Compress Compress Reduction % Reduction
903 344 559 61.90
1088 408 680 62.50
960 361 599 62.40
1088 416 672 61.76
1152 400 752 65.28
1091 400 691 63.33
1216 456 760 62.50
1112 408 704 63.31
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
HCC Compression
•Archive High (Sample)No
Compress Compress Reduction(MB) % Reduction
903 264 639 70.76
1088 304 784 72.06
960 272 688 71.67
1088 312 776 71.32
1152 336 816 70.83
1091 328 763 69.94
1216 352 864 71.05
1112 304 808 72.66
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Partitioning and Compression
Summary
•What can partitioning accomplish
▪ Improve Performance▪ Break large table into chunks reducing I/O
▪ Reduction in I/O though only reading partitions needed
▪ Minimize Management Cost▪ Utilize interval and Ref partitioning where new
partitions are created automatically.
▪ Manage though individual Partitions adding flexibility for Table and index Management
▪ Improve Database backup Performance▪ Mark tablespaces holding older Data partitions
Read-Only as it eliminates the need to backup with each full backup of the database.
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Partitioning and Compression
Summary
•What Can Compression Accomplish?
▪ Reduce Disk Space Costs
▪Compress partitioned tables reducing the size of tables improve space impact of partitioning
▪ Improve Performance
▪Compress tables to reduce I/O read operations
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet
Partitioning Conclusions•Partitioning can improve I/O utilization
•Partitioning can improve performance
•Partitioning increases space utilization
•Compression reduces space utilization and minimizes the space impact of partitioning
•Compression can improve performance
•Compression with partitioning can improve performance more then either of them alone and can reduce space utilization.
•Interval Partitioning and Ref Partitioning reduces maintenance impact for using partitioning
Rolta Proprietary & Confidential June 26, 2017
ROLTA Where Expertise & Technology Meet