when smart scan plays dumb, teach exadata some new tricks

Copyright 2013, Zero Defect Computing, Inc.

When Smart Scan Plays Dumb, Teach Exadata Some New Tricks

Jim Czuprynski

Zero Defect Computing, Inc.

April 17, 2013

- 2 -Copyright 2013, Zero Defect Computing, Inc.

My Credentials30+ years of database-centric IT experienceOracle DBA since 2001Oracle 9i, 10g, 11g OCP95+ articles on databasejournal.com and ioug.orgTeach core Oracle DBA courses (G/I+RAC, Exadata,

Performance Tuning, Data Guard)2009: Oracle Education Partner Trainer of the YearSpeaker at Oracle OpenWorld, IOUG

COLLABORATE11, and IOUG COLLABORATE13Oracle-centric blog (Generally, It Depends)


Our AgendaSmart Scan is part of the Exadata Database Machine’s “secret sauce” ... but it’s not perfect. This presentation therefore aims to:Demonstrate how Smart Scan typically reduces

query execution timeIdentify why Smart Scan sometimes “plays dumb”

and chooses sub-optimal retrieval methodsReveal when Smart Scan isn’t the cheapest method

to answer a queryExplore how to resolve these issues by helping

Smart Scan to raise its “query IQ” whenever possible

Copyright 2013, Zero Defect Computing, Inc. - 4 -

Exadata Database I/O

User issues query:SELECT key_id, key_desc FROM ap.ehcc_parted WHERE key_sts <> ‘Y’;

SmartScan-enabled execution plan:TABLE ACCESS STORAGE FULL (AP.EHCC_PARTED)

1

5

2

3AUs

3

6

iDB command generated and issued to Exadata storage cells

Storage cells determine which ASM AUs answer the queryResult set built from

just rows and columns needed to satisfy query

Result set containing just data elements needed is FETCHed

4 rows X 2 cols

80bytes

4

30M rows(4 GB)


Smart Scan and Smart StorageComplex DSS and OLAP queries may need huge

numbers of database blocks to be retrieved, usually via full scans of extremely large row sources (tables as well as indexes)

Smart Scan retrieves only the necessary data as efficiently as possible

Smart Storage determines if corresponding blocks are already in Exadata servers’ buffer caches

Exadata knows not only which database blocks are already available in memory, but how applications are using each block


Smart Flash FeaturesLeverages Exadata storage cells’ write-through-

cache flash memoryOvercomes intrinsic limitations of HDDsSmart Flash Cache extends database’s buffer cache

size dramaticallySmart Flash Logging can accelerate online redo log

writes (and thus speed COMMITs)Flash-based Grid Disks can be deployed for intense

random access requirementsTop rated speed per storage cell: 10K IOPSTotal I/O response time per X2-2 full rack: 1.5M IOPS


Column and Predicate FilteringExadata also dramatically limits the amount of data retrieved when selection criteria and specific columns are mentioned in a SQL statement:Predicate filtering captures only those rows

necessary to answer queryColumn filtering (aka column projection) captures

only those columns necessary to answer queryOnly the data elements (columns X rows) needed

for the statement are returnedThink: SQL Results Query Cache


Storage IndexesAutomatically built in Exadata storage cells’ memoryRegion indexes map which AUs contain ranges of

column data valuesStorage indexes are built upon region indexesStorage cells thus know exactly which AUs can be

used to answer a predicateOnly way to “see” them:

Positive impact on query performanceCell-specific instance statistics

Sensitive to sort order during initial data load


Hybrid Columnar CompressionHybrid Columnar Compression (HCC) provides two

extra data compression methods:Warehouse Compression (10X – 15X): Data that’s

accessed relatively infrequentlyArchival Compression (15X – 50X): Data that’s only

retained for long-term periods or just for regulatory requirements

Compute nodes perform compressionStorage cells perform decompressionCompression units store compressed data


HCC: Compression UnitsCompression Unit

Block HeaderCU Header

C1

C2

Block Header

C2

C3C4

C5

Block Header

C5

C6

Block Header

C8C7

A compression unit (CU) is a new logical structure that spans multiple database blocksRow and column data is completely reorganized based

on individual column valuesEvery column is separately compressedRows do not span multiple CUsSmart Scan benefits dramatically from CUs

Why Didn’t Smart Scan Happen?


Доверяй, но проверяй

Did Smart Scan Really Happen?

So trust the expected EXPLAIN PLAN …

… but then verify:The actual post-execution planInstance statistics specific to Smart ScanWait events related to Smart Scan

(Doveryai, no proveryai)Trust, but verify

As Reagan once said to Gorbachev:


Conditions Preventing Smart ScanFor DSS or Data Warehousing application workloads, Smart Scan may not occur because:Dynamic Sampling is invokedA Flashback Versions Query is requestedPredicates specified for a virtual columnQuery against a LONG or LOB datatypeRow-level dependency is enabledOver 255 columns in a query*

* Unless HCC is used on all tables in query!


Conditions Preventing Smart ScanFor OLTP or hybrid OLTP / DSS application

workloads, other factors may come into play:Fast Full Index Scans are prohibited for:

Reverse key indexes (common for RAC)Compressed indexes (common for parent-child tables)

Full scans on index-organized tables (IOTs)Full scans on clustered tablesORA_ROWSCN virtual column requestedOptimizer decides to return rows in ROWID orderTables are just too small!


A successful Smart Scan of the PK index is shown in both the EXPLAIN PLAN and statistics.

----------------------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |----------------------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 1 | 6 | 1804 (2)| 00:00:22 || 1 | SORT AGGREGATE | | 1 | 6 | | || 2 | PX COORDINATOR | | | | | || 3 | PX SEND QC (RANDOM) | :TQ10000 | 1 | 6 | | || 4 | SORT AGGREGATE | | 1 | 6 | | || 5 | PX BLOCK ITERATOR | | 1500K| 8789K| 1804 (2)| 00:00:22 ||* 6 | INDEX STORAGE FAST FULL SCAN| RANDOMIZED_SORTED_PK | 1500K| 8789K| 1804 (2)| 00:00:22 |----------------------------------------------------------------------------------------------------------

Statistic Name Bytes (MB)-------------------------------------------------------------------------------- ----------physical read total bytes 51.26cell physical IO interconnect bytes 23.33cell physical IO bytes eligible for predicate offload 51.20cell physical IO bytes saved by storage index .00cell physical IO interconnect bytes returned by smart scan 23.27cell IO uncompressed bytes .00

SELECT MIN(key_id), AVG(key_id), MAX(key_id) FROM ap.randomized_sorted WHERE key_id BETWEEN 1000000 AND 2499999;

Smart Scan can use the primary key index for table AP.RANDOMIZED_SORTED:

A Successful Smart Scan


Even though Fast Full Scan of PK index is indicated, statistics and waits show it didn’t happen at all!

-------------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |-------------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 1 | 4 | 5 (0)| 00:00:01 || 1 | SORT AGGREGATE | | 1 | 4 | | || 2 | INDEX STORAGE FAST FULL SCAN| INVOICES_PK_IDX | 4500 | 18000 | 5 (0)| 00:00:01 |-------------------------------------------------------------------------------------------------Statistic Name Bytes (MB)-------------------------------------------------------------------------------- ----------physical read total bytes .22physical write total bytes .00cell physical IO interconnect bytes .22cell physical IO bytes eligible for predicate offload .00cell physical IO bytes saved by storage index .00cell physical IO interconnect bytes returned by smart scan .00cell IO uncompressed bytes .00

Total Total Wait Avg WaitWait Event Waits Time (s) Time (s)------------------------------------------------------------ ---------- ---------- ----------cell single block physical read 8 .02 .0031cell multiblock physical read 3 0 .0011

SELECT MIN(invoice_id), MAX(invoice_id) FROM ap.invoices;

In this example, the primary key index for table AP.INVOICES cannot be used for Smart Scan:

Impact: REVERSE KEY Index


Conditions Preventing Smart ScanSome factors may be incidental, difficult to isolate, and perhaps non-negotiable:Indexed retrieval may still be faster than table scansInitialization parameters may affect optimizer

decisions (e.g. OPTIMIZER_INDEX_COST_ADJUST)Parallel processing may not have been chosen“Hidden” initialization parameters actually govern

whether Smart Scan can be utilized


Secret Initialization ParametersThree hidden parameters impact Smart Scan:_small_table_threshold

Determines if a table is “small”Specified in number of blocks (e.g. 512 x 8KB = 4MB)

_serial_direct_readWill force Smart Scan when there’s no parallelismDefault: FALSE

_direct_read_decision_statistics_drivenNew in Oracle Database 11.2.0.3.1 Set to TRUE by default


Fast Full Scan of PK index fails because the total number of blocks scanned is well below the size of _smart_table_threshold

------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 1 | 6 | 441 (1)| 00:00:06 || 1 | SORT AGGREGATE | | 1 | 6 | | ||* 2 | INDEX RANGE SCAN| RANDOMIZED_SORTED_PK | 200K| 1171K| 441 (1)| 00:00:06 |------------------------------------------------------------------------------------------

Statistic Name Bytes (MB)-------------------------------------------------------------------------------- ----------physical read total bytes 3.55cell physical IO interconnect bytes 3.55cell physical IO bytes eligible for predicate offload .00cell physical IO bytes saved by storage index .00cell physical IO interconnect bytes returned by smart scan .00cell IO uncompressed bytes .00

Total Total Wait Avg WaitWait Event Waits Time (s) Time (s)------------------------------------------------------------ ---------- ---------- ----------cell single block physical read 454 .22 .0005

SELECT MIN(key_id), AVG(key_id), MAX(key_id) FROM ap.randomized_sorted WHERE key_id BETWEEN 1000000 AND 1199999;

If a table or index is < 2% of current buffer cache size, then it’s excluded from Smart Scan:

Impact: _small_table_threshold

Smart Scan: The Good, The Bad, and The Ugly

(With apologies to Sergio Leone)


SET SERVEROUTPUT ONDECLARE comp_blks NUMBER; unco_blks NUMBER; pcti_blks NUMBER; comp_rspb NUMBER; unco_rspb NUMBER; comp_ratio NUMBER; comp_type VARCHAR2(100); pcti_rspb NUMBER;BEGIN DBMS_COMPRESSION.GET_COMPRESSION_RATIO ( scratchtbsname => 'USERS' ,ownname => 'AP' ,tabname => 'RANDOMIZED_SORTED' ,partname => NULL ,comptype => DBMS_COMPRESSION.COMP_FOR_QUERY_HIGH ,blkcnt_cmp => comp_blks ,blkcnt_uncmp => unco_blks ,row_cmp => comp_rspb ,row_uncmp => unco_rspb ,cmp_ratio => comp_ratio ,comptype_str => comp_type ); pcti_blks := ((comp_blks / unco_blks) * 100); pcti_rspb := ((comp_rspb / unco_rspb) * 100); DBMS_OUTPUT.PUT_LINE('Compression Type: ' || comp_type); DBMS_OUTPUT.PUT_LINE('Compression Ratio: ' || TO_CHAR(comp_ratio, '999999.99')); DBMS_OUTPUT.PUT_LINE('Uncompressed Blocks: ' || TO_CHAR(unco_blks, '999999999')); DBMS_OUTPUT.PUT_LINE('Compressed Blocks: ' || TO_CHAR(comp_blks, '999999999')); DBMS_OUTPUT.PUT_LINE('Block Compression %: ' || TO_CHAR(pcti_blks, '999999.99')); DBMS_OUTPUT.PUT_LINE('Uncompressed Rows Per Block: ' || TO_CHAR(unco_rspb, '999999999')); DBMS_OUTPUT.PUT_LINE('Compressed Rows Per Block: ' || TO_CHAR(comp_rspb, '999999999')); DBMS_OUTPUT.PUT_LINE('Rows Per Block Compression %: ' || TO_CHAR(pcti_rspb, '999999.99'));END;/

Compression Advisor self-check validation successful. select count(*) on both Uncompressed and EHCC Compressed format = 3000000 rowsCompression Type: "Compress For Query Low"Compression Ratio: 2.90Uncompressed Blocks: 15654Compressed Blocks: 5305Block Compression %: 33.89Uncompressed Rows Per Block: 191Compressed Rows Per Block: 566Rows Per Block Compression %: 296.34

Using DBMS_COMPRESSION

DBMS_COMPRESSION Results for AP.RANDOMIZED_SORTED

Compression Level: WAREHOUSE LOW

WAREHOUSE HIGH

ARCHIVE LOW

ARCHIVE HIGH

Compression Ratio 2.9% 5.4% 5.5% 7.4%

Compressed Blocks 5305 2885 2833 2090

Block Compression Ratio

33.9% 18.4% 18.1% 13.5%

Compressed Rows Per Block

566 1040 1059 1435

Rows Per Block Compression Ratio

296.3% 544.5% 554.5% 751.3%

Original table statistics: 3M rows, 15654 blocks


---------------------------------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |---------------------------------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 3184K| 60M| | 3506 (4)| 00:00:43 || 1 | PX COORDINATOR | | | | | | || 2 | PX SEND QC (ORDER) | :TQ10006 | 3184K| 60M| | 3506 (4)| 00:00:43 || 3 | SORT ORDER BY | | 3184K| 60M| | 3506 (4)| 00:00:43 || 4 | PX RECEIVE | | 3184K| 60M| | 3506 (4)| 00:00:43 || 5 | PX SEND RANGE | :TQ10005 | 3184K| 60M| | 3506 (4)| 00:00:43 || 6 | SORT GROUP BY | | 3184K| 60M| | 3506 (4)| 00:00:43 || 7 | PX RECEIVE | | 3184K| 60M| | 3506 (4)| 00:00:43 || 8 | PX SEND HASH | :TQ10004 | 3184K| 60M| | 3506 (4)| 00:00:43 || 9 | SORT GROUP BY ROLLUP | | 3184K| 60M| | 3506 (4)| 00:00:43 || 10 | VIEW | | 3184K| 60M| | 3468 (3)| 00:00:42 || 11 | SORT UNIQUE | | 3184K| 24M| 49M| 3468 (53)| 00:00:42 |…Statistic Name Bytes (MB)-------------------------------------------------------------------------------- ----------physical read total bytes 54.27cell physical IO interconnect bytes 25.25cell physical IO bytes eligible for predicate offload 47.66cell physical IO interconnect bytes returned by smart scan 18.64cell IO uncompressed bytes 180.51

Total Total Wait Avg WaitWait Event Waits Time (s) Time (s)------------------------------------------------------------ ---------- ---------- ----------cell list of blocks physical read 1 0 .0005cell single block physical read 8 .05 .0056

EHCC_PARTED: Partitioned on KEY_DATE, so parallelism can be used effectively

--------------------------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |--------------------------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 3184K| 60M| | 35109 (2)| 00:07:02 || 1 | SORT GROUP BY ROLLUP | | 3184K| 60M| | 35109 (2)| 00:07:02 || 2 | VIEW | | 3184K| 60M| | 34961 (2)| 00:07:00 || 3 | SORT UNIQUE | | 3184K| 24M| 49M| 34961 (56)| 00:07:00 || 4 | UNION-ALL | | | | | | || 5 | HASH GROUP BY | | 1629K| 12M| 25M| 15808 (2)| 00:03:10 ||* 6 | TABLE ACCESS STORAGE FULL| RANDOMIZED_UNSORTED | 1652K| 12M| | 3899 (2)| 00:00:47 || 7 | HASH GROUP BY | | 1184K| 9256K| 18M| 12555 (2)| 00:02:31 |. . .------------------------------------------------------------------------------------------------------------Statistic Name Bytes (MB)-------------------------------------------------------------------------------- ----------physical read total bytes 330.12cell physical IO interconnect bytes 42.33cell physical IO bytes eligible for predicate offload 330.07cell physical IO interconnect bytes returned by smart scan 42.29cell IO uncompressed bytes 330.30

Total Total Wait Avg WaitWait Event Waits Time (s) Time (s)------------------------------------------------------------ ---------- ---------- ----------cell smart table scan 62 .05 .0008cell single block physical read 6 .01 .001

RANDOMIZED_UNSORTED: Same 30M rows, sorted in KEY_DESC order, but no parallelism------------------------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |------------------------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 3184K| 60M| | 36716 (2)| 00:07:21 || 1 | SORT GROUP BY ROLLUP | | 3184K| 60M| | 36716 (2)| 00:07:21 || 2 | VIEW | | 3184K| 60M| | 36568 (2)| 00:07:19 || 3 | SORT UNIQUE | | 3184K| 24M| 49M| 36568 (56)| 00:07:19 || 4 | UNION-ALL | | | | | | || 5 | HASH GROUP BY | | 1629K| 12M| 25M| 16344 (2)| 00:03:17 ||* 6 | TABLE ACCESS STORAGE FULL| RANDOMIZED_SORTED | 1652K| 12M| | 4435 (1)| 00:00:54 || 7 | HASH GROUP BY | | 1184K| 9256K| 18M| 13091 (2)| 00:02:38 |. . .------------------------------------------------------------------------------------------------------------Statistic Name Bytes (MB)-------------------------------------------------------------------------------- ----------physical read total bytes 368.49cell physical IO interconnect bytes 42.56cell physical IO bytes eligible for predicate offload 368.09cell physical IO interconnect bytes returned by smart scan 42.15cell IO uncompressed bytes 368.33

Total Total Wait Avg WaitWait Event Waits Time (s) Time (s)------------------------------------------------------------ ---------- ---------- ----------cell smart table scan 73 .05 .0006cell multiblock physical read 1 0 .0006cell single block physical read 49 .03 .0005

RANDOMIZED_SORTED: 30M rows, sorted in KEY_ID order, but no parallelism possible

HCC can make incredible differences in performance!

Hybrid Columnar Compression ImpactSELECT yrnbr, wknbr, SUM(counted) FROM (SELECT TO_CHAR(key_date, 'yyyy') yrnbr ,TO_CHAR(key_date, 'ww') wknbr ,COUNT(*) counted FROM ap.randomized_sorted WHERE key_date BETWEEN TO_DATE('2003-07-01', 'yyyy-mm-dd') AND TO_DATE('2008-12-31', 'yyyy-mm-dd') GROUP BY TO_CHAR(key_date, 'yyyy') ,TO_CHAR(key_date, 'ww')UNIONSELECT TO_CHAR(key_date, 'yyyy') yrnbr ,TO_CHAR(key_date, 'ww') wknbr ,COUNT(*) counted FROM ap.randomized_sorted WHERE key_date BETWEEN TO_DATE('2007-06-30', 'yyyy-mm-dd') AND TO_DATE('2011-06-30', 'yyyy-mm-dd') GROUP BY TO_CHAR(key_date, 'yyyy') ,TO_CHAR(key_date, 'ww')UNIONSELECT TO_CHAR(key_date, 'yyyy') yrnbr ,TO_CHAR(key_date, 'ww') wknbr ,COUNT(*) counted FROM ap.randomized_sorted WHERE key_date >= TO_DATE('2011-10-01', 'yyyy-mm-dd') GROUP BY TO_CHAR(key_date, 'yyyy') ,TO_CHAR(key_date, 'ww')) GROUP BY ROLLUP (yrnbr, wknbr) ORDER BY yrnbr, wknbr;;


Impact of Sorted Row SourcesColumns typically used as predicate targets can benefit dramatically from sorting during initial load

A simple query against data sorted on ascending KEY_ID order when loaded:

SELECT * FROM ap.randomized_sortedWHERE key_id BETWEEN 110000 AND 130000;

----------------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |----------------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 20002 | 644K| 186 (0)| 00:00:03 || 1 | TABLE ACCESS BY INDEX ROWID| RANDOMIZED_SORTED | 20002 | 644K| 186 (0)| 00:00:03 ||* 2 | INDEX RANGE SCAN | RANDOMIZED_SORTED_PK | 20002 | | 46 (0)| 00:00:01 |----------------------------------------------------------------------------------------------------Statistic Name Bytes (MB)-------------------------------------------------------------------------------- ----------physical read total bytes 1.21cell physical IO interconnect bytes 1.21cell physical IO bytes eligible for predicate offload .00cell physical IO bytes saved by storage index .00cell physical IO interconnect bytes returned by smart scan .00cell IO uncompressed bytes .00

Total Total Wait Avg WaitWait Event Waits Time (s) Time (s)------------------------------------------------------------ ---------- ---------- ----------cell single block physical read 155 .09 .0006

SELECT * FROM ap.randomized_unsortedWHERE key_id BETWEEN 110000 AND 130000;

-------------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |-------------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 20002 | 644K| 3893 (1)| 00:00:47 ||* 1 | TABLE ACCESS STORAGE FULL| RANDOMIZED_UNSORTED | 20002 | 644K| 3893 (1)| 00:00:47 |-------------------------------------------------------------------------------------------------Statistic Name Bytes (MB)-------------------------------------------------------------------------------- ----------physical read total bytes 110.09cell physical IO interconnect bytes .83cell physical IO bytes eligible for predicate offload 110.02cell physical IO bytes saved by storage index .27cell physical IO interconnect bytes returned by smart scan .76cell IO uncompressed bytes 109.76

Total Total Wait Avg WaitWait Event Waits Time (s) Time (s)------------------------------------------------------------ ---------- ---------- ----------cell single block physical read 8 .02 .0028cell smart table scan 45 .05 .0012

And now the same query, but columns not sorted in KEY_ID order, results in a full table scan!


Is Smart Scan Always Optimal?Smart Scan may actually be more expensive than expected when:Row source row migration is excessiveRow sources are partially or completely modified

and these changes haven’t been committed yetStorage cell CPUs are overloaded

These cases may be unavoidable without changing application logic or execution patterns!


Excessive Row MigrationRow migration requires additional mandatory single-block reads to retrieve the migrated rows:

This query uses a scalar function to filter about 18% of all rows based on the contents of the KEY_DESC column.

SELECT * FROM ap.randomized_unsortedWHERE UPPER(SUBSTR(key_desc,1,3)) IN ('AAA','BBB','CCC','XXX','YYY','ZZZ');

-------------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |-------------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 175K| 5657K| 4514 (15)| 00:00:55 ||* 1 | TABLE ACCESS STORAGE FULL| RANDOMIZED_UNSORTED | 175K| 5657K| 4514 (15)| 00:00:55 |-------------------------------------------------------------------------------------------------Statistic Name Bytes (MB)-------------------------------------------------------------------------------- ----------physical read total bytes 110.08cell physical IO interconnect bytes 20.72cell physical IO bytes eligible for predicate offload 110.02cell physical IO interconnect bytes returned by smart scan 20.67cell IO uncompressed bytes 110.12


Then, those same rows are updated and the changes committed. Since the row size grows, row migration occurs.

Note the resulting increase in physical reads and cell physical I/O interconnect bytes!

-------------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |-------------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 175K| 12M| 9814 (7)| 00:01:58 ||* 1 | TABLE ACCESS STORAGE FULL| RANDOMIZED_UNSORTED | 175K| 12M| 9814 (7)| 00:01:58 |-------------------------------------------------------------------------------------------------Statistic Name Bytes (MB)-------------------------------------------------------------------------------- ----------physical read total bytes 256.30cell physical IO interconnect bytes 30.33cell physical IO bytes eligible for predicate offload 256.20cell physical IO interconnect bytes returned by smart scan 30.22cell IO uncompressed bytes 256.40

Total Total Wait Avg WaitWait Event Waits Time (s) Time (s)------------------------------------------------------------ ---------- ---------- ----------cell single block physical read 10 .01 .0005cell multiblock physical read 1 0 .0006cell smart table scan 41 .02 .0005


Partially Updated Row SourcesIf a query’s row sources have been partially modified, but those changes are as yet uncommitted, it will cost significantly more to retrieve the unmodified data:

We’ll use the identical query, which filters an 18% sample of the entire table based on the KEY_DESC column’s contents.

SELECT * FROM ap.randomized_unsortedWHERE UPPER(SUBSTR(key_desc,1,3)) IN ('AAA','BBB','CCC','XXX','YYY','ZZZ');

-------------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |-------------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 175K| 5657K| 4514 (15)| 00:00:55 ||* 1 | TABLE ACCESS STORAGE FULL| RANDOMIZED_UNSORTED | 175K| 5657K| 4514 (15)| 00:00:55 |-------------------------------------------------------------------------------------------------Statistic Name Bytes (MB)-------------------------------------------------------------------------------- ----------physical read total bytes 110.08cell physical IO interconnect bytes 20.72cell physical IO bytes eligible for predicate offload 110.02cell physical IO interconnect bytes returned by smart scan 20.67cell IO uncompressed bytes 110.12


Then, about 30% of the table’s rows are updated … but since the row size grows, it forces row migration:

Note the significant increases in physical reads, cell physical I/O interconnect bytes, and waits.


Total Total Wait Avg WaitWait Event Waits Time (s) Time (s)------------------------------------------------------------ ---------- ---------- ----------cell single block physical read 21153 14.04 .0007cell list of blocks physical read 1 0 .0009cell smart table scan 140 .05 .0003


This query will retrieve a significant number of rows (well over 99%) from the table.

SELECT * FROM ap.randomized_unsorted WHERE key_sts = 'Y'


Total Total Wait Avg WaitWait Event Waits Time (s) Time (s)------------------------------------------------------------ ---------- ---------- ----------cell smart table scan 100 .03 .0003cell single block physical read 7 .02 .0035

Then all of the table’s rows are updated by changing KEY_STS to “Z”, but again, no changes are committed.

Note the horrendous increase in physical reads, cell physical I/O interconnect bytes, and waits!

-------------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |-------------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 2999K| 94M| 3929 (2)| 00:00:48 ||* 1 | TABLE ACCESS STORAGE FULL| RANDOMIZED_UNSORTED | 2999K| 94M| 3929 (2)| 00:00:48 |------------------------------------------------------------------------------------------------- Statistic Name Bytes (MB)-------------------------------------------------------------------------------- ----------physical read total bytes 555.61cell physical IO interconnect bytes 555.64cell physical IO bytes eligible for predicate offload 110.02cell physical IO interconnect bytes returned by smart scan 110.05cell IO uncompressed bytes 110.02

Total Total Wait Avg WaitWait Event Waits Time (s) Time (s)------------------------------------------------------------ ---------- ---------- ----------cell single block physical read 57035 34.45 .0006cell smart table scan 98 .04 .0004

If a query’s row sources have been completely changed but those changes are as yet committed, the additional retrieval costs could be enormous!

Fully Updated Row Sources


Statistic Name Bytes (MB)-------------------------------------------------------------------------------- ----------physical read total bytes 175.80physical write total bytes .00cell physical IO interconnect bytes 28.33cell physical IO bytes sent directly to DB node to balanceCPU u 24.16cell physical IO bytes saved during optimized file creation .00cell physical IO bytes saved during optimized RMAN file restore .00cell physical IO bytes eligible for predicate offload 175.80cell physical IO bytes saved by storage index .00cell physical IO interconnect bytes returned by smart scan 28.23table fetch continued row .00cell IO uncompressed bytes 175.80

Total Total Wait AvgEvent Waits Time (s) Time (s)------------------------------------------------------------ ---------- ---------- ----------cell smart table scan 3178 18.36 0.17

Exadata may simply relocate query operations back to the buffer cache of one or more compute nodes when it detects a storage cell’s CPUs are overloaded:

Overloaded CPUs on Storage Cells


Other Resources

Tanel Poder’s excellent recent blogs on Smart ScanFritz Hoogland’s recent blogging on

Oracle Exadata Smart Scan internals IOUG 2012 Exadata SIG presentation on Smart Scan

from UBS Bank

If I have seen farther than others, it is because I was standing on the shoulders of giants.

- Sir Isaac Newton

http://blog.tanelpoder.com/

http://blog.tanelpoder.com/

http://fritshoogland.wordpress.com/2013/03/31/observing-how-oracle-exadata-smartscans-internals-work-part-1/


Thank You For Your Kind AttentionPlease fill out a session evaluation form!

Session P65When Smart Scan Plays Dumb, Teach Exadata Some New Tricks

If you have any questions or comments, feel free to:E-mail me at jczuprynski@zerodefectcomputingcomFollow my blog (Generally, It Depends):

http://jimczuprynski.wordpress.comFollow me on Twitter (@jczuprynski)Connect with me on LinkedIn (Jim Czuprynski)

mailto:[email protected]

http://jimczuprynski.wordpress.com/

when smart scan plays dumb, teach exadata some new tricks

Documents