io waits kyle hailey kyle_hailey@hotmail.com
Post on 20-Dec-2015
218 Views
Preview:
TRANSCRIPT
IO Waits
Kyle Hailey
Kyle_hailey@hotmail.com
http://perfvision.com/ftp/sb
#.2
Copyright 2006 Kyle Hailey
Waits Covered in this Section1. db file sequential read2. db file scattered read3. db file parallel read4. read by other session5. direct path read 6. direct path write7. direct path read temp8. direct path write temp9. direct path write (lob)10. file open11. file identify12. Local write wait13. data file init write14. sort segment request
Standard I/OStandard I/O
Direct I/ODirect I/O
Special CasesSpecial Cases
#.3
Copyright 2006 Kyle Hailey
Buffer Cache
REDOREDO Lib Lib CacheCache
Buffer Buffer CacheCache
LocksLocks
NetworkNetwork
I/OI/O
#.4
Copyright 2006 Kyle Hailey
IO Waits
Data FilesData Files
DBWRDBWR
Buffer Buffer CacheCache
User1User1
SGASGA
Library Library CacheCache
Log Log BufferBuffer
#.5
Copyright 2006 Kyle Hailey
Standard Table/Index IO
1. db file sequential read Single block read
2. db file scattered read Multi block read
3. db file parallel read Non-contiguous multi block read
4. read by other session Wait for another session to do the io
#.6
Copyright 2006 Kyle Hailey
db file sequential read
ShadowShadowProcessProcess
1) Search Buffer Cache for the block by rowid
2) Fails3) Reads of disk
• File• Block
Buffer CacheBuffer Cache The top wait Single Block Read
Data via Index and/or RowidRollback
Select * from emp where empno = 9999;Select * from emp where empno = 9999;
Note: “sequential” meansA sequence like a rowid
Index on emp.empno
#.7
Copyright 2006 Kyle Hailey
db file scattered read
ShadowShadowProcessProcess 1) Search Buffer Cache for the
blocks
2) Fails3) Reads off disk
• File• Block• Multi Block Read Count
Buffer Cache
Multi Block ReadFull Table Scan Index Fast Full Scans
SQL> select * from all_objects;SQL> select * from all_objects;
Note: Scattered Means
Blocks are read and scattered throughout buffer cache
#.8
Copyright 2006 Kyle Hailey
db file parallel read
Shadow
Process
1) Search Buffer Cache for the blocks
2) Fails
3) Reads that block off Disk
Buffer CacheBuffer Cache
Process issues multiple single block reads in parallel Documentation says only for recovery But seems to happen for normal read ops Async Call – wait for all reads to complete
Examples Not contiguous multi block read Index full or range scan Where values in Select * from emp where
Empno in (1,2,3,4,5,6);
Select * from emp where
Empno in (1,2,3,4,5,6);
#.9
Copyright 2006 Kyle Hailey
Read by other Session
1. Block not found in cache2. Read block from disk3. Found other session already
reading block from disk4. Wait for the other session to finish
IO
S1S1
Buffer CacheBuffer Cache
S2S2
Multiple sessions reading the same data that requires IO
Ex) to users doing a full table scan at the same time
#.10
Copyright 2006 Kyle Hailey
Db file Sequential Read: Statspack
Top 5 Timed Events
~~~~~~~~~~~~~~~~~~ % TotalEvent Waits Time (s) Ela Time------------------------ ------------ ----------- --------db file scattered read 7,549 5 52.26CPU time 2 23.29db file sequential read 884 2 19.11control file sequential read 154 0 2.68control file parallel write 255 0 1.87
Top 5 Timed Events
~~~~~~~~~~~~~~~~~~ % TotalEvent Waits Time (s) Ela Time------------------------ ------------ ----------- --------db file scattered read 7,549 5 52.26CPU time 2 23.29db file sequential read 884 2 19.11control file sequential read 154 0 2.68control file parallel write 255 0 1.87
What SQL ?
What Object ?
#.11
Copyright 2006 Kyle Hailey
Db file sequential read
#.12
Copyright 2006 Kyle Hailey
db file scattered Read
#.13
Copyright 2006 Kyle Hailey
Read by other session
#.14
Copyright 2006 Kyle Hailey
IO – Further Investigationselect parameter1, parameter2, parameter3 from v$event_nameselect parameter1, parameter2, parameter3 from v$event_name
NAME P1 P2 P3NAME P1 P2 P3
----------------------- ----------- --------- ------------------------------- ----------- --------- --------
read by other session file# block# class#read by other session file# block# class#db file sequential read file# block# blocksdb file sequential read file# block# blocksdb file scattered read file# block# blocksdb file scattered read file# block# blocks
direct path read file number first dba block cntdirect path read file number first dba block cnt
direct path read temp file number first dba block cntdirect path read temp file number first dba block cnt
direct path write file number first dba block cntdirect path write file number first dba block cnt
direct path write temp file number first dba block cntdirect path write temp file number first dba block cntdb file parallel read files blocks requestsdb file parallel read files blocks requests
P1 and P2 are the same, file# , block# P1 and P2 are the same, file# , block#
We can use P1, P2, P3 from ASH to get more info We can use P1, P2, P3 from ASH to get more info
Exception:Exception:db file parallel read – p1,p2,p3 not usefuldb file parallel read – p1,p2,p3 not useful
#.15
Copyright 2006 Kyle Hailey
ASH – sequential readsselectselect event,event, ash.p3,ash.p3, o.object_name objn,o.object_name objn, o.object_type otype,o.object_type otype, CURRENT_FILE# filen,CURRENT_FILE# filen, CURRENT_BLOCK# blockn,CURRENT_BLOCK# blockn, ash.SQL_IDash.SQL_IDfrom v$active_session_history ash,from v$active_session_history ash, all_objects oall_objects owhere event like 'db file sequential read'where event like 'db file sequential read' and o.object_id (+)= ash.CURRENT_OBJ#and o.object_id (+)= ash.CURRENT_OBJ#order by sample_time;order by sample_time;
EVENT P3 OBJN OTYPE FILEN BLOCKN SQL_IDEVENT P3 OBJN OTYPE FILEN BLOCKN SQL_ID
----------------------- -- ------------------------- ----- ----- ------ ------------------------------------ -- ------------------------- ----- ----- ------ -------------
db file sequential read 1 49890 MGMT_METRICS_1HOUR_ INDEX 3 41737db file sequential read 1 49890 MGMT_METRICS_1HOUR_ INDEX 3 41737db file sequential read 1 50908 MGMT_DB_SGA_ECM TABLE 3 28489 35at43xqj7bs0db file sequential read 1 50908 MGMT_DB_SGA_ECM TABLE 3 28489 35at43xqj7bs0db file sequential read 1 55303 TOTO TABLE 1 60218 7xftj55rvjw9sdb file sequential read 1 55303 TOTO TABLE 1 60218 7xftj55rvjw9s
#.16
Copyright 2006 Kyle Hailey
ASH – scattered readsselectselect event,event, ash.p3,ash.p3, o.object_name objn,o.object_name objn, o.object_type otype,o.object_type otype, CURRENT_FILE# filen,CURRENT_FILE# filen, CURRENT_BLOCK# blockn,CURRENT_BLOCK# blockn, ash.SQL_IDash.SQL_IDfrom v$active_session_history ash,from v$active_session_history ash, all_objects oall_objects owhere event like 'db file scattered read'where event like 'db file scattered read' and o.object_id (+)= ash.CURRENT_OBJ#and o.object_id (+)= ash.CURRENT_OBJ#order by sample_time;order by sample_time;
EVENT P3 OBJN OTYPE FILEN BLOCKN SQL_IDEVENT P3 OBJN OTYPE FILEN BLOCKN SQL_ID
---------------------- -- ------------------------- ----- ----- ------ ----------------------------------- -- ------------------------- ----- ----- ------ -------------
db file scattered read 8 9078 WRH$_SYSMETRIC_HISTO TABLE 3 52363db file scattered read 8 9078 WRH$_SYSMETRIC_HISTO TABLE 3 52363db file scattered read 5 8781 WRI$_ALERT_HISTORY TABLE 3 2676db file scattered read 5 8781 WRI$_ALERT_HISTORY TABLE 3 2676db file scattered read 8 57 OBJAUTH$ TABLE 1 33993 3t2mk1prj24hudb file scattered read 8 57 OBJAUTH$ TABLE 1 33993 3t2mk1prj24hu
#.17
IO by TBS & Fileselect io.cnt,select io.cnt, round(io.cnt/(&v_minutes*60),2) aas, round(io.cnt/(&v_minutes*60),2) aas, io.event, io.event, io.p1 p1, io.p1 p1, f.tablespace_namef.tablespace_namefrom ( selectfrom ( select count(*) cnt,count(*) cnt, substr(event,0,25) event,substr(event,0,25) event, ash.p1 p1ash.p1 p1 from v$active_session_history ashfrom v$active_session_history ash where ( event like 'db file s%' or event like 'direct%' )where ( event like 'db file s%' or event like 'direct%' ) and sample_time > sysdate - &v_minutes/(60*24)and sample_time > sysdate - &v_minutes/(60*24) group by group by event , event , ash.p1ash.p1 ) io,) io, dba_data_files fdba_data_files fwhere where f.file_id = io.p1f.file_id = io.p1Order by io.cnt ;Order by io.cnt ;
CNT AAS EVENT P1 TABLESPACECNT AAS EVENT P1 TABLESPACE
---- --- ----------------------- --- -------------- --- ----------------------- --- ----------
1 .00 db file sequential read 1 SYSTEM1 .00 db file sequential read 1 SYSTEM 2 .00 db file sequential read 3 SYSAUX2 .00 db file sequential read 3 SYSAUX 38 .06 db file sequential read 6 SOE38 .06 db file sequential read 6 SOE 179 .30 db file sequential read 7 SOEINDEX179 .30 db file sequential read 7 SOEINDEX
#.18
IO by Objectselectselect
count(*) cnt,count(*) cnt, CURRENT_OBJ#||' '||o.object_name objn,CURRENT_OBJ#||' '||o.object_name objn, o.object_type otypeo.object_type otypefrom v$active_session_history ash,from v$active_session_history ash, all_objects oall_objects owhere ( event like 'db file s%' or event like 'direct%' )where ( event like 'db file s%' or event like 'direct%' ) and o.object_id (+)= ash.CURRENT_OBJ#and o.object_id (+)= ash.CURRENT_OBJ# and sample_time > sysdate - &1/(60*24)and sample_time > sysdate - &1/(60*24) and session_state='WAITING'and session_state='WAITING'group by group by CURRENT_OBJ#, o.object_name ,CURRENT_OBJ#, o.object_name , o.object_type o.object_type Order by count(*)Order by count(*)
CNT AAS OBJN OTYPECNT AAS OBJN OTYPE
---- ------- ------------------------- ---- ------- ------------------------- ------------------------------
79 .00 52949 ORDER_ITEMS TABLE 79 .00 52949 ORDER_ITEMS TABLE PARTITIONPARTITION 97 .00 -197 .00 -1 130 .00 53117 ORD_STATUS_IX INDEX130 .00 53117 ORD_STATUS_IX INDEX 498 .01 53120 CUST_EMAIL_IX INDEX498 .01 53120 CUST_EMAIL_IX INDEX 512 .01 0512 .01 01632 .03 53112 ITEM_ORDER_IX INDEX1632 .03 53112 ITEM_ORDER_IX INDEX
#.19IO by Object/tbs/waitSelect io.cnt cnt, Select io.cnt cnt,
io.aas aas,io.aas aas,
io.event event,io.event event,
substr(io.obj,1,20) obj, substr(io.obj,1,20) obj,
io.p1 p1,io.p1 p1,
f.tablespace_name tablespace_namef.tablespace_name tablespace_name
from ( selectfrom ( select
count(*) cnt,count(*) cnt,
round(count(*)/(&v_minutes*60),2) aas,round(count(*)/(&v_minutes*60),2) aas,
substr(event,0,15) event, substr(event,0,15) event,
nvl(o.object_name,decode(CURRENT_OBJ#,-1,0,CURRENT_OBJ#)) obj,nvl(o.object_name,decode(CURRENT_OBJ#,-1,0,CURRENT_OBJ#)) obj,
ash.p1, ash.p1,
o.object_type otypeo.object_type otype
from v$active_session_history ash,from v$active_session_history ash,
all_objects oall_objects o
where ( event like 'db file s%' or event like 'direct%' )where ( event like 'db file s%' or event like 'direct%' )
and o.object_id (+)= ash.CURRENT_OBJ#and o.object_id (+)= ash.CURRENT_OBJ#
and sample_time > sysdate - &v_minutes/(60*24)and sample_time > sysdate - &v_minutes/(60*24)
group by group by
substr(event,0,15) , substr(event,0,15) ,
CURRENT_OBJ#, o.object_name ,CURRENT_OBJ#, o.object_name ,
o.object_type ,o.object_type ,
ash.p1 ) io,ash.p1 ) io,
dba_data_files fdba_data_files f
where f.file_id = io.p1where f.file_id = io.p1
Order by io.cnt;Order by io.cnt;
CNT AAS EVENT OBJ P1 TABLESPACE_NAMECNT AAS EVENT OBJ P1 TABLESPACE_NAME
--- --- --------------- ----------------- --- ------------------ --- --------------- ----------------- --- ---------------
1 .02 db file sequent ORDER_ITEMS_PK 7 SOEINDEX1 .02 db file sequent ORDER_ITEMS_PK 7 SOEINDEX 1 .02 db file sequent 0 3 SYSAUX1 .02 db file sequent 0 3 SYSAUX 2 .03 db file scatter WAIT_OBJECTS 1 SYSTEM2 .03 db file scatter WAIT_OBJECTS 1 SYSTEM 2 .03 db file sequent ORDER_ITEMS 6 SOE2 .03 db file sequent ORDER_ITEMS 6 SOE 3 .05 db file sequent CUST_EMAIL_IX 7 SOEINDEX3 .05 db file sequent CUST_EMAIL_IX 7 SOEINDEX 4 .07 db file sequent CUST_LNAME_IX 7 SOEINDEX4 .07 db file sequent CUST_LNAME_IX 7 SOEINDEX 6 .10 db file sequent ORD_ORDER_DATE_IX 7 SOEINDEX6 .10 db file sequent ORD_ORDER_DATE_IX 7 SOEINDEX 13 .22 db file sequent ITEM_ORDER_IX 7 SOEINDEX13 .22 db file sequent ITEM_ORDER_IX 7 SOEINDEX 25 .42 db file sequent 0 2 UNDOTBS125 .42 db file sequent 0 2 UNDOTBS1
#.20IO by SQL selectselect
sum(cnt) over ( partition by io.sql_id order by sql_id ) tcnt,sum(cnt) over ( partition by io.sql_id order by sql_id ) tcnt, io.sql_id,io.sql_id, io.cnt cnt,io.cnt cnt, io.aas aas,io.aas aas, io.objn objn,io.objn objn, io.obj obj,io.obj obj, io.p1 p1,io.p1 p1, f.tablespace_name tablespace_namef.tablespace_name tablespace_namefrom from (( selectselect sql_id,sql_id, count(*) cnt,count(*) cnt, round(count(*)/(&v_minutes*60),2) aas,round(count(*)/(&v_minutes*60),2) aas, CURRENT_OBJ# objn,CURRENT_OBJ# objn, nvl(o.object_name,decode(CURRENT_OBJ#,-1,0,CURRENT_OBJ#)) obj,nvl(o.object_name,decode(CURRENT_OBJ#,-1,0,CURRENT_OBJ#)) obj, o.object_type otype,o.object_type otype, ash.p1ash.p1 from v$active_session_history ashfrom v$active_session_history ash ,all_objects o,all_objects o where ( event like 'db file s%' or event like 'direct%' )where ( event like 'db file s%' or event like 'direct%' ) and o.object_id (+)= ash.CURRENT_OBJ#and o.object_id (+)= ash.CURRENT_OBJ# and sample_time > sysdate - &v_minutes/(60*24)and sample_time > sysdate - &v_minutes/(60*24) group by group by CURRENT_OBJ#, CURRENT_OBJ#, o.object_name ,o.object_name , o.object_type ,o.object_type , ash.p1,ash.p1, sql_idsql_id) io,) io, dba_data_files fdba_data_files fwherewhere f.file_id = io.p1f.file_id = io.p1Order by tcnt, io.sql_id, io.cntOrder by tcnt, io.sql_id, io.cnt
TCNT SQL_ID CNT AAS OBJN OBJ P1 TABLESPAC
---- ------------- --- --- ----- --------------- -- ----------
30 0yas01u2p9ch4 1 .01 53113 ITEM_PRODUCT_IX 7 SOEINDEX 1 .01 53079 ORDER_ITEMS_UK 7 SOEINDEX 28 .16 53112 ITEM_ORDER_IX 7 SOEINDEX 58 6v6gm0fd1rgrz 4 .02 54533 WAIT_OBJECTS 1 SYSTEM 54 .30 0 0 2 UNDOTBS1
#.21
Copyright 2006 Kyle Hailey
Missing Object IDs dba_extents – get object name
Dba_extents is notoriously slow Options
Create a table as select – use table recreate as object extents change
Catch blocks in x$bh when waits occur
select segment_name, segment_typefrom dba_extents where file_id = P1 and P2 between block_id and block_id + blocks – 1;
select segment_name, segment_typefrom dba_extents where file_id = P1 and P2 between block_id and block_id + blocks – 1;
#.22
Copy of dba_extentsEnter value for file: 3Enter value for file: 3Enter value for block: 6619Enter value for block: 6619OWNER SEGMENT_NAME SEGMENT_TYPEOWNER SEGMENT_NAME SEGMENT_TYPE
--------------- -------------------- --------------------------------- -------------------- ------------------
WMSYS LOG_TAB_PK INDEXWMSYS LOG_TAB_PK INDEX
Elapsed: 00:00:41.25Elapsed: 00:00:41.25
create table myextents as select * from dba_extentscreate table myextents as select * from dba_extents
Table created.Table created.
Elapsed: 00:01:25.73Elapsed: 00:01:25.73
CNT OWN SEGMENT_NAME SEGMENT_TYPECNT OWN SEGMENT_NAME SEGMENT_TYPE
--- ---- ---------------- ------------ --- ---- ---------------- ------------
11 SYS SMON_SCN_TO_TIME CLUSTER11 SYS SMON_SCN_TO_TIME CLUSTER 993 SYS _SYSSMU7$ TYPE2 UNDO993 SYS _SYSSMU7$ TYPE2 UNDO150 rows selected.150 rows selected.
Elapsed: 00:00:01.03Elapsed: 00:00:01.03
#.23
Copyright 2006 Kyle Hailey
IO Solutions1. Check average read times per file
Should be between 5-20 ms Data in Statspack under “File IO Stats”
2. Check Cache buffer Hit ratio Check db_cache_advice 9i and higher Data in Statspack under “Buffer Pool Advisory” Want to optimize data caching
3. Tune High IO SQL
#.24
Copyright 2006 Kyle Hailey
Step 1a: Ave Read TimeFile IO Stats DB/Inst:labsf03 Snaps: 1-2
Tablespace Filename------------------------ ----------------------------------------------------
Av Mx Av
Av Rd Rd Av Av Buffer BufWt
Reads Reads/s (ms) Bkt Blks/Rd Writes Writes/s Waits (ms)
---------- ---- ------- ----- --- ------- ------------ -------- ---------- --
SYSTEM /u01/app/oracle/oradata/labsf03/system01.dbf
445 15 0.4 16 1.0 1,157 39 2,744 93.3
USERS /u01/app/oracle/oradata/labsf03/users01.dbf
223 7 0.5 ### 1.0 9,725 324 4 100.0
File IO Stats DB/Inst:labsf03 Snaps: 1-2
Tablespace Filename------------------------ ----------------------------------------------------
Av Mx Av
Av Rd Rd Av Av Buffer BufWt
Reads Reads/s (ms) Bkt Blks/Rd Writes Writes/s Waits (ms)
---------- ---- ------- ----- --- ------- ------------ -------- ---------- --
SYSTEM /u01/app/oracle/oradata/labsf03/system01.dbf
445 15 0.4 16 1.0 1,157 39 2,744 93.3
USERS /u01/app/oracle/oradata/labsf03/users01.dbf
223 7 0.5 ### 1.0 9,725 324 4 100.0
IO should be under 20msIO should be under 20msFastest possible is around 7msFastest possible is around 7msFaster speeds are due to read cachingFaster speeds are due to read caching
#.25
Copyright 2006 Kyle Hailey
Step 1c: Ave Read Time
IO should be under 20msIO should be under 20msFastest possible is around 7msFastest possible is around 7msFaster speeds are due to read cachingFaster speeds are due to read caching
select
to_char(begin_interval_time,'yyyy-mm-dd hh24:mi') snap_time,
file#, readtim/nullif(phyrds,0) avg_read_ms, phyrds
from
DBA_HIST_FILESTATXS f,
dba_hist_snapshot s
where f.snap_id=s.snap_id ;
select
to_char(begin_interval_time,'yyyy-mm-dd hh24:mi') snap_time,
file#, readtim/nullif(phyrds,0) avg_read_ms, phyrds
from
DBA_HIST_FILESTATXS f,
dba_hist_snapshot s
where f.snap_id=s.snap_id ;
SNAP_TIME FILE# AVG_READ_MS PHYRDS
---------------- ---------- ----------- ----
2008-01-05 12:00 9 36.67 392008-01-05 12:00 10 32.31 392008-01-05 13:00 1 11.63 1782242008-01-05 13:00 2 56.37 20142008-01-05 13:00 3 17.38 506682008-01-05 13:00 4 9.39 5653212008-01-05 13:00 5 38.78 412008-01-05 13:00 6 28.29 412008-01-05 13:00 7 27.44 392008-01-05 13:00 8 42.56 392008-01-05 13:00 9 36.67 392008-01-05 13:00 10 32.31 39
SNAP_TIME FILE# AVG_READ_MS PHYRDS
---------------- ---------- ----------- ----
2008-01-05 12:00 9 36.67 392008-01-05 12:00 10 32.31 392008-01-05 13:00 1 11.63 1782242008-01-05 13:00 2 56.37 20142008-01-05 13:00 3 17.38 506682008-01-05 13:00 4 9.39 5653212008-01-05 13:00 5 38.78 412008-01-05 13:00 6 28.29 412008-01-05 13:00 7 27.44 392008-01-05 13:00 8 42.56 392008-01-05 13:00 9 36.67 392008-01-05 13:00 10 32.31 39
#.26
Copyright 2006 Kyle Hailey
Step 1c: Ave Read Time
IO should be under 20msIO should be under 20msFastest possible is around 7msFastest possible is around 7msFaster speeds are due to read cachingFaster speeds are due to read caching
select
to_char(begin_time,'yyyy-mm-dd hh24:mi') begin_time, file_id fid, average_read_time *10 avgrd_ms, average_write_time *10 avgwr_ms, physical_reads pr, physical_writes pwfrom V$FILEMETRIC_HISTORY forder by begin_time;
select
to_char(begin_time,'yyyy-mm-dd hh24:mi') begin_time, file_id fid, average_read_time *10 avgrd_ms, average_write_time *10 avgwr_ms, physical_reads pr, physical_writes pwfrom V$FILEMETRIC_HISTORY forder by begin_time;
BEGIN_TIME FID AVGRD_MS AVGWR_MS PR PW
---------------- ---- -------- -------- ---------- ----------
2008-01-30 12:25 1 5.2 2.5 100 24
2008-01-30 12:25 5 80.0 20.0 2 2
2008-01-30 12:25 2 36.7 2.0 3 218
2008-01-30 12:25 4 22.8 1.8 89 2754
2008-01-30 12:25 3 22.9 2.6 14 47
BEGIN_TIME FID AVGRD_MS AVGWR_MS PR PW
---------------- ---- -------- -------- ---------- ----------
2008-01-30 12:25 1 5.2 2.5 100 24
2008-01-30 12:25 5 80.0 20.0 2 2
2008-01-30 12:25 2 36.7 2.0 3 218
2008-01-30 12:25 4 22.8 1.8 89 2754
2008-01-30 12:25 3 22.9 2.6 14 47
#.27
Copyright 2006 Kyle Hailey
Step 2 : Buffer Pool Advisory
Buffer Pool Advisory Size for Size Buffers for Read EstimatedP Est (M) Factor Estimate Factor Physical Reads--- -------- ------ ------------ ------ --------------D 56 .1 6,986 2.3 58,928D 112 .2 13,972 1.6 42,043D 224 .4 27,944 1.0 25,772D 336 .6 41,916 1.0 25,715D 448 .8 55,888 1.0 25,715D 596 1.0 74,351 1.0 25,715D 728 1.2 90,818 1.0 25,715D 840 1.4 104,790 1.0 25,715D 952 1.6 118,762 1.0 25,715D 1,064 1.8 132,734 1.0 25,715
Buffer Pool Advisory Size for Size Buffers for Read EstimatedP Est (M) Factor Estimate Factor Physical Reads--- -------- ------ ------------ ------ --------------D 56 .1 6,986 2.3 58,928D 112 .2 13,972 1.6 42,043D 224 .4 27,944 1.0 25,772D 336 .6 41,916 1.0 25,715D 448 .8 55,888 1.0 25,715D 596 1.0 74,351 1.0 25,715D 728 1.2 90,818 1.0 25,715D 840 1.4 104,790 1.0 25,715D 952 1.6 118,762 1.0 25,715D 1,064 1.8 132,734 1.0 25,715
#.28
Copyright 2006 Kyle Hailey
Step 3: Find Top I/O SQL
select count(*),sql_id, event from v$active_session_historywhere event in ('db file sequential read', 'db file scattered read', 'db file parallel read')group by sql_id, eventorder by count(*);
select count(*),sql_id, event from v$active_session_historywhere event in ('db file sequential read', 'db file scattered read', 'db file parallel read')group by sql_id, eventorder by count(*);
COUNT(*) SQL_ID EVENT
--------- ------------- ------------------------
10 8hk7xvhua40va db file sequential read 335 3hatpjzrqvfn7 db file sequential read 343 0uuqgjq7k12nf db file sequential read 481 db file scattered read 549 75621g9y3xmvd db file sequential read 1158 db file sequential read 1311 0bzhqhhj9mpaa db file sequential read 1523 41zu158rqf4kf db file sequential read 1757 0yas01u2p9ch4 db file sequential read
COUNT(*) SQL_ID EVENT
--------- ------------- ------------------------
10 8hk7xvhua40va db file sequential read 335 3hatpjzrqvfn7 db file sequential read 343 0uuqgjq7k12nf db file sequential read 481 db file scattered read 549 75621g9y3xmvd db file sequential read 1158 db file sequential read 1311 0bzhqhhj9mpaa db file sequential read 1523 41zu158rqf4kf db file sequential read 1757 0yas01u2p9ch4 db file sequential read
#.29
Copyright 2006 Kyle Hailey
Step 3: OEM find top SQL
#.30
Copyright 2006 Kyle Hailey
IO Solutions
CheckFile IO response timesBuffer Cache Hit Ratio
Always Check SQL to Tune
#.31
Copyright 2006 Kyle Hailey
IO Configuration Issues
Unix Buffer Cache Disk Spindles Raid 5
#.32
Copyright 2006 Kyle Hailey
Machine Memory Machine Memory
Unix File Buffer Cache
SGA SGA Unix File Unix File CacheCache
If average IO read times are underIf average IO read times are under
7ms them probably coming from 7ms them probably coming from
Unix File CacheUnix File Cache
#.33
Copyright 2006 Kyle Hailey
IO Histograms
IO at > 1ms comes IO at > 1ms comes
From Unix File CacheFrom Unix File Cache
#.34
Copyright 2006 Kyle Hailey
IO Throughput IO Operations/Sec 150 IOPS/sec optimistic
Large Disks – Few IOPSLarge Disks – Few IOPS
Small Disks – Many IOPSSmall Disks – Many IOPS
#.35
Copyright 2006 Kyle Hailey
Disk 2Gb vs 70GBSeagate Barracuda 4LP
Seagate Cheetah 73LP
Capacity 2.16GB 73.4GB
Rotation Speed 7200rpm 10000rpm
Rotational Delay(avg) 4.1ms 3ms
Time to read 32Kb 6ms 3ms
Seek Time (avg) 9.4ms 4.9
Total time for Single I/O 19.5ms 10.9ms
I/O per second (conservative)
51 92
IOPs/sec per 100GB 2550 126
#.36
Copyright 2006 Kyle Hailey
Raid 5
http://www.miracleas.com/BAARF/BAARF2.html
B – Battle
A – Against
A – Any
R – Raid
F - Five
#.37
Copyright 2006 Kyle Hailey
Direct I/O WAITSDirect I/O : This mechanism lets the client bypass the buffer cache forI/O intensive operations. The disk blocks are written into and read from process
private memory.
direct path read :Parallel Query
direct path writesqlldr loading/reading LOBsparallel DMLscreate table as select create index
direct path read temp , direct path write tempSorting
#.38
Copyright 2006 Kyle Hailey
Direct IO ShadowShadow
ProcessProcess
Buffer CacheBuffer CachePGA PGA
Sort AreaSort AreaDirect PathDirect PathPQO ReadsPQO Reads
XX
#.39
Copyright 2006 Kyle Hailey
direct path read
Parallel QuerySee v$px_session
_serial_direct_read = true_serial_direct_read = true
select parameter1, parameter2, parameter3 select parameter1, parameter2, parameter3 from v$event_namefrom v$event_namewhere name='direct path read';where name='direct path read';
PARAMETER1 PARAMETER2 PARAMETER3
---------- ---------- ----------
file number first dba block cntShadowShadow
ProcessProcess
Buffer CacheBuffer CachePGAPGAPQO
XX
ShadowShadow
ProcessProcess
Buffer CacheBuffer CachePGAPGAPQO
XX
#.40
Copyright 2006 Kyle Hailey
Further Investigation ASH
SID QSID P3 OBJN OTYPE FN BLOCKN SQL_ID--- ---- -- ----- ------ -- ------ -------------149 152 8 TOTO TABLE 1 194072 4gp8tg0b2s722144 152 8 TOTO TABLE 1 192304 4gp8tg0b2s722 60 152 8 TOTO TABLE 1 190592 4gp8tg0b2s722 54 152 6 TOTO TABLE 1 201274 4gp8tg0b2s722149 152 8 TOTO TABLE 1 198552 4gp8tg0b2s722144 152 8 TOTO TABLE 1 200464 4gp8tg0b2s722 60 152 8 TOTO TABLE 1 196784 4gp8tg0b2s722
SID QSID P3 OBJN OTYPE FN BLOCKN SQL_ID--- ---- -- ----- ------ -- ------ -------------149 152 8 TOTO TABLE 1 194072 4gp8tg0b2s722144 152 8 TOTO TABLE 1 192304 4gp8tg0b2s722 60 152 8 TOTO TABLE 1 190592 4gp8tg0b2s722 54 152 6 TOTO TABLE 1 201274 4gp8tg0b2s722149 152 8 TOTO TABLE 1 198552 4gp8tg0b2s722144 152 8 TOTO TABLE 1 200464 4gp8tg0b2s722 60 152 8 TOTO TABLE 1 196784 4gp8tg0b2s722
select session_id sid, QC_SESSION_ID qsid, ash.p3, CURRENT_OBJ#||' '||o.object_name objn, o.object_type otype, CURRENT_FILE# fn, CURRENT_BLOCK# blockn, ash.SQL_IDfrom v$active_session_history ash, all_objects owhere event like 'direct path read' and o.object_id (+)= ash.CURRENT_OBJ#Order by sample_time;
select session_id sid, QC_SESSION_ID qsid, ash.p3, CURRENT_OBJ#||' '||o.object_name objn, o.object_type otype, CURRENT_FILE# fn, CURRENT_BLOCK# blockn, ash.SQL_IDfrom v$active_session_history ash, all_objects owhere event like 'direct path read' and o.object_id (+)= ash.CURRENT_OBJ#Order by sample_time;
#.41
select
ash.SQL_ID, QC_SESSION_ID qsid, count(*) cnt, count (distinct session_id) deg, nvl(o.object_name,to_char(CURRENT_OBJ#)) obj, o.object_type otype, decode(session_state, 'WAITING',event,'CPU') eventfrom v$active_session_history ash, all_objects owhere o.object_id (+)= ash.CURRENT_OBJ# and qc_session_id is not nullgroup by qc_session_id, sql_id, o.object_name, o.object_type, CURRENT_OBJ#, event, session_stateOrder by qc_session_id, sql_id
PQO - ASH
SQL_ID QSID CNT DEG OBJ OTYPE EVENTSQL_ID QSID CNT DEG OBJ OTYPE EVENT
------------- ---- ---- --- ------------ ----- ------------- ---- ---- --- ------------ ----- ----------------------------------------------------
6rvy3wj2u0h6u 123 42 4 WAIT_OBJECTS TABLE direct path read6rvy3wj2u0h6u 123 42 4 WAIT_OBJECTS TABLE direct path read 123 3 3 WAIT_OBJECTS TABLE CPU123 3 3 WAIT_OBJECTS TABLE CPU 123 96 4 0 db file sequential read123 96 4 0 db file sequential read 123 9 4 0 read by other session123 9 4 0 read by other session 123 2 2 0 CPU123 2 2 0 CPU7p3jt75phub2d 144 386 4 WAIT_OBJECTS TABLE PX Deq Credit: send blkd7p3jt75phub2d 144 386 4 WAIT_OBJECTS TABLE PX Deq Credit: send blkd 144 4 3 WAIT_OBJECTS TABLE PX qref latch144 4 3 WAIT_OBJECTS TABLE PX qref latch 144 37 1 WAIT_OBJECTS TABLE db file sequential read144 37 1 WAIT_OBJECTS TABLE db file sequential read 144 3 2 WAIT_OBJECTS TABLE direct path read144 3 2 WAIT_OBJECTS TABLE direct path read 144 70 1 WAIT_OBJECTS TABLE CPU144 70 1 WAIT_OBJECTS TABLE CPU 144 21 4 0 PX Deq Credit: send blkd144 21 4 0 PX Deq Credit: send blkd 144 12 4 0 db file sequential read144 12 4 0 db file sequential read
#.42
Copyright 2006 Kyle Hailey
direct path write
ShadowShadow
ProcessProcess
Buffer CacheBuffer Cache
PGAPGA
DBWRDBWR
DATADATA
ShadowShadow
ProcessProcess
Buffer CacheBuffer Cache
PGAPGA
DBWRDBWR
DATADATA
Occurs when: insert /*+ APPEND */ sql*loader direct=yCreate table as select
select parameter1, parameter2, parameter3 select parameter1, parameter2, parameter3 from v$event_namefrom v$event_namewhere name='direct path write';where name='direct path write';
PARAMETER1 PARAMETER2 PARAMETER3PARAMETER1 PARAMETER2 PARAMETER3
---------- ---------- -------------------- ---------- ----------
file number first dba block cntfile number first dba block cnt
#.43
Copyright 2006 Kyle Hailey
Direct Path Read TempDirect Path Write Temp
Sorting Write to Temp Read from Temp
ShadowShadow
ProcessProcess
Buffer CacheBuffer Cache
PGAPGA
TEMPTEMP DATADATA
ShadowShadow
ProcessProcess
Buffer CacheBuffer Cache
PGAPGA
TEMPTEMP DATADATA
select parameter1, parameter2, parameter3 select parameter1, parameter2, parameter3 from v$event_namefrom v$event_namewhere name in ('direct path write temp‘,where name in ('direct path write temp‘, ‘ ‘direct path read temp’);direct path read temp’);
PARAMETER1 PARAMETER2 PARAMETER3PARAMETER1 PARAMETER2 PARAMETER3
---------- ---------- -------------------- ---------- ----------
file number first dba block cntfile number first dba block cntfile number first dba block cntfile number first dba block cnt
#.44
Copyright 2006 Kyle Hailey
Further investigation“real time“ can use Select
su.username, su.segtype, s.sql_id, su.blocksfrom v$sort_usage su, v$session sWhere s.saddr = su.session_addr and s.serial# = su.session_num;
Select su.username, su.segtype, s.sql_id, su.blocksfrom v$sort_usage su, v$session sWhere s.saddr = su.session_addr and s.serial# = su.session_num;
USERNAME SEGTYPE SQL_ID BLOCKSUSERNAME SEGTYPE SQL_ID BLOCKS
--------- ------- ------------- --------------- ------- ------------- ------
SYS SORT 3xp88dstadjs4 109440SYS SORT 3xp88dstadjs4 109440
NoteNote: SQL_ID : SQL_ID doesn’t seem filled doesn’t seem filled in for in for v$sort_usage.sql_idv$sort_usage.sql_id
#.46
Copyright 2006 Kyle Hailey
file open Args
P1 fib P2 iov P3 0
Wait on an open file The time it takes to open the file Times should be order of a block read To tune, try pre-opening files Leave sessoins connected
High reconnection rates can exacerbate
Can try holding log files open (see script from Steve Adams)
#.47
Copyright 2006 Kyle Hailey
file identify
The time it takes to identify a file so that it can be opened later.
Datafile should be identified just once Logfiles are identified every checkpoint
Reduce checkpoints Arguments
p1 fib p2 filename p3 opcode (1 test for create, 2 get details)
#.48
Copyright 2006 Kyle Hailey
Data file init write
When autoextend is set and many extensions are happening
#.49
Copyright 2006 Kyle Hailey
Local Write Wait
Truncating a table Wait for Data Cache to be cleared of all
blocks of truncated table Wait by shadow for DBWR
If a problem try GTT Reduce fast_start_mttr_target
#.50
Copyright 2006 Kyle Hailey
Further Investigation
NAME P1 P2 P3
----------------- ----- ------ -----
local write wait file# block#
NAME P1 P2 P3
----------------- ----- ------ -----
local write wait file# block#
select * from v$event_namewhere name = ‘local write wait'
select ash.p1,ash.p2, ash.SQL_ID, count(*)from v$ash ashwhere event='local write wait'group byash.p1,ash.p2, ash.SQL_ID;
P1 P2 SQL_ID COUNT
--- -- ------------- -----
201 2 0q8k9xfmz0k2k 194201 5 0q8k9xfmz0k2k 34201 6 0q8k9xfmz0k2k 9201 4 0q8k9xfmz0k2k 10
#.51
Copyright 2006 Kyle Hailey
Temporary File #’s
SQL> select file# from v$datafile; FILE#
----------
1 2 3 4 5 6 7 8SQL> select file# from v$tempfile; FILE#
----------
2 1
SQL> show parameters db_files
NAME VALUE
---------- -------------
db_files 200
Wait Temporary File#’s =
Db_files + file#
File# 201 = v$tempfile 1
#.52
Copyright 2006 Kyle Hailey
Local Write Wait:extent allocation
CREATE TEMPORARY TABLESPACE "NEWTEMP" TEMPFILE '/d3/temp01.dbf' SIZE 100M AUTOEXTEND ON EXTENT MANAGEMENT LOCAL UNIFORM SIZE 64K;
select a.*, b.*
from dba_objects a, dba_objects b
order by a.owner
ALTER DATABASE DEFAULT TEMPORARY TABLESPACE newtemp;
#.53
Copyright 2006 Kyle Hailey
Local Write Waits
#.54
Copyright 2006 Kyle Hailey
Local Write Wait: Solutions
Truncating a tableReduce cache sizeUse GTT
Temporary Table Space Allocations Increase extent size
#.55
Copyright 2006 Kyle Hailey
Summary I/O
Buffer Cache IOdb file sequential readdb file scattered readdb file parallel read
Tune I/O’s should be 5-15msCheck Buffer Cache AdvisoryTune SQL
#.56
Copyright 2006 Kyle Hailey
Summary Direct I/O
direct path read : PQO direct path write : direct path operations direct path read temp: sorts direct path write temp: sorts direct path write (lob) : stored NOCACHE
#.57
Copyright 2006 Kyle Hailey
IO Summary Other
sort segment request : SMON occupied file open : log files or session reconnects file identify : log files or session reconnects local write wait –
Temp extensions Truncates
data file init write – data file extension
top related