hitchhikers guide to free oracle tuning tools rost - free oracle tuning tools.pdfmoats: the mother...

hitchhikers guide to free Oracle tuning toolsa quick guide

Björn Rost

• Consultant

– Oracle Database HA, Tuning, etc

– Linux&Solaris automation

• Oracle ACE Director

• (ex) president RAC SIG

• organizer OTN EMEA tour

© 2016 Pythian3

ABOUT ME

ABOUT PYTHIAN

Pythian’s 400+ IT

professionals help companies

adopt

and manage disruptive

technologies to better

compete

© 2016 Pythian. Confidential 4


Systems currently

managed by Pythian

EXPERIENCED

Pythian experts

in 35 countries

GLOBAL

Millennia of experience

gathered and shared over

19 years

EXPERTS

11,800 2400


TECHNICAL EXPERTISE

CLOUD

ADVANCED

ANALYTICS

DATABASES

BIG DATA

DEVOPS

Using the disruptive nature of cloud for accelerated, cost-effective growth

INFRASTRUCTURE

MANAGEMENT

Mining data for insights & business transformation using data science

Ensuring databases are reliable, secure, available and continuously optimized

Harnessing the transformative power of data on a massive scale

Providing critical velocity in software deployment by adopting DevOps practices

Transforming and managing the IT infrastructure that supports the business

WHY ORACLE?

7

• basic operations same

between DBs

• instrumentation makes the

difference

• allows to deeply investigate

and explain issues

© 2016 Pythian

(WRONG) TUNING CYCLE

© 2016 Pythian8

guess

change

hope

DON’T BE A STORMTROOPER DBA

© 2016 Pythian9

TUNING CYCLE

© 2016 Pythian10

understand

change

measure

TUNING IS SIMPLE, REALLY

11 © 2016 Pythian

LAST WEEK’S EXAMPLE

• increasing gap in standby db– change max_archive_processes parameter

– change some other parameters

– try rman recovery vs MRP

– increase parallelism

– reduce parallelism

– try more silver bullets…

– have SRE look into IO issues

– have SRE look at CPU issues

12 © 2016 Pythian

guess

change

hope

LAST WEEK’S EXAMPLE

• select event from v$session

where program like '%MRP%’

– PX deq credit: send blkd

– PX deq credit: need buffer

• search for that on MOS– 10.2 Parallel Standby Recovery Process Is Very Slow (Doc ID 420337.1)

• PARALLEL_EXECUTION_MESSAGE_SIZE=16k

13 © 2016 Pythian

understand

change

measure

TYPES

14

report real-time historic drill-down

© 2016 Pythian

ENTERPRISE MANAGER

• incredibly useful and thought through

• from vendor, perfectly integrated

• many common views and methods– coloured AAS view

– Top-SQL view

– Real-Time SQL monitor

– blocking session tree

– AWR Reports

– ADDM and other advisors

16 © 2016 Pythian

ENTERPRISE MANAGER


SO WHY USE CLI TOOLS?

• not licensed for Diag&Tuning

– Standard Edition

– EE without Diag&Tuning (a cardinal sin)

• very low throughput or no GUI access

• EM not setup/installed

– 12c EM express can get the job done

• you simply want to look like a nerd

18 © 2016 Pythian

WHY OPEN SOURCE

• see and learn– you can actually look at the code you are running

• collaborate and contribute– want a new feature? implement yourself!

• free(*) means– no costs (duh)

– get up and running really fast

– don’t worry about audits

19 © 2016 Pythian

RLSQLPLUS

• sqlplus on steroids

• edit sqlplus CLI

• history

– up/down arrow

– search with CTRL-r

• auto-completion

– against custom wordlist

21 © 2016 Pythian

RLSQLPLUS INSTALLATION STEP1

22

[root@kickpuppet ~]# yum install http://dl.fedoraproject.org/pub/epel/6/x86_64/rlwrap-0.42-1.el6.x86_64.rpm

================================================================================

Package Arch Version Repository Size

================================================================================

Installing:

rlwrap x86_64 0.41-1.el6 epel 92 k

Transaction Summary

================================================================================

Install 1 Package(s)

Installing : rlwrap-0.41-1.el6.x86_64 1/1

Verifying : rlwrap-0.41-1.el6.x86_64 1/1

Installed:

rlwrap.x86_64 0:0.41-1.el6

Complete!

© 2016 Pythian

RLSQLPLUS INSTALLATION STEP2

• optional: download or generate wordfile

– for autocompletion

23

$ curl -o wordfile.txt

http://blog.tanelpoder.com/files/scripts/setup/wordfile_11gR2.txt

http://blog.tanelpoder.com/files/scripts/setup/

© 2016 Pythian

RLSQLPLUS INSTALLATION STEP 3

• create aliases around sqlplus

– and rman

– and dgmgrl

• avoid overriding the original name

24

alias rlsqlplus='rlwrap -D2 -irc -b'\''"@(){}[],+=&^%#;|\'\'' -f ~/wordfile.txt

sqlplus'

alias rlrman='rlwrap -D2 -irc -b'\''"@(){}[],+=&^%#;|\'\'' rman'

© 2016 Pythian

SQLCL

• sqlplus “replacement” from SQLdeveloper team

– history and editor

– auto-completion

– auto-formatting

• ansiconsole for col widths

• json, xml, csv etc

– tons of good stuff

25 © 2016 Pythian

AAS VIEW

• my favourite real-time DB dashboard

• one-glance DB activity overview

• colour-coded

– red -> blocking, app, concurrency

– blue -> IO, missing indexes, etc

– green -> CPU, executing too much?

– other -> weird, take a look

27 © 2016 Pythian

AAS VIEW

28 © 2016 Pythian

WEB-ASH

29

• simple and free

• reads v$session by default

• stores data in plain files

• lean javascript to display

http://datavirtualizer.com/web-ash-w-ash/

© 2016 Pythian

MOATS

• top-like output that refreshes

• excellent overview

• needs small installation on database

• samples v$-views while running

• improved forks

– MOATS 2.0

– SQL Dashboard

30 © 2016 Pythian

MOATS 2.0 OUTPUT

32 © 2016 Pythian

SNAPPER

• realtime session monitor and drill-downs

• just one sqlscript to run– no installation needed

• runs for sample period– “snaps” v$session etc

– reports on diffs

• two modes– “ash” for v$session

– stats for v$sesstat

33

http://tech.e2sn.com/oracle-scripts-and-tools/session-snapper

© 2016 Pythian

SNAPPER ARGUMENTS

• snapper ash 5 1 all

– mode can be ash, stat or custom column list

– seconds to run/sample

– number of runs

– “all” or filter on v$session columns

• sid=42

• username=brost

34 © 2016 Pythian

AAS IN SNAPPER

36

SQL> @snapper ash=wait_class 5 1 all

Sampling SID all with interval 5 seconds, taking 1 snapshots...

-- Session Snapper v4.11 - by Tanel Poder ( http://blog.tanelpoder.com )

-- Enjoy the Most Advanced Oracle Troubleshooting Script on the Planet!

-------------------------

Active% | WAIT_CLASS

-------------------------

44% | System I/O

19% | Commit

7% | Scheduler

7% | ON CPU

5% | User I/O


© 2016 Pythian

SNAPPER STATS MODE

37

SQL> @snapper stats,gather=tw 15 1 sid=70

Sampling SID sid=70 with interval 15 seconds, taking 1 snapshots...

-- Session Snapper v4.11 BETA - by Tanel Poder ( http://blog.tanelpoder.com ) - Enjoy the Most Advanced Oracle Troubleshooting Script on the Planet!

:)

---------------------------------------------------------------------------------------------------------------------------------------------------

SID, USERNAME , TYPE, STATISTIC , DELTA, HDELTA/SEC, %TIME, GRAPH , NUM_WAITS, WAITS/SEC, AVERAGES

---------------------------------------------------------------------------------------------------------------------------------------------------

70, SOE , TIME, repeated bind elapsed time , 300, 19.8us, .0%, [ ], , ,

70, SOE , TIME, parse time elapsed , 9272, 611.99us, .1%, [ ], , ,

70, SOE , TIME, PL/SQL execution elapsed time, 230999, 15.25ms, 1.5%, [# ], , ,

70, SOE , TIME, DB CPU , 569000, 37.56ms, 3.8%, [@ ], , ,

70, SOE , TIME, sql execute elapsed time , 1730975, 114.25ms, 11.4%, [## ], , ,

70, SOE , TIME, DB time , 2038543, 134.55ms, 13.5%, [## ], , , .88 % unaccounted

70, SOE , WAIT, log file sync , 235408, 15.54ms, 1.6%, [W ], 183, 12.08, 1.29ms average

70, SOE , WAIT, db file sequential read , 1123224, 74.14ms, 7.4%, [W ], 397, 26.2, 2.83ms average

70, SOE , WAIT, db file parallel read , 52763, 3.48ms, .3%, [ ], 3, .2, 17.59ms average

70, SOE , WAIT, latch: In memory undo latch , 499, 32.94us, .0%, [ ], 1, .07, 499us average

70, SOE , WAIT, resmgr:cpu quantum , 51794, 3.42ms, .3%, [ ], 3, .2, 17.26ms average

70, SOE , WAIT, SQL*Net message to client , 1458, 96.23us, .0%, [ ], 393, 25.94, 3.71us average

70, SOE , WAIT, SQL*Net message from client , 60177, 3.97ms, .4%, [ ], 393, 25.94, 153.12us average

70, SOE , WAIT, PL/SQL lock timer , 12918348, 852.66ms, 85.3%, [WWWWWWWWW ], 1328, 87.65, 9.73ms average

70, SOE , WAIT, events in waitclass Other , 960, 63.36us, .0%, [ ], 2, .13, 480us average

-- End of Stats snap 1, end=2014-11-20 08:04:39, seconds=15.2

© 2016 Pythian

SQLDEVELOPER

39 © 2016 Pythian

SQLDEVELOPER

40 © 2016 Pythian

HISTORIC PERFORMANCE

• moats, snapper etc only sample when running

• EE (+diag/tuning) has ASH and AWR

41 © 2016 Pythian

AWR REPORTS -> STATSPACK

• good for a high-level overview

• AWR has a bit more “stuff” than statspack

• AWR is setup automatically (60min snaps)

– be mindful of averages

• but AWR needs EE and diag&tuning pack

42 © 2016 Pythian

STATSPACK IN 12C

• no problem in non-CDB

• with multitenant would make most sense in CDB

– but not really supported due to common user c#...

• there is a nasty and unsupported workaround

43

https://jonathanlewis.wordpress.com/2013/07/04/12c-statspack-hack/

© 2016 Pythian

ORASASH

• does pretty much what ASH does (but for free)

• sample and store information from v$-views

• mostly compatible with EE ASH

– ashmon

– ashmasters etc

• stores data from multiple sources in repo db

– minimal footprint in source, only view and dblink

44

http://pioro.github.io/orasash/

© 2016 Pythian

SQLT

• oracle supported scripts

• needs a schema an installation in db

– (this is annoying)

• will gather *everything* about a single SQL

– option to use diag&tuning

• output is one zipped html archive

45

Metalink Doc id 215187.1

© 2016 Pythian

SQLD360

• sql scope (just like SQLT)

• needs no installation

• one zipfile including html output

46

http://mauro-pagano.com/2015/02/16/sqld360-sql-diagnostics-collection-made-faster/

© 2016 Pythian

SQLD INCLUDES SQL

49

SELECT MIN(a.snap_id) snap_id,

TO_CHAR(a.begin_interval_time, 'YYYY-MM-DD HH24:MI') begin_time,

TO_CHAR(a.end_interval_time, 'YYYY-MM-DD HH24:MI') end_time,

NVL(TRUNC(SUM(b.elapsed_time_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) elapsed_time,

NVL(TRUNC(SUM(b.cpu_time_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) cpu_time,

NVL(TRUNC(SUM(b.iowait_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) iowait,

NVL(TRUNC(SUM(b.clwait_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) clwait,

NVL(TRUNC(SUM(b.apwait_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) apwait,

NVL(TRUNC(SUM(b.ccwait_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) ccwait

FROM (SELECT snap_id, instance_number, elapsed_time_total, cpu_time_total,

iowait_total, clwait_total, apwait_total, ccwait_total, executions_total

FROM dba_hist_sqlstat

WHERE sql_id = 'gm9dkz4u1hwfv') b,

(SELECT snap_id, instance_number, begin_interval_time, end_interval_time

FROM dba_hist_snapshot

WHERE snap_id BETWEEN 0 AND 166142) a

WHERE a.snap_id = b.snap_id(+)

AND a.instance_number = b.instance_number(+)

AND 'Y' = 'Y'

AND a.instance_number = a.instance_number

GROUP BY

TO_CHAR(a.begin_interval_time, 'YYYY-MM-DD HH24:MI'),

TO_CHAR(a.end_interval_time, 'YYYY-MM-DD HH24:MI')

ORDER BY

TO_CHAR(a.end_interval_time, 'YYYY-MM-DD HH24:MI');

1850 rows selected.

© 2016 Pythian

EDB360

• database scope

• no installation needed!

– just run a sqlfile

• creates one zipfile with html for offline viewing

– also AWR etc

• T | D | N control for access packs

• convenient: graphs

50

http://www.enkitec.com/products/edb360

© 2016 Pythian

TUNAS360

• best of sqld and snapper

• samples v$-views for period

– works without AWR/ASH license!

• produces output like sqld

52 © 2016 Pythian

GOOD OLD 10046 TRACE

• guaranteed to catch everything (unlike sampling)

– why not turn on instance-wide?

• overhead vs usefulness?

• no license needed at all

55 © 2016 Pythian

READING TRACEFILES

• sqldeveloper

• trcanlzr– needs installing but now also part of SQLT

• Kyle Hailey’s script– https://github.com/khailey/oracle_trace_parsing/

• Chris Antognini’s script– https://antognini.ch/downloads/top2/chapter03/tvdxtat

_40beta10_20140630.zip

56 © 2016 Pythian

https://github.com/khailey/oracle_trace_parsing/

REVIEW

• make sqlplus great again– rlsqlplus

– sqlcl

• real-time analysis– snapper

– moats

– web-ash

– sqldeveloper instance viewer

– TUNAs360

57

• historical analysis– Orasash / S-ASH

– statspack

– SQLT

– eDB360

– sqld360

• 10046 tracing– raw

– various analyzers

© 2016 Pythian

hitchhikers guide to free oracle tuning tools rost - free oracle tuning tools.pdfmoats: the mother...

Documents