db2 access path analysis workshop offer

EXPLAIN TABLES COME OUT OF THE CLOSET

Gerald HodgeHLS Technologies

© HLS Technologies

A Presentation to

Tridex DB2 User Group March 2007

www.hlstechnologies.com

Agenda

A brief history of the PLAN_TABLE

V4, V5, V6, V7 Visible Tables

V4, V5, V6, V7 Hidden Tables

V8 Visible Tables

V8 Opaque Tables ?

What does this mean for V8 migrations ?

2


PLAN_TABLE History

PLAN_TABLE – Access Path information and Optimization HINT(S)

DSN_STATEMNT – Cost Estimation and Reason

DSN_FUNCTION – Before V8 all functions, after V8

only User Defined Functions

3


PLAN_TABLE Sizes

V2R3 30 Columns V3R1 34 Columns V4R1 40 Columns V5R1 46 Columns V6R1 49 Columns V7R1 51 Columns V8R1 57 Columns VNext ? Columns

4


PLAN_TABLE

Information Improves from V1 through V8

Purpose is to give an exegesis of the Access Path for each statement

Requires a tool to interpret raw data, or the risk is eisogesis

V6 PLAN_TABLE table will work through V8, but without new information

5


PLAN_TABLE

Should but may not match the actual Access Path in the Directory, e.g., bind without explain may produce a different Access Path in the Directory

At present there is no method to rebuild the PLAN_TABLE from the Directory

Other Explain information may never be rebuilt because it is no longer present

6


STATEMENT & FUNCTION Tables

Introduced with V6 as new tables

DB2 performs an existence check

If the table is there it is updated

EXPLAIN_TIME = BIND_TIME in the PLAN_TABLE and the Catalog

Requires a tool to interpret raw data and match with other tables for a richer explain

7


Hidden Tables

Were they hidden or obscure?

Was the data valid?

They became “known” with V4

They are not “there” with V8

Requires EXPLAIN(ALL) not EXPLAIN(YES)

8


Hidden Tables

Absolutely but selectively used by some Explain Vendors.

Requires EXPLAIN(ALL) vs. EXPLAIN(YES)

Written to after an existence check from V5 onwards

Information placement, column names, etc. is altered from V5 to V6

9


Hidden Tables

V4 and V5 tables do not work forward

V6 and V7 tables do not work backwards

Column Names and format must absolutely match or EXPLAIN fails

Data has moved between tables and format changed

Why?

10


Where Explain Data Comes From

All or part of a Control Block During the Bind With Explain Process

DB2 Control BlockExternalized DataIFCID or Explain Data

11


A Note on the Optimization Process

The Optimizer makes a list of the valid Access Paths and the related costs

It then chooses the “best” Access Path based on the cost estimation

The externalized row in a Hidden Table shows some of the data associated with the selected choice in the list

12


Hidden Tables Information

Each table has a timestamp column that matches Explain Time and Bind Time

Some fields are self explanatory, e.g., OPENCOST, while others are more obscure

Many parts of the tables can be matched to the entries in prefix.SDSNSAMP(DSNWCBDS)

13


Hidden Tables Information

Selected Path examined has an entry

Detailed cost for selected Path is present

The amount of data can be overwhelming

Programming is required to effectively use the information

The row in the Hidden Tables may be for the row before the row in the PLAN_TABLE

14


How to use the tables

Define the tables with an ownerid that only you can use (This prevents problems)

Set the SPRMXPL value to “1” for on and “0” for off in the DSNZPARM

Set the BIND to go to a different package and with EXPLAIN(ALL) not EXPLAIN(YES)

BIND and then view the tables with SPUFI

15


The Hidden Tables V6 & V7

COST_TABLE very detailed information on the Access Path selected

REFERENCE_TABLE filter information

PREDICATE_TABLE Left vs. Right Hand, Type of Predicate

STRUCTURE_TABLE Row Count and Times processed

16


Hidden Tables V6 & V7

PGROUP_TABLE parallel access information

ESTIMATES_TABLE CPU & I/O costs with some supporting data

TABSTATS_TABLE table statistics

COLSTATS_TABLE column statistics

INDXSTATS_TABLE index statistics

17


V8 Visible Tables

PLAN_TABLE new columns, but V6 and V7 Tables will work, just no new information

DSN_STATEMNT_TABLE

DSN_FUNCTION only UDFs

INDEX FOR OPTIMISATION HINTS ON PLAN_TABLE

18


Opaque Tables

These are tables that are defined for Visual Explain in V8 only

A View is defined to present data An XML document presents the DDL X:\products\visual explain\config An existence check determines if they are to be

written to No DSNZPARM Change

19


Opaque Tables

DSN_PREDICAT_TABLE DSN_STRUCT_TABLE DSN_PGROUP_TABLE DSN_PTASK_TABLE DSN_FILTER_TABLE DSN_DETCOST_TABLE DSN_SORT_TABLE DSN_SORTKEY_TABLE DSN_PGRANGE_TABLE DSN_VIEWREF_TABLE

20


Opaque Tables

Examine the XML

Note the comments and use in Visual Explain

Try, test and test again

21


Disclaimer and Excuse

The examples are taken with permission from Terry Purcell’s presentation at the Madrid Technical Conference

They are a quick look into Intelligent Visual EXPLAIN

So what can this do for us…

22


IVE Table Details

23


IVE index Details

24


IVE Index Scan Details

25


Index Predicate Information

26


Fetch Details

27


Sort Details

28

“The idea is to be approximately right rather than exactly wrong”

John W. Tukey

The Technical Tools of StatisticsAmerican Statistician, 19 (1965)

Page 23-28


What does this mean for V8 Conversions

You absolutely want the newest version of Visual Explain and to set up the opaque tables

You will at least have to do a mass rebind in V7

Your Application CPU Costs from the conversion will be between 4% to 20%

The CPU hit depends upon how much “up front” effort you are willing to do

Save a separate copy of your V7 Explain Tables, even if you convert the PLAN_TABLE to V8 format

30


Bind History - Binds older than V7

Old Binds These can go back to V3 or even before in limited cases

Directory Data Issues With each Version of DB2 there have been changes to the Skeleton Cursor data (SQL Access path Information) and DBD (Data base Structure Information)

Each Version of DB2 must support the older structures, which are not converted until a bind or rebind

This is an overhead hit 31


Primary Exposures in V7 Binding or Rebinding

Reserved Words or Syntax issues will cause a bind action to fail

The customer must correct the Reserved Word and / or Syntax issues in the code and then precompile, compile, link bind and test

If the customer does not have the code then they must modify the DBRM, which is non-trivial

If the customer does not have the source code or the DBRM then the DBRM must be recreated from the DB2 Catalog.

32


Primary Exposures in V7 Binding or Rebinding

If CCSID issues exist then these can only be corrected by binds

Check to see if ISO or other Time / Date encoding is specified

Each time the DB2 Optimizer examines an SQL Statement there is an opportunity for choosing a different Access Path

Most Access Path changes are improvements some are not

The customer needs to compare the before and after Access Paths to determine where negative change has occurred

33


V7 Bind / Rebind Issues before V8 Conversion

Too many customers have had to fall back to V7 from V8 because they could not bind in V8, only to find they could not bind in V7

Bind / Rebind corrects DB2 Directory issues and improves DB2 performance

Bind / Rebind gets the best Access Path settings before V8 conversion

34


V8 Compatibility Mode Bind / Rebind Issues

As with V7 there are new Reserved Words

V7 Directory Structures are 31 bit mode and V8 uses 64 bit mode

V8 converts V7 Directory Structures to 64 bit mode with each use of the structure, this adds overhead

Bind / Rebind in V8 is the only method to convert both SQL and DBD structures to 64 bit mode

V8 Compatibility Mode Bind / Rebind Issues

35


Cursor Gets & Loads

36


DB2 V8 New Function Mode Issues

SQL in the DB2 Directory that is not in UNICODE will be translated for execution

Bind Copy or Bind / Rebind of EBCDIC DBRMs will create UNICODE DB2 Directory entries, but SYSSTMT and SYSPACKSTMT remain in EBCDIC

Bind / Rebind will get most V8 NFM performance improvements

Precompile, Compile, Link and Bind are required to get PLIST improvements, e.g., Query Numbers > 32K

37


Bind / Rebind Points in V7 to V8 Conversion

Mass Rebind here does not mean everything at once, but it does mean rebind everything!

V7 mass bind / rebind to ensure binds can be done and that all DB2 Directory Structures are current

V8 Compatibility Mode if you are in CM for more than a few days then mass Bind / Rebind to ensure binds can be done and to remove 64 bit conversion overhead

38


Bind / Rebind Points in V7 to V8 Conversion

V8 NFM bind / rebind to both directory issues and obtain full DB2 Optimizer improvements

V8 NFM precompile, compile, link and bind to obtain PLIST performance improvements

If you presently do not use Hints, learn how and develop a Hints Strategy

39


Fixes: Static SQL

Find the issues. Use the Statistics Advisor in VE

Set old access path from saved Explain Tables as a hint, “fix” SQL, “fix” statistics or structures.

Poor Access Path Choice:

Fix application code and get new DBRM, edit the existing DBRM.

Will not bind:

40


Fixes: Dynamic SQL

Find issues. This is more difficult as the same SQL may work in one situation and not in others.

Monitor for issues.

Determine whyPoor Performance:

Correct the SQL, statistics or structures

Varying performance:

Set a hintVarying performance:

41

“For a successful technology, reality must take precedence over public relations, for Nature cannot be

fooled.”

Richard P. Feynman on the 1986 loss of the Challenger as quoted by the Task Group on the Columbia Disaster


EOM and EOS on V7

http://www-306.ibm.com/software/data/db2/zos/support/plc/

DB2 V7 EOM ?

DB2 V7 EOS June 2008

What does this mean to you ?

43


HLS Website Items

http://www.hlstechnologies.com/downloads.html Version to Version Access Path Comparison is a series of

17 test cases and instructions for implementing them on any of your subsystems. You are then positioned to compare your findings with ours

A presentation discussing how Access Paths are Selected in DB2. This presentation also provides insights into access path selection differences between versions 5, 6, 7 and 8

HLS DESCRIBE (free software with JAVA Source)

44

db2 access path analysis workshop offer

Documents