c o m p u w a r e c h u c k h o o v e r s e r i e s everything you ever wanted to know about db2...

C O M P U W A R EC H U C K H O O V E R S E R I E S

Everything You Ever Wanted toKnow About DB2 Buffering

Table of Contents Table of Contents

DB2 Database Services

Basic Functions of the Buffer Manager

Improvements V3 - V6

The Current Sub-System

Configuring & Tuning


Basic Functions of the Buffer Manager

Improvements V3 - V6

The Current Sub-System

Configuring & Tuning

DB2 Address Spaces DB2 Address Spaces

IBMResource

Lock Manager(IRLM)

DB2System Services

AddressSpace


AddressSpace

Allied AgentAddressSpace

Virtual BufferPool(s)

1.6 GB Max.

R I DSort Area

Sort WorkArea

EDM Pool

LogBuffers

BSDSBuffers

Locking ControlElements

IMS, CICS,TSO, Batch

Applications

Optional“Stuff”

DistributedData Facility

Hiperpool(s)

StoredProcedureAddressSpace(s)

Data Spaces

OS/390

DB2Data-base

DB2BootstrapData Set

BSDS

DB2Catalog

Directory

DB2Log

DB2 Database Services DB2 Database Services

4K PhysicalBlocks

on DASD

Buffer Manager Data ManagerRelational

Data System(RDS)

ApplicationProgram

LogicalRecords(Rows)

Within Indexes& Table Spaces

Results Set Consistingof Columns from

Multiple Rows fromMultiple Tables

Fetch Columnsfrom Rowsof ResultsTable (Set)

Lock Manager, Index Manager

andLog Manager

DB2Data-base

PhysicalBlocks(Pages)

DB2Attach

OS/390/DFP

Relational Data System Relational Data System

RelationalData System

(RDS)

ApplicationProgram

Optimizer forAccess Path

Selection

Join, UnionLogic

Stage TwoPredicates

Column andScalar Functions

ColumnMovement

Sorting

QualifiedRows from

Data Manager

Database Manager Database Manager

Data Manager

LogicalRecords(Rows)

Within Indexesand Table Spaces

Lock Manager,Index Manager

andLog Manager

Data PageManagement

Index PageManagement

Page SpaceManagement

Stage OnePredicates

Work FileManagement

Buffer ManagerInterface

GETPAGE

RELPAGE

SET WRITEPENDING

Access PathNavigation

DFP/VSAM DFP/VSAM

DB2 BufferPool

Device ErrorHandling

Access MethodLogic

OS/390/DFP

DB2Catalog

Directory

DB2WorkingStorage

DB2Data-base

DB2Data-base

DB2Data-base

Buffer Manager Major Functions Buffer Manager Major Functions

Buffer Manager

Data SetOPEN / CLOSE

Buffer/CacheManagement

ReadEngines

HiperspaceManagement

WriteEngines

DFP / DASDInterface

CouplingFacility

Communication

BP49

BP3

BP2

BP1

BP0

..

BP49

BP3

BP2

BP1

BP0

..

BP49

BP3

BP2

BP1

BP0

..

BP49

BP3

BP2

BP1

BP0

..

4K Buffers

8K Buffers

16K Buffers

32K Buffers

GETPAGE, RELPAGE, SET WRITE PENDING

OS/390/DFP

OS/390/ADMF

OS/390/XES

MVPGV3 With

Hiperpools

V4 WithData Sharing

DB2 Database Services Flow DB2 Database Services Flow


Data System(RDS)

ApplicationProgram

Fetch Cursor

DECLARE ... CURSOR FORSELECT (20 COLUMNS) FROM CUSTOMER WHERE SC1 = :HV1 AND SC2 = :HV2 AND SC3 = NSC1

DECLARE ... CURSOR FORSELECT (20 COLUMNS) FROM CUSTOMER WHERE SC1 = :HV1 AND SC2 = :HV2 AND SC3 = NSC1

DB2 Database Services Flow DB2 Database Services Flow


Data System(RDS)

ApplicationProgram

. . . . . . . . . . . . . . . . . . . . .Fetch Cursor

GETPAGE(OBID X PAGE 2)OBID X PAGE 2

RELPAGE (OBID X PAGE 2)GETPAGE (OBID X PAGE 3)

Fetch Cursor

Row 1

Row 2

Row 5

Row 1

Row 2

Row 6

Row 3

Row 4

Row 6

OBID X PAGE 3

Fetch Cursor

Row 1

Row 2

Row 6

Fetch CursorRELPAGE (OBID X PAGE N)

Return Code +100

Row 3

Row 4

Row 6

Process Columns from Page 3 Row 1

Process Columns from Page N Row 6

GETPAGE

Post LRU Chainand Return

Yes

Yes

No

No

Find Least-Recently-UsedBuffer in VBP

Steal L-R-U Bufferfor Requested Page

Write L-R-U BufferContents to Hiperpool

Free Buffer in Hiperpool

Buffer PoolHit

Page inVirtual Buffer Pool?

Move L-R-U BufferContents to HP?

Empty Bufferin VBP?

Page inHiperpool?

Read Page FromDisk - BP Miss

Move Page to Virtual Buffer Pool - Hiperpool Hit

Buffer Pool Organization Buffer Pool Organization

ControlInformationand L-R-U

Anchor Points

Virtual Buffer Pool N

Buffer HeadersHash Table

Look-Aside Hash Search Look-Aside Hash Search

Data Manager

Please GETPAGEOBID X Page Y

HashingAlgorithm

Hash Table

Buffer Headers

P(3) = 0.14038

P(5) = 0.17547

P(<=5) = 0.61596

P(>10) = 0.00284

Repeat Logic forHiperpool (if one exists) Stop if DB2 V2.R3 (VBP Miss)

WhichPool?

“Miss”

OBID XPage G

OBID WPage Q

OBID TPage B

OBID ZPage B

OBID YPage N

L-R-U* Buffer Chaining L-R-U* Buffer Chaining

Hash Table

Control Informationand L-R-U Anchor Points

* Least-Recently-Used

Most Recent SequentialLeast Recent Sequential (STEAL)Most Recent Random (HOT)Least Recent Random (COLD)

For Locating For Caching

OBID X

Page G

OBID W

Page Q

OBID TPage B

OBID ZPage B

OBID YPage N

OBID X

Page B

OBID W

Page E

OBID XPage C

OBID XPage D

OBID XPage E

OBID X

Page I

OBID YPage E

OBID ZPage H

OBID ZPage D

OBID YPage F

OBID ZPage B

OBID YPage C

OBID XPage A

OBID YPage H

OBID WPage G

Page P1

Buffer Aging and Stealing Buffer Aging and Stealing

Page Set X1

on Disk

Random GETPAGE Calls for Page Set X1 Page P1

t0 tn

Page P1

Virtual Buffer Pool

From Time t0 to Time tn Eight GETPAGE CallsTwo synchronous disk readsOverlaid with new page onceNine L-R-U chain modifications

OverlayLeast-

Recently-Used PageWith Targetof CurrentGETPAGE

Call

RandomSynchronous

Disk Read

RandomSynchronous

Disk Read

Buffering Versus Caching Buffering Versus Caching

FillBuffer

Set“A”

Output“Flat”File

Input“Flat”File

Batch or Sequential Process

Buffer Set “A”

Batch Program Waits

Buffer: Something that reduces the shock of impact An area to store data temporarily


FillBuffer

Set“B”


Buffer Set “A”

Buffer Set “C”

Batch Program Reads and ProcessesLogical Records in Buffer Set “A”

and Writes Updated LogicalRecords in Buffer Set “C”

Buffer Set “B”



Input“Flat”File


FillBuffer

Set“A”


Buffer Set “A”

Buffer Set “C”

Batch Program Reads and ProcessesLogical Records in Buffer Set “B”

and Writes Updated LogicalRecords in Buffer Set “D”

Buffer Set “B”

WriteBuffer

Set“C”

Buffer Set “D”



Input“Flat”File

ReadEngine


DB2TableSpace



VirtualBufferPool

WriteEngine

DB2 Index Probe DB2 Index Probe

Buffer Manager Data Manager Relational Data System(RDS)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . SELECT...GETPAGE (Index X Page 2)Index X PAGE 2

ApplicationProgram

Key X1 NLPage N1Key X2 NLPage N2Key X3 NLPage N3Key X4 NLPage N4Key X5 NLPage N5Key X6 NLPage N6

Index X PAGE N5

Key X40 Leaf Pg L40Key X42 Leaf Pg L42Key X44 Leaf Pg L44Key X46 Leaf Pg L46Key X48 Leaf Pg L48Key X50 Leaf Pg L50

Index X PAGE L44

Key X422 RID X422Key X425 RID X425Key X429 RID X429Key X433 RID X433Key X437 RID X437Key X440 RID X440

Table Space X PG Y

TableLook-Up

Equal or Higher

TableLook-Up

Equal or Higher

TableLook-Up

Equal or Higher

GETPAGE (Index X Page N5)

GETPAGE (Index X Page L44)

GETPAGE (Table Space X Page Y)

Table Space X PG Y

Return Code 0

Index Caching Index Caching

Cache: A place for storing provisions or valuablesfor an extended period of time

IndexBufferPool

TableSpaceBufferPool

DB2Indexes

Online or Random DB2 Process

DB2TableSpace

DB2 Buffer Manager SearchesIndex B-Tree in Index Buffer Pool to

Locate RID of Target Rowin Table Space

GETPAGE INDEX X PAGE 2

GETPAGE INDEX X NLPAGE Y

GETPAGE INDEX X LPAGE Z

(3 GETPAGE Calls - 3 “Hits”)

GETPAGE TABLE SPACE Q PAGE R

(1 GETPAGE Call - 1 “Miss”)

Index Cache Synchronous I/O Buffers

Deferred Write Queue Deferred Write Queue Page Set x

ControlInformation


Buffer Pool N HeadersCounter = 7Oldest Modified BufferNewest Modified Buffer

3

1

2 45

67

W

Write Engines Write Engines


6

7

Page SetX

Asyn-chronous

WriteEngine

41532

Address Span <= 150 Pages

DB2Buffer

Manager

Page Set XControl

InformationCounter = 2Oldest Modified BufferNewest Modified Buffer

W

W

W

W

Deferred Write Engine Triggers Deferred Write Engine Triggers

AbbreviatedName

IMTH

DWQT

VDWQT

PermittedRange

NotAlterable0-90%

0-90%

DefaultValue

98%

50%

10%

100

80

60

40

20

0

Full Name

Immediate WriteThreshold

Deferred Write Queue ThresholdVertical Deferred

Write Queue Threshold

Plus - DB2 Checkpoint - QUIESCE Command - Data Set Close - DB2 Shutdown

Sequential Prefetch

Sequential Prefetch

Buffer Pool

DB2PageSet

TriggerPages

Please RELPAGE Zand GETPAGE A

Data Manager

Asynch-ronousRead

Engine

X Y Z

A

Sequential Prefetch - P Quantity

Sequential Prefetch - P Quantity

Asynch-ronousRead

Engine

4K Pools 32K PoolsNo. of

Buffersin Pool

8 to 223224 to 999

> 999

No. ofBuffersin Pool

1 to 1212 to 99

> 99

Applica-tion

Programs

81632

Applica-tion

Programs

124

Utilities

163264

Utilities

248

Prefetch Quantity (P)

DB2PageSet

Sequential Detection

Sequential Detection

Plan B Statement Y Table Space Scan with Prefetch

Plan A Statement X Index Probe

CUB

CUB CUB = Cursor Block One per Open Cursor "5-Out-of-8 Within P/ 2" Trigger Page = (P/ 2) + 1 P

TTTT

T

T

DB2PageSet

DB2PageSet

Prefetch Quantity (P)

Asyn-chronous

ReadEngine

Asyn-chronous

ReadEngine

List Prefetch

List Prefetch

RID Pool

T

T

SortedRIDs

T

P

Plan A Statement X Multiple Index Access

Plan B Statement Y Table Space Scan with Prefetch

DB2PageSet

DB2PageSet

Asyn-chronous

ReadEngine

Asyn-chronous

ReadEngine

Buffer Inventory

Buffer Inventory

Active Buffers(in Virtual Buffer Pool)

Available (Stealable) Buffers

Free Buffers(Empty)

InvalidBuffers

InvalidBuffers

Version 4With DataSharing

UncommittedModified Pages

Free Buffers(Empty)

Free Buffers(Empty)

Being Readfor Application Thread

by Data Manager

Write Pending(Changed Page)

Virtual Buffers

Hiperpool Buffers

Group BufferPool Buffers

WritePending

PageLock*

BeingRead

WritePending

BufferStatus Description of Buffer and Contents

None No No Stealable Contents can be overlaid or reclaimed

No Non-Stealable Contents in use by Application ThreadS or U

X

Yes

Yes Yes

None

X

Non-Stealable Contents in use and modified by Application.Unit-of-recovery not yet committed.

X

No

No

No No

Yes

Yes

Stealable

Non-Stealable

Non-Stealable

* Assuming page-level locking by Application Thread.** Buffer contents can be reclaimed and used by any application prior to being rewritten.

Contents modified by Application. Unit-of-recovery not yet committed.

Contents modified by Application. Unit-of-recovery committed. Contents to be rewritten todisk by asynchronous write engine.**Contents modified by Application. Unit-of-recovery not committed. Contents have been re-written to disk and can be overlaid or reclaimed.

Buffers Buffer Status Buffer Status

More Buffer Categories

More Buffer Categories

* Sequential Prefetch* Sequential Detection* DSNDB07 Work Files --ORDER BY --GROUP BY --UNION --Merge Scan Joins

* Index Probes* Index Look-Aside* Online Transactions Using Small to Large Tables* Joins of Smaller Tables (or Smaller Results Sets)

Free Buffers

SequentialBuffers

Random Buffers

More Buffer Pool Thresholds More Buffer Pool Thresholds 100

80

60

40

20

0

AbbreviatedName

IMTH

DWQT

VDWQT

DMTH

SPTH

VPSEQT

VPPSEQT

PermittedRange

NotAlterable0-90%

0-90%

NotAlterable0-100%

0-100%

0-100%

DefaultValue

98%

50%

10%

95%

90%

80%

40%

Full Name


Deferred WriteQueue ThresholdData Set Deferred

Write Queue ThresholdData Manager

ThresholdSequential Prefetch

ThresholdVirtual Buffer Pool

Sequential ThresholdVBP Parallel

Sequential Threshold

Buffer Pools - Pre-Version 3 Buffer Pools - Pre-Version 3

Expanded Storageas Auxiliary Storage DBM1 Virtual

Address Space

A =DB2

B =OS/390

A

B2

CentralStorage

B1

Disk as Auxiliary Storage

Disk forDB2 Object

Storage

DB2PM Systems Statistics:

“PAGE-INS REQ-READ” / time interval in secondsOS/390 Virtual Storage Manager page-ins from auxiliary storage to

central storage per second to process DB2 data

“PAGE-INS REQ. FOR WRT” / time interval in secondsOS/390 Virtual Storage Manager page-ins from auxiliary storage to

central storage per second to write DB2 updated data backto DB2 disk page set

All page-ins and -outs should be between Central Storage andExpanded Storage (Need help from OS/390 RMF Report*). If diskpaging is used, pool is too big or expanded storage is too small

If total PAGE-INS > 100 per second and expanded storage is available, implementhiperpool (Try HP = 2 x VBP size)

Monitor DBM1 page movement between CS and ES - If “To ES” >> “To CS”consider multiple buffer pools

Monitoring Storage Utilization Monitoring Storage Utilization DB2

Hiperpool

ExpandedStorage

OS/390 ESOHiperspace

DBM1 (Virtual Buffer Pool)

* Paging Report from Program IFASMFDP Reporting on Type 70:79 Records

DB2 Version 3 Hiperpools DB2 Version 3 Hiperpools

OS/390

IBMResource

LockManager(IRLM)

DB2System

ServicesAddressSpace

DB2DatabaseServicesAddressSpace

AlliedAgent

AddressSpace

VirtualBuffer

Pools(s)1.6 GBMax.

2 GBMax.

DB2 Hiperpool =One to Four

OS/390 Hiperspaces8 GB Max.

™ Hiperspace is a trademark of the IBM Corporation

2 GBMax.

2 GBMax.

2 GBMax.

OS/390Hiperspaces™

Data

Virtual Buffer Pool / Hiperpool Virtual Buffer Pool / Hiperpool

DB2Indexes

DB2Table

Spaces

DB2 Data ServicesAddress Space

(DBM1)

OS/390 / ESO Hiperspace(s)

VBPn HPn

Virtual BufferPool n

Hiperpool n

DB2Catalog

Directory

DB2WorkingStorage

DiskI/O

MVPGSynchronousSingle PageMovement

ADMFAsynchronousMultiple Page

Movement

Migration Sub-tasks Migration Sub-tasks

DB2 Data ServicesAddress Space (DBM1)

OS/390 / ESO Hiperspace

VBPn HPn

Virtual Buffer Pool n Hiperpool n

AsynchronousMigrationSub-task

TARGET: (1) Empty Pages (2) L-R-U Pages

SOURCE: L-R-U Unmodified Pages

OVERLAYS

Virtual Buffer Pool / Hiperpool . . . Virtual Buffer Pool / Hiperpool . . .

Virtual Storage Address Spaces

Hiperpool nVirtual Buffer Pool n

Central Storage

Expanded Storageas Auxiliary Storage

Expanded Storageas Hiperspace

Disk asAuxiliaryStorage

Disk forDB2

Objects

A =DB2

B =OS/390

B1

B2A2

A1

A Hot Page - Efficient Caching A Hot Page - Efficient Caching

Hiperpool

Page Set X1

on Disk


t0 tn

Page P1

Virtual Buffer Pool

From Time t0 to Time tn Eight GETPAGE CallsOne synchronous disk readOne ADMF page move to the hiperpoolOne MVPG move to the virtual poolNine L-R-U chain modifications

Page P2

A Cold Page - Inefficient Caching A Cold Page - Inefficient Caching

HiperpoolPage Set X1

on Disk


t0 tn

Page P2

Virtual Buffer Pool

“Dies ofOld Age”


Hiperpool

From Time t0 to Time tn GETPAGE Calls = 2VBP Hits “Reref Ratio” = 0 (0%)Synchronous Reads = 2 (100%)Hiperpool ADMF = 2 (?)

“Sent to OldPeople’sHome”


Minutes, Hours, Days??

Hiperpool Parameters Hiperpool Parameters 100

80

60

40

20

0

Plus CASTOUT = YES or CASTOUT = NO

AbbreviatedName

IMTH

DWQT

VDWQT

DMTH

SPTH

VPSEQT

VPPSEQT

HPSEQT

PermittedRange

NotAlterable0-90%

0-90%

NotAlterable0-100%

0-100%

0-100%

0-100%

DefaultValue

98%

50%

10%

95%

90%

80%

40%

80%

Full Name


Deferred WriteQueue ThresholdData Set Deferred

Write Queue ThresholdData Manager

ThresholdSequential Prefetch

ThresholdVirtual Buffer Pool

Sequential ThresholdVBP Parallel

Sequential ThresholdHiperpool Sequential

Steal Threshold

"ALTER-able" Pools "ALTER-able" Pools

ALTER BUFFERPOOL (bpname) VPSIZE (integer) HPSIZE (integer) VPSEQT (integer) HPSEQT(integer) VPPSEQT (integer) DWQT (integer) VDWQT (integer) CASTOUT (YES | NO)

DISPLAY BUFFERPOOL (BPX) DETAIL (INTERVAL) LSTATS

1

2 ALTER BUFFERPOOL (BPX) VDWQT (2)

DB2 Online Pool Statistics DB2 Online Pool Statistics BUFFERPOOL NAME BPX, BUFFERPOOL ID X, USE COUNT 1

VIRTUAL BUFFERPOOL SIZE ALLOCATED IN-USE / UPDATED

HIPERPOOL SIZE ALLOCATED BACKED BY ES

THRESHOLDS - VP SEQUENTIAL DEFERRED WRITE

RANDOM GETPAGESEQ. GETPAGEDMTH HIT

SEQUENTIAL PREFETCH - REQUESTS PAGES READ

SYS PAGES UPDATEDASYNC WRITE I/O

DW HITNO WRITE ENGINE

= 0

= YES = 0

= 80= 20

= 15114= 23

= 229

= 13658= 1820

= 1821

Only One Page Set Opened?

22.6%

Good!

Huh?

= 2500 BUFFERS= 2500 TO BE DELETED= 564

= 5000 BUFFERS CASTOUT= 5000 TO BE DELETED= 5000

= 80 HP SEQUENTIAL= 50 VERTICAL DEFERRED WRT

= 260473 SYN READ I/O (R)= 65536 SYN READ I/O (S)= 0

= 35 PREFETCH I/OS= 229

= 87708 SYS PAGE WRITTEN= 2411 SYNC WRITE I/O

= 0 VERTICAL DWT HIT= 0

AuxiliaryStorage

CentralStorage

0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

0 20 40 60 80

Total Pool Size in Thousands of Pages

OS

/390

Pag

ing

in P

ag

es

pe

r S

ec

on

d

5,000 Pages in VBP + Excess in Hiperpool

All Pages in Virtual Buffer Pool

Virtual Storage(Virtual Buffer Pool)

PagingDisks

ExpandedStorage OS/390 Paging Versus Pool Size OS/390 Paging Versus Pool Size

Performance Versus Pool Size Performance Versus Pool Size

325

330

335

340

345

350

355

360

365

0 20 40 60 80


Tran

sact

ions

per

Sec

ond


Resp

onse

Tim

e in

Sec

onds





Stats101

Source: SHARE 83 Session 6418, "DB2 V3 User Experience With Data Compression and Bufferpool Enhancements", R. Perih, Smith Barney

Percentage of 34,096 Page Pool in Hiperpool

CPUTime

Total CPU Time

DB2 CPU Time

OS/390 PagingCPU Time

Minimum at2:1 HP:VBP

OS/390 Paging vs. DB2 Hiperpool

Management

OS/390 Paging vs. DB2 Hiperpool

Management

OS/390 OS/390

MemberG00n

OS/390

MemberG002

MemberG001

... DB2 SubsystemMembers

A DB2 Data Sharing Group A DB2 Data Sharing Group

Shared Disk & Data Sets Shared Disk But “Dedicated” Data Sets

...

Sysplex Timers

SharedData-base

SharedDFPCata-log

SharedDB2

Catalog-Directory

SharedTarget &Dist.DB2Libraries

LogBSDS

ZPARMSof G001

LogBSDS

ZPARMSof G002

LogBSDS

ZPARMSof Gnnn

DB07for

G001

DB07for

G002

DB07for

Gnnn

ESCON Channels & Switching

Coupling Facilities

. . .

DB2Member

M001

IRLMGXM1

Coupling Facility Functions Coupling Facility Functions

Group Shared Database Locks

Shared CommunicationsArea (SCA)

Buffer Pool Directories

A Coupling Facility

Group Buffer Pool 0

Group Buffer Pool 1

:

DB2Member

M002

IRLMGXM2

:

VBP0HP0

VBP0HP0

VBP1HP1

VBP1HP1

:

DB2 Page Sets on Shared Disk

Group BP N

Coupling FacilitySystem Communication Area

Group Locking Elements

Group BufferPool N Directory

DB2 Group A - Member C

DB2 Group A - Member B

Local CacheVector

DB2 Group A - Member A

Group Buffer Pools, Virtual Buffer Pools

& Hiperpools

Group Buffer Pools, Virtual Buffer Pools

& Hiperpools

Page Registration Page Registration

GBP N Directory

GBP N Buffers


PageSet Y

Member AVBP N

VBP NLocal Cache

Vector

Database X

X.Y.ZQ 0 0

BufferQ

PageZ

ReadingDatabase XPage Set Y

Page Zinto

Buffer Q

Cache Coherency Cache Coherency Member A Member C

UOR 1: S-Lock Page Z GETPAGE Z (1st Read) RELPAGE Z COMMIT Release S-Lock

UOR 2: X-Lock Page Z GETPAGE Z (2nd Read) SET WRITE PENDING RELPAGE Z COMMIT Write Page Z to GBP (3) Release X-Lock

UOR 3: S-Lock Page GETPAGE Z Page Is Invalid Retrieve from GBP N (4) RELPAGE Z COMMIT Release S-Lock

Group Buffer Pool N


Local CacheVector

1


Local CacheVector

2

XI

4

Database X Page Set Y

Page Z

3

I

Hiperpool N

Cross-Member Invalidation Cross-Member Invalidation

Member C

Group Buffer Pool N


GBPn Directory Entries

Virtual Pool N Vector

Hiperpool N Vector

Hiperpool N

Member A


X1.Y1.Z1 Q1 0 S7X2.Y1.Z6 Q2 R1 0X1.Y2.Z3 0 R6 S2X1.Y1.Z2 Q3 R3 S5

System Administrator Decisions: - Number of Pools and Buffers - Number of Directory Entries - Total Structure Size

VVVVVVVVVVVVVVVVV

VVVVVVVVVVVVVVVVV

Virtual Pool N Vector

Hiperpool N Vector

VVVVVV IVVVVVVVVVV

VVVVVVVVVVVVVVVVV

I

CHANGED X1.Y1.Z1

DB2

CREATE TABLESPACEALTER TABLESPACECREATE INDEXALTER INDEX BUFFERPOOL bpname GBPCACHE ALL

Group Buffer Pool Caching Group Buffer Pool Caching

NONE (V6)

CHANGED

Casting Out to Disk Casting Out to Disk

Member M00x

1 - Group Buffer Pool Checkpoint2 - Exceed Group Threshold3 - Exceed Class Threshold4 - End Inter-DB2 R/W Interest5 - “-STOP DATABASE” Command

Group Buffer Pool n

Triggering Events

Virtual Buffer Pool n

Private Bufferfor Member

1

4

3

2

Group Buffer Pool Duplexing Group Buffer Pool Duplexing

DB2 Group A - Member C

DB2 Group A - Member B


I

II

I

II

II

Local CacheVector

Local CacheVector

Coupling Facility XSystem Communication Area

Group Locking Elements

PGBP NDirectory

Coupling Facility Y

SGBP NDirectory

PrimaryGroup BP N

SecondaryGroup BP N

• Removes a single point-of-failure• Overlapped movement of updated pages to both pools• Secondary ready for immediate take over• Updated pages rewritten to disk from primary --secondary pages deleted if rewrite successful• Excellent for VALUABLE READ/WRITE page sets

Page in VirtualBuffer Pool?

Post LRU Chain and Return

Yes

“GBPCACHEALL”?

No

No

Yes

A

A

Page in GroupBuffer Pool?

Yes

No

Yes

Flagged asValid?

Page inHiperpool?

Yes Yes

No

Flagged asValid?

No

No

GETPAGE

Cache Miss - Sync.Disk Read - 15 ms

Group Buffer PoolHit - 0.2 to 0.4 ms

Hiperpool Hit -0.04 ms

Virtual BufferPool Hit - 0 ms

Read Page fromDisk into VBP

Move Page fromGBP into VBP

Move Page fromHP into VBP

Logical AddressSpace of 2 GB Max.Central

StorageOnly Storage

Supporting InstructionExecution andInput/OutputOperations

Virtual Storage

2 GBOS/390

Address Spacesfor

System andApplication Program

Execution

AuxiliaryStorage

Temporary Storagefor Address Space

Pages That OverflowCentral Storage

Operating SystemNucleus, Virtual

Storage Managementand I/O Supervisor

Implementedwith a

Combination ofExpanded Storageand Disk Storage

Processor Storage Processor Storage

BipolarTechnology

CMOSTechnology

High-Speed

ProcessorCache

CentralStorage

2 GBMax.

ExpandedStorage

8 GBMax.

CentralStorage32 GBMax.

Addressableat the

Bit Level

Addressableat the

4K Page Level

Addressableat the

Bit LevelBut Can OnlyAddress up to

2 GB

High-Speed

ProcessorCache

A

OS/390 Data Spaces for DB2 Data Caching OS/390 Data Spaces for DB2 Data Caching

New way to configure virtual buffer pools Maximum of 128 x 2 gigabyte address spaces

(Eight times too big for current processors) Use subset of 32 gigabyte current maximum Watch for future processor and OS/390

announcements

DBM1AddressSpace

ESOHiperspace

READ

WRITEHiper-

pool

VirtualBufferPool

Move 4K-Pages

DBM1MVCL A,BMVCL C,D

OS/390Data Space

Can Address Data Space When in“Access Register (AR) Mode”

B

C D

VBPLook-AsidePool

Page Size and Pool Options Page Size and Pool Options

BP0 to BP49BP8K0 to BP8K9BP16K0 to BP16K9BP32K to BP32K9

Virtual Pools

HP0 to HP49HP8K0 to HP8K9HP16K0 to HP16K9HP32K to HP32K9

Hiperpools

GBP0 to GBP49GBP8K0 to GBP8K9GBP16K0 to GBP16K9GBP32K to GBP32K9

Group Pools

Index page sets must have 4K page sizeSort work file may be either 4K or 32K page sizeMust REORGanize to change page size (no ALTER command)

Use DROP CREATE or DSNTIAUL to convertEverything still 4K physical blocks on disk

Uses for Larger Pages Uses for Larger Pages

RowSize

2K3K4K5K6K7K8K

4K

211--------

8K

4221111

16K

8543222

32K

161086544

4K

241212--------

8K

2412126666

16K

2415129666

32K

2415129766

Rows per TrackRows per Page

Improved Sequential Processing

Improved Volume Packing

Best Single Row Random Processing

Group Buffer Pool Options Group Buffer Pool Options

XRAYBASEINDEX

AUXIL-IARY

INDEX

GBP XPrimary GBPCACHE ALL

GBPCACHE CHANGED

GBPCACHE NONE

Any page accessedby any membercopies into GBP

Any page modifiedby any membercopied into GBP

Coupling facility usedfor cross-validationonly - Write at commit

VBP X

XRAYAUXILIARY

TABLE

GBP YPrimary

VBP Y

VBP Z

XRAY_BASE

Page P2

A Cold Page -- Efficient “Caching” A Cold Page -- Efficient “Caching”

Page Set X1

on Disk


t0 tn

Page P2

Smaller Virtual Buffer PoolNo Hiperpool

From Time t0 to Time tn Two GETPAGE callsTwo synchronous disk readsNo L-R-U chain modifications (round- robin algorithm)

Buffer Stolenand Overlaid

with Fresh Data

Buffer Stolenand Overlaid

with Fresh Data

DB2 Data Retrieval Hierarchy DB2 Data Retrieval Hierarchy

External(Physical)

I/O

“The Disk

Farm”

SLOW“The Best I/O

is NO I/O”

VirtualBufferPools

Central andAuxiliaryStorage

Internal (Logical) I/O

Hiper-pools

ExpandedStorage

GroupBufferPools

CentralStorage

Disk -- The Original Concept Disk -- The Original Concept

Channel ControlUnit

CentralStorage“Core”

Service Time = 1. Seek Time + 2. Rotational Latency + 3. Data Transfer

No Control Unit Storage and Very Little Logic

1.Seek

2.Rotate

3.Transfer

Physical Volume

Request forRandom Access

Flow of an External I/O Flow of an External I/O

OS/390

DB2Buffer

Manager

DFPVSAM

AccessMethod

IOSInput

OutputSuper-visor

Device UnitControl Block(UCB) BusyQUEUING

Delay

Channel BusyControl Unit Busy

Device BusyPENDING

Delay

DISKCONTROL

UNIT

CHANNEL

Searchand DataTransfer

CONNECTTime

SeekRotational

Position SensingDISCONNECT

Time

QUEUING Delay + PENDING Delay + CONNECT Time + DISCONNECT Time

= DISK RESPONSE TIME FOR EXTERNAL I/O

CACHE

The Second Era The Second Era

Small Cache Control Units

Goal:• Faster Batch Sequential Read and WriteUsing:• Read-Ahead Track Caching• Write-Behind Block CachingBasic

Microcode

< 64 MB of Cache StorageSome (KB) Non-Volatile Storage

ControlUnit

Channel

Control Unit Read-Ahead / Write-Behind Control Unit Read-Ahead / Write-Behind

OS/390AccessMethodI/O Area

CacheStorage

Rest ofTrackOne Physical Block

ControlUnit

ChannelOS/390AccessMethodI/O Area

CacheStorageX Physical Blocks One Block

at a time

With Non-Volatile Storage Disk Fast Write (DFW)

Three Types of Page Sets Three Types of Page Sets

Type 1 - High Access Activity All of the Time

Type 2 - High Access Activity Some of the Time

and Low Access Activity the Rest of the Time

Type 3 - Low Access Activity All of the Time

0400 0800 1200 1600 2000

Type 1

Type 2

Type 3

Incremental Improvements Incremental Improvements

Hardware• More Cache Storage• Multi-track Caching• Cylinder Caching• More Non-Volatile StorageMicrocode• Bypass Cache• Record Cache (RC)• Dynamic Cache Management Extended (DCME)

Goals:• Better Balance of Sequential vs. Random Processing• Adapt to Dynamic Access Patterns

The Current Era The Current Era

Larger Cache Control Units

Goals:• Faster Batch Sequential Read and Write• Channel Speed Retrieval of Cached DataUsing:• Cheap CMOS Storage• Sophisticated Control Unit Management• Data Compression

>= 256 MB of Cache StorageMore (MB) Non-Volatile Storage

AdvancedMicrocode

DB2 4K Page Read/Write Times DB2 4K Page Read/Write Times

QueuingPendingDisconnect (Seek & Rotation)Connect (Search & Transfer)Total

Read Hit RatioWeighted Average

3380E

??

25.3

1.4

26.7

- - -

26.7

3380K

??

24.3

1.4

25.7

- - -

25.7

3380J

??

20.3

1.4

21.7

- - -

21.7

3390-3

??

22.1

1.0

23.1

20%

18.6

3390-2

??

19.6

1.0

20.6

20%

16.7

3390-1

??

16.6

1.0

17.6

20%

14.3

U 2XP

??

11.7

0.4

12.1

50%

6.1

Caching Efficiency Caching Efficiency

A. RandomGETPAGE Calls

193,239

VBP Hits171,141

(A-B-C) 88.5%

VBP Misses22,098(B+C)

B. HiperpoolHits

14,054 (7.3%)

C. SynchronousReads

8,044 (4.2%)

Disk CacheControllerHits ???

Disk Reads???

DB2 System Statistics SMF 42-6 Records

Would8,000 hitsbe goodor bad?

Internal I/O External I/O

SEQCACH Parameter versus Disk Microcode SEQCACH Parameter versus Disk Microcode

DB2 SEQCACH *

BYPASS (Default)Signal disk controllerto avoid caching for DB2 read requests toeverything but sort workfiles

SEQ (Optional)Signal disk controllerto use caching for- sequential prefetch- dynamic prefetch

BYPASS (Default)Signal disk controllerto avoid caching for DB2 read requests toeverything but sort workfiles

SEQ (Optional)Signal disk controllerto use caching for- sequential prefetch- dynamic prefetch

* BY SUBSYSTEM

Disk ControllerMicrocoded Logic

CurrentCache

Hit Ratio**

Low

Medium

High

Range

< 10%

10 to 55%

>55%

Action

Inhibit caching

Cache track

Cache cylinder

** BY PAGE SET

Santa Teresa Recommendations Santa Teresa Recommendations

OS/390 Options:

• Install Third Generation RAMAC Virtual Arrays• Install as Much Disk Control Unit Cache Storage as Possible • Define Maximum Number of Virtual Disk Volumes (UCBs)• Set Target of 50 - 80% Disk Cache Hit Ratio• Ignore Access Arm Movement Analysis

DB2 Options:

• Install as Much Central Storage as Possible for DB2 Virtual Buffer Pools, Hiperpools and Group Buffer Pools• Set Sequential Caching Parameter SEQCACH to SEQ• Use DB2 Compression in Concert with Control Unit Compression• Use SnapShot for Fast Backup

“Until you can measuresomething, and expressit in numbers, you haveonly the beginning ofunderstanding.”

-- Lord Kelvin

DB2 Processing Cost Model DB2 Processing Cost Model

BufferManager

Relational Data System (RDS) SELECT

With Host Variables

Indexes

TableSpaces

- - - - - - -- - - - - - -- - - - - - -

Index Entries"Scanned"

________________________________________________________

Rows"Scanned"

________

________________

________

Columnsfrom

Stage TwoQualified

Rows

DataManager

Rows"Scanned"

Optimized Access Path

Stage OneQualified

Rows

RID and SORTWork Areas

TemporaryWork Files

SELECT & FETCH Basic Cost Factors SELECT & FETCH Basic Cost Factors Description of

Cost Factor

NRDMENSA1*NQ1NCR

RDSData

ManagerBuffer

Manager

* Arguments used in index probes should not be counted

PathLength

9,68055

2,640

2,4207,260

42,240

9275

44022

20,00044075

176440

40,000880

4,84033

660

NOFC NHVNF

NPSINPS

NSA2NCFNQ2NSINRS

NSWPNCA

NPNIOSNIOA

Number of Open/Fetch/Close Cursor Loops......................Number of Host Variables in Predicates............................ Number of Application Fetches (NQ2+1)...........................

Number of Pages Processed (GETPAGE Calls)............ ..........................................................................Number of I/O Operations-Synchronous..................................................................................................... Number of I/O Operations-Asynchronous..................................................................................................

Number of Rows Data Manager Examines ............................................................Number of Search Arguments Evaluated at Stage 1*.............................................Number of Rows Qualified at Stage 1.....................................................................Number of Data Columns Passed to RDS..............................................................

Number of RID Pointer Sorts Initiated...............................Number of RID Pointers Sorted.........................................Number of Search Arguments Evaluated at Stage 2........ Number of Column Functions Evaluated.......................... Number of Rows Qualified at Stage 2.............................. Number of Data Sorts Initiated.........................................Number of Rows Sorted...................................................Number of Sort Work File Pages Processed...................Number of Data Columns Passed to Application............

Class 2 Accounting Trace Factor

NSA1

NRDMENQ1NCR

Table Space Scan Estimation Table Space Scan Estimation

RDS:NOFC = 1NHV = 2NF = 1,001

NSA2 = 1 x NQ1NQ2 = 1,000*NCA = 20 x NQ2

Disk Model = 3390-3Disk Cache = Turned OffBlocks to Read = 571,429 at 35 rows per page Buffers > 1,000Type of I/O = AsynchronousAsynchronous = 1,143 Sec. Read Time or 19 Min.

DATA MANAGER:NSA1 = 2 x NRDMENRDME= 20,000,000NQ1 = 50,000*NCR = 21 x NQ1

HistoryTable with20,000,000

100-byte rows

* Analyst’s Best “Guess”

ApplicationProgram

SELECT (20 COLUMNS) FROM CUSTOMER WHERE SC1 = :HV1 AND SC2 = :HV2 AND SC3 = NSC1

Relational Data System (RDS)

DataManager

BufferManager

Optimized Access Path = Table Space Scan

ColumnsReturned

BUFFER MANAGER:NP = 571,429NIOS = 0NIOA = 17,858

NPNIOSNIOA

17,858Channel

Programs

NSA1

NRDMENQ1NCR

571,429Pages

50,000Rows

20,000Columns

SequentialPrefetch

CPU Cost of Table Space Scan CPU Cost of Table Space Scan Database Services Address Space

System Services Address Space

RDSFactor

DataManagerFactor

BufferManagerFactor

Description

Number of Open/Fetch/Close Cursor LoopsNumber of Host Variables in PredicatesNumber of Application Fetches (NQ2+1)

Number of Pages Processed (GETPAGE Calls)Number of Synchronous I/O OperationsNumber of Asynchronous I/O Operations

Number of Rows ExaminedNumber of Search Arguments Evaluated at Stage 1Number of Rows Qualified at Stage 1Number of Data Columns Passed to RDS

Number of Search Arguments Evaluated at Stage 2Number of Rows Qualified at Stage 2Number of Data Columns Fetched by Application

Class 2 Accounting Trace Factor

TimesPerformed

NOFCNHVNF

NSA2NQ2NCA

NRDMENSA1NQ1NCR

NPNIOSNIOA

12

1,001

571,4290

17,858

20,000,0002*20,000,000

50,00021*50,000

1*50,0001,000

20*1,000

1,003

PathLength

9,68055

2,640

2,4207,260

42,240

9275

44022

7544033

660

CPULoad

%Con-tribu-tion

9,680110

2,642,640

1,382,858,1800

754,321,920

1,840,000,0003,000,000,000

22,000,00023,100,000

3,750,000440,000660,000

661,980

7,030,444,510

0.0%0.0%0.0%

19.7%

10.7%

26.2%42.7%0.3%0.3%

0.1%0.0%0.0%

0.0%

100.0%Total Processor Path Length:

Elapsed ("Wall Clock") Time Elapsed ("Wall Clock") Time

Time, in Seconds

9021-711 CPU Time = 119.2 sec.

9672-RX5 CPU Time = 211.1 sec.

1,143 sec. = Asynchronous Disk Read Time

~ 19 Minutes

DB2 I/O Model for Retrieval DB2 I/O Model for Retrieval

IBM3390

Models 1,2 and 3

CentralStorage (CS)

IBM 3990Cached

Controller

OS/390Paging DiskAux. Storage

(AS)

Data Manager

VirtualBufferPools

CacheStorage

OS/390Paging

Aux.Storage

(AS)

IBMRAMAC

VA

Parallel SysplexCoupling Facility

DBM1Address Space Virtual Storage

(2 GB)

GroupBufferPools

Buffer Manager

ExpandedStorage (ES)

CacheStorage

Asynchron.Read

Engines

DB2ESO

Hiper-pools

* 3390 Model 1

InVBP

YYY

NNNN

InCS

YNN

In ES

as AS

YY

ReplacePage in

CS

NY

YN

YN

In DiskCache

YNNN

On Disk

Seq.Pref

Y

ElapsedTime*

040 mic.75 mic.

40 mic.75 mic.

300 mic.300 mic.

980 mic.1.7* mil.8.1* mil.

17.6* mil.

RelativeSpeed

04075

4075

300300

9801,7008,100

17,600

DB2 Retrieval Times DB2 Retrieval Times

InHiper-pool

YY

NNNN

InGBP

YY

NNNN

Seq.

Y

Ran-dom

Y

In ASon

Disk

AVOID!!!

* 3390 Model 1

InVBP

YYY

NNNN

InCS

YNN

In ES

as AS

YY

ReplacePage in

CS

NY

YN

YN

In DiskCache

YNNN

On Disk

Seq.Pref

Y

ElapsedTime*

040 mic.75 mic.

40 mic.75 mic.

300 mic.300 mic.

980 mic.1.7* mil.8.1* mil.

17.6* mil.

RelativeSpeed

04075

4075

300300

9801,7008,100

17,600

DB2 Retrieval Times DB2 Retrieval Times

InHiper-pool

YY

NNNN

InGBP

YY

NNNN

Seq.

Y

Ran-dom

Y

In ASon

Disk

AVOID!!!

Sum of all working sets istoo big for your central storage (CS)

and expanded storage (ES)

DB2 I/O Model for Writing DB2 I/O Model for Writing

ExpandedStorage (ES)Paging Disk

AuxiliaryStorage (AS)

VirtualBufferPools

Buffer Manager

SynchronousWrite

Asynchronous Deferred Write:Best Case: 32 consecutive pages on disk = 2 ms or Writing to NVS = 2 msWorst Case: 1 page in aux. storage = 60 msSanta Teresa Skip-Sequential (Min. Seek) = 4 ms

Synchronous Write:Rare -- Sign of Very Hot Pages = 20 ms

IBM3390

Models 1,2 and 3 IBM 3990

CachedController

CacheStorage

IBMRAMAC

VACache

Storage

Non-VolatileStorage(NVS)

CentralStorage (CS)

AsynchronousWrite Engines

DBM1Address Space Virtual Storage

Sizing - Proactive or Reactive? Sizing - Proactive or Reactive?

Virtual BufferPool(s)

1.6 GB Max.

Hiperpools8 GB Max.

Group BufferPool(s)

? GB Max.

Source: SG24-2244 DB2 for OS/390 Capacity Planning

A. Total Buffer Pool Size (in Megabytes) =

40 +(Processor “M-Value” / 10)

Given an IBM 9672-R15 (5th generation CMOS @ 112 MIPS)

Pool Size = 40 + (2710/10) = 311 megabytes (about 76,000 pages)

B. Total Buffer Pool Size (in Megabytes) =

40 + (Transactions per Second * 10)

Assuming 95 transactions per second

Pool Size = 40 + (95 * 10) = 990 megabytes (about 242,000 pages)NOTE: 10 megabytes per transaction = 2,500 pages!

GETPAGE Rates vs. Buffer Pool Size GETPAGE Rates vs. Buffer Pool Size

Hour of the Day

GE

TP

AG

Es

per

Sec

on

d

20MB

PoolSize

40MB

PoolSize

Source: SHARE 76 Session I468, "A Systems Perspective of Subsystem Performance", Tom Birmingham, IBM WSC

80MB

PoolSize

System Characteristics vs. Buffer Pool Size

System Characteristics vs. Buffer Pool Size

GETPAGEsDuring

Prime ShiftSource: SHARE 76 Session I468, "A Systems Perspective of Subsystem Performance", Tom Birmingham, IBM WSC

No. ofSynchro-

nousReads

CPU perGETPAGE

TransactionResponse

Time

100 100 100 100

112.2

132.1

40.6

28.7

88.3

76.5

91.8

72.1

Key Statistics for Sizing Key Statistics for Sizing

PoolToo

Small

Poolis

Optimalfor Users

PoolToo

Large

DB2PM Version 3 Statistics Report - Long DB2PM Version 3 Statistics Report - Long BP0 GENERAL--------------------------------------------- CURRENT ACTIVE BUFFERSUNAVAIL.BUFFER-VPOOL FULL

NUMBER OF DATA SETS OPENS

MAX NUMBER OF ALLOC.BUFFERSBUFFERS ALLOCATED - VPOOLBUFFERS ALLOCATED - HPOOLHPOOL BUFFERS BACKED

DFHSM MIGRATED DATA SETDFHSM RECALL TIMEOUTS

HPOOL EXPANS. OR CONTRACT.VPOOL EXPANS. OR CONTRACTVPOOL OR HPOOL EXP.FAILURE

GETPAGE OPERATIONS-FAILEDCONCUR.PREF.I/O STREAMS-HWMPREF.I/O STREAMS REDUCTIONPARALLEL QUERY REQUESTSPARALL.QUERY REQ.REDUCTIONPREF.QUANT.REDUCED TO 1/2PREF.QUANT.REDUCED TO 1/4

QUANTITY----------------

710

0

0100012000020000

00

000

0000000

BP0 READ OPERATIONS----------------------------------------------- GETPAGE REQUESTGETPAGE REQUEST-SEQUENTIALGETPAGE REQUEST-RANDOM

SYNCHRONOUS READSSYNCHRON. READS-SEQUENTIALSYNCHRON. READS-RANDOM

GETPAGE PER SYN.READ-RANDOM

SEQUENTIAL PREFETCH REQUESTSEQUENTIAL PREFETCH READSPAGES READ VIA SEQ.PREFETCHS.PREF.PAGES READ/S.PRF.READ

LIST PREFETCH REQUESTSLIST PREFETCH READSPAGES READ VIA LIST PREFTCHL.PRF.PAGES READ/L.PREF.READ

DYNAMIC PREFETCH REQUESTED.DYNAMIC PREFETCH READSPAGES READ VIA DYN.PREFETCHD.PRF.PAGES READ/D.PRF.READ

PREF.DISABLED-NO BUFFERPREF.DISABLED-NO READ ENG

SYNC.HPOOL READASYN.HPOOL READ-WITHOUT ADMHPOOL READ(NO ADM) - FAILEDASYN.HPOOL READ-WITH ADMHPOOL READ(ADM) - FAILED

QUANTITY----------------

61346025941363193239

8570526

8044

24.0

7037551850

159771730.8

1681160

5.5

32077

181523.6

00

140546781

0240138

0

/MINUTE-------------

153.4K148.8K

4831.76

214.2813.15

201.13

1759.661296.46

39.9K

4.200.281.50

8.001.93

45.38

0.000.00

351.41169.55

0.006004.43

0.00

A

B

C

D

E

F

G

Buffer Pool Hit Ratio Buffer Pool Hit Ratio

Batting Average = Number of Safe Hits / Times at Bat

Buffer Pool Hit Ratio = (A - C) / A

No. of Times Page Found in Buffer Pool No. of GETPAGE Calls

Buffer Pool Hit Ratio = (A - C) / A

No. of Times Page Found in Buffer Pool No. of GETPAGE Calls

Random Hit Ratio = (193,239 - 8,044) / 193,239 = 0.958

Sequential Hit Ratio = (5,941,363 - 526) / 5,941,363 = 0.99991

Caching Efficiency Caching Efficiency

VBP0171,141

Hits(A-B-C)

B. HP014,054

HitsB+C

VBP05,693,918

Hits

E+F. HP0 246,919

Hits(D-E-F-G) E+F+G

A. RANDOMGETPAGEs

193,239

VBP0Misses22,098

C. BPMisses8,044

88.56%

7.27% (or 63.60%)

4.16%

D. SEQUENTIALGETPAGEs5,941,363

VBP0Misses247,445

95.84%

4.15% (or 99.78%)

0.01%

G. BPMisses 526

“Just Make the Pool Bigger”

“Just Make the Pool Bigger”

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Hit

Rat

io

Size of Pool

Investment inFirst 1,000 BuffersDelivers 78% Hits

2,000

Second 1,000Buffers Only

Gains 8% More

4,000 6,000 8,000

Third 1,000 OnlyGains 3% More

Obviously can waste “Virtual Storage” Waste of central and expanded storage Places extra burden on OS/390 Virtual Storage Manager May force OS/390 paging to auxiliary disk storage

Random Hit Ratio vs. Coverage

Random Hit Ratio vs. Coverage

DB2 / IMS -TM Transaction Processing BenchmarkAvg. GETPAGE at 10% Coverage =(0.90 * 0.0) + (0.10 * 17.6) = 1.76 ms

% Database Covered by Buffer Pool

Ra

nd

om

Hit

Rat

io

Source: ZZ05-0462 IBM WSC Tech. Bulletin: MVS/ESA and Data in Memory Performance Studies

90% Buffer Pool Hitsat 10% Coverage

90% Buffer Pool Hitsat 10% Coverage

0%

20%

40%

60%

80%

100%

0% 10% 20% 30% 40% 50% 60%

Average GETPAGE Time vs. Coverage Average GETPAGE Time vs. Coverage

GETPAGE Time at 10% Coverage (90% Hits) RAMAC VA = 1.58 ms 3390-1 = 1.76 ms +11.4% 3390-2 = 2.06 ms +30.4% 3390-3 = 2.31 ms +46.2%


GE

TP

AG

E T

ime

0 ms0% 5% 10% 15% 20%

Random Hit Ratio

RAMAC VA3390-1

3390-23390-3

5 ms

10 ms

15 ms

20 ms

25 msRandom Hit Ratio

Random Hit Ratio Versus “Coverage” Random Hit Ratio Versus “Coverage”

Coverage =Size of Buffer Pool

Size of Page Set

Hit

Ra

tio

fo

r R

an

do

m G

ET

PA

GE

s

100% Random Accessto All Pages in Page Set

High Random Accessto Small Working

Set of Pages

Hot Pages vs. Cold Pages Hot Pages vs. Cold Pages


Ran

do

m H

it R

ati

o

0%

20%

40%

60%

80%

100%

0% 10% 20% 30% 40% 50%

Pages With VeryLow Probability of Re-Use

“Transient Pages”

PagesWith

Interme-diateLevel

ofRe-Use

Pages Heavily Re-Used“Resident Pages”

Classic Questions Classic Questions

How to Configure the Pool(s)?

A Single Large Pool versus Multiple, Not-So-Large?

Which Pages Sets in Which Pools?

Any Scientific Method to Help?

DB2 Capabilities DB2 Capabilities

80 Virtual Buffer Pools, Hiperpools and Group Buffer Pools

Dynamic Alter of Sizes and Thresholds

Dynamic Reassignment of Page Sets

Improved Statistical Display

Categories of Buffer Pool Statistics

Data Manager/Buffer Manager Trace

Synchronous Reads During Prime Shift Synchronous Reads During Prime Shift

Virtual Buffer Pool Number

Syn

chro

no

us R

ead

Ra

te

Synchronous Reads During Night Shift Synchronous Reads During Night Shift


Syn

chro

no

us R

ead

Ra

te

Synchronous Read During Midnight Shift Synchronous Read During Midnight Shift


Syn

chro

no

us R

ead

Ra

te

Density of AccessMode of AccessSize of Page Set

Rereferencing Pattern

Read-Only or Read/Write

Key Page Set

Characteristics

DB2

Static or Erratic Behavior

“We need to identify thespecific page setscontaining a highpercentage of pages with thecharacteristic of being mainlyresident.”

“Once the page sets have been identified......”

A Tough Step A Tough Step

Two Basic Tracing Options Two Basic Tracing Options

IFCID 6 - Start of Reading Page from Disk

IFCID 198 - GETPAGE, RELPAGE and SET WRITE PENDING Calls to Buffer Manager

DB2 Loader

and SQL Logic

G-Series (IFCID 198 Trace) G-Series (IFCID 198 Trace)

Filter

RandomGETPAGE

Calls

• DBID• OBID• No. of Unique Pages Used• GETPAGE Count for Object• Virtual BP Hit Count

~ 3,400 recordsper second for“operational” buffer pool*

~ 9,000 recordsper second ~ 900 records

per sample

10 Traces of One Minute Each* For production page sets (no DB2 catalog, sort work files or decision support objects)

Data ManagerBuffer Manager Interface

G-Series

SummarizedIFCID 198Records

All GETPAGE,

RELPAGE and Set

Write Pending Calls

Time Sequence of IFCID Records Time Sequence of IFCID Records VBP HitGETPAGE Call (Write 198) Flagged as Buffer Pool Hit Possible Set Write Pending Call (Write 198)RELPAGE Call (Write 198)

HP HitGETPAGE Call (Write 198) Flagged as Buffer Pool Miss Possible Set Write Pending Call (Write 198)RELPAGE Call (Write 198)

GBP HitGETPAGE Call (Write 198) Flagged as Buffer Pool Miss Possible Set Write Pending Call (Write 198)RELPAGE Call (Write 198)

NOTE: No IFCID 6’s or 7’s are written--no I/O with hits.

Time Interval T1

Time Interval T2

Interarrival Rate of GETPAGE Calls Interarrival Rate of GETPAGE Calls

HiperpoolPage Set X1

on Disk

Random GETPAGE Calls for Page P2 of Page Set X1

t0 tn

Virtual Buffer Pool



Hiperpool



Seconds, Minutes, Hours?

Page P2Page P2

Reread Interval Reread Interval

Size of

Virtual B

uffer P

ool

plus Hiperp

oolTime Between

Rereads, in minutes

Nu

mb

er

of

Pa

ge

s R

erea

d

R-Series (IFCID 6 Trace) R-Series (IFCID 6 Trace)

~ 100 recordsper second for

“operational” page sets

• DBID• OBID• No. of Pages Read• No. of Pages Reread

10 Traces of 10 Minutes Each

DB2 Loader

and SQL Logic

Start of ReadPage from Disk

Buffer ManagerVSAM DFP Interface

R-Series

SummarizedIFCID 6Records

Time Sequence of IFCID Records Time Sequence of IFCID Records

VBP, HP, GBP All Missed

GETPAGE Call (Write 198) Flagged as Buffer Pool MissWrite 6--Start of Disk I/O Call DFP/VSAM

Posted by DFP/VSAMWrite 7--End of Disk I/O* Possible Set Write Pending Call (Write 198)RELPAGE Call (Write 198)

Time Interval T3

MicrosoftExcel

Downstream Analysis Downstream Analysis

MicrosoftAccess

Download

G-SeriesSummarizedIFCID 198

Records

R-SeriesSummarized

IFCID 6Records

10 G-Series Samples of GETPAGE Calls 10 G-Series Samples of GETPAGE Calls

Total GETPAGEs

0

50,000

100,000

150,000

200,000

250,000

G01 G02 G03 G04 G05 G06 G07 G08 G09 G10

1,542 Unique Page Sets Observed

GETPAGEs by Page Set

Total GETPAGEs

91 WellBehaved

ConsistentPage Sets

FiveInconsistentPage Sets

96 Page Sets Generated 90.1% of GETPAGE Calls

96 Page Sets Generated 90.1% of GETPAGE Calls

Does the 80-20 Rule Apply? Does the 80-20 Rule Apply? C

umul

ativ

e C

ontr

ibut

ion

to T

otal

Wor

kloa

d

20% ofPage Sets

10% ofPage Sets

100 200 300 400

4

3 21

115

Size of Page Set Size of Page Set

PAD = Page Access Density= Random GETPAGEs per

Minute per PagePAD

PAD

Distribution of PAD Values Distribution of PAD Values 1,459

Range: 0.1 to 3,413Mean: 74.29> 200 = 83 Page Sets

Cut-Off Point

No. ofPageSets

PAD Values

IAGPC - Inter-Arrival Time IAGPC - Inter-Arrival Time

1 PAD = Inter- Arrival Time of GETPAGE Requests for the “Average” Page

Given: PAD = 3,413 GETPAGEs per Minute per Page

IAGPC = 1 / PAD = 1 / 3,413

= 0.000293 Minutes Between GETPAGE Calls

for the “Average” Page

:

Rereferencing Pattern Rereferencing Pattern

12 2334 27 26 25 24

“Hot” to “Cold” Access Density “Hot” to “Cold” Access Density

RootNon-LeafLeafData

Clustering Index:One Root Page86 Non-Leaf Pages 37,640 Leaf Pages

Table: 170,700 Data Pages

6,506.4 / min.75.7 / min.

0.173 / min.0.038 / min.

9.2 ms.793.1 ms.5.79 sec.26.2 min.

= 1,000,000 GETPAGEs/Page= 11,628 GETPAGEs/Page= 27 GETPAGEs/Page

= 6 GETPAGEs/Page

PAD IAGPCPage Type

But How About Tables? But How About Tables?

New YorkCity AreaCode 212

New MexicoArea Code

505

Identifying Objects of Interest Identifying Objects of Interest

GETPAGE Calls

Page Access Density(PAD)

VBP Misses

4529,652

86345

1,148

323,208

5404366

LEGENDNo. of Page Sets

No. of GETPAGE CallsAverage Pages UsedPage Access Density

VBP Misses

177112,960

2,75241

29,447

34883,636

2,254392

29,649

4102,655

215477126

55832,22418,604

45109,902

00000


GETPAGE Calls

Page Access Density(PAD)

VBP Misses

2.92%0.31%1.45%0.61%

0.19%1.96%1.14%0.04%

LEGEND% of Page Sets

% of Distinct Pages% of GETPAGE Calls

% of VBP Misses

11.48%10.00%5.52%

15.67%

2.20%8.19%

43.22%15.77%

0.26%0.78%5.02%0.07%

3.57%67.60%40.70%58.47%

0.00%0.00%0.00%0.00%

UninterestingBPZ1,4011,000- 0 -

173,2255,823

3047,072

Buffer Pool Allocation Buffer Pool Allocation

GETPAGE Calls

Page Access Density

(PAD)VBP Misses

BPY55

5,50035,000

832,22418,604

45109,902

BPX86

2,5005,000

1,039,151 3,095336

30,989

DB2 SortBP1

?2,000- 0 -

????

DB2 Catalog& Directory

BP0?

1,0004,000

????

LEGENDBuffer Pool ID

No. of Page SetsNo. of VBP Pages

No. of Hiperpool PagesNo. of GETPAGE CallsAverage Pages UsedPage Access Density

VBP Misses

R-Series Samples of Pages Read/Reread R-Series Samples of Pages Read/Reread

1

10

100

1,000

10,000

100,000

R01 R02 R03 R04 R05 R06 R07 R08 R09 R10

Total Read

98 WellBehaved

ConsistentPage Sets

Two ErraticInconsistentPage Sets

Top Page Sets for Disk Reads Top Page Sets for Disk Reads

Does the 80-20 Rule Apply? Does the 80-20 Rule Apply?

Con

trib

utio

n to

Tot

al W

orkl

oad

Number of Page Sets

20% ofPage Sets

10% ofPage Sets

134

2

3

1

Distribution of Read Values Distribution of Read Values 1,813

Range: 1 to 113,940Mean: 327No. of

PageSets

No. of Reads

Distribution of Reread Values Distribution of Reread Values

Range: 0 to 17,167Mean: 48

No. ofPageSets

No. of Rereads

Distribution of % Rereads Distribution of % Rereads 591

Range: 0 to 61.12%Mean: 14.68%

No. ofPageSets

% of Rereads

Distribution of Weighted Rereads Distribution of Weighted Rereads

Range: 0 - 2,587Mean: 8.69

No. ofPageSets

Weighted Re-reads

346

147

42

Mean Value(224 Page Sets Above Average)

847


Weighted Rereads

RereadsReads

268172

10.57%8

106,011172

2.86%13

LEGENDNo. of Page Sets

Total ReadsTotal Rereads

Reread PercentageWeighted Rereads

17214,5633,999

27.46%1,296

157525,83977,68514.77%13,276

1927,035

8773.24%

64

000

0.00%0

7214,6493,441

23.49%921


0.53%0.97%0.19%0.08%

LEGEND% of Total Page Sets

% of Total Reads% of Total Rereads

% of Weighted Rereads

0.11%0.11%0.08%0.05%

9.06%2.35%4.39%7.53%

Weighted Rereads

RereadsReads

0.00%0.00%0.00%0.00%

8.27%84.69%85.20%

777.14%

3.79%2.36%3.77%5.35%

1.00%4.35%0.96%0.37%

Buffer Pool Allocation Buffer Pool Allocation LEGENDBuffer Pool ID

No. of Page SetsNo. of Reads

No. of RereadsReread PercentageWeighted Rereads

VBP SizeHiperpool Size

DB2 Catalog& Directory

BP0?????

1,0004,000

Weighted Rereads

RereadsReads

BPX1

401555,05185,12515.34%15,4935,000

30,000BPY1

1,49965,8686,0519.18%1,7172,000- 0 -

DB2 SortBP1

?????

2,000- 0 -

Comparison of Traces Comparison of Traces

R-Series401 Page Sets

90.02% ofWeightedRereads

G-Series141 Page Sets

91.52% ofGETPAGE

Calls74.96% of

VBP Misses

Degree of Overlap Degree of Overlap

No.78

1,443,63919,64573.49

135,19310,943

No.63

427,7362,094

204.275,698

33

No.323

104,9793,16533.17

29,5804,550

%16.355.13

11.50

15.7426.44

Page SetsGETPAGEsPages UsedPADVBP MissesWgtd. Rereads

G NOT R

R NOT GG AND R

%3.19

20.927.61

3.030.19

%3.95

70.6171.38

71.9363.59

G NOT R G AND R R NOT G

Standalone Standalone

R-Series

No.141

1,871,37521,73986.08

140,89110,976

No.401

1,548,61822,81067.89

164,77315,493

%20.3075.7482.88

87.6690.02

Page SetsGETPAGEsPages UsedPADVBP MissesWgtd. Rereads

G-Series

%7.14

91.5378.99

74.9663.78

G-Series R-Series

Ranking of Critical Page Sets Ranking of Critical Page Sets

Object ID

370.006379.004370.037379.023416.263420.024371.065416.102370.004402.012420.014370.029402.002401.008401.014

Weighted Rereads

GETPAGE Calls

Page Access Density (PAD)

VB Pool Misses

1.2.3.4.5.6.7.8.9.

10.11.12.13.14.15.

2,5871,284

805587498456441389315285238212204201185

1.2.3.4.9.

91.24.

--14.23.

173.5.

22.7.6.

29,18715,235

8,7994,6523,111

2711,452N/R 2,0891,545

1304,2951,6144,0094,098

168,29878,608

100,03539,45554,007

8,85836,423

N/R 6,207

15,805684

100,8069,938

52,90751,290

1.7.4.

14.11.39.15.

--53.25.

175.3.

35.12.13.

59.9256.29

142.14103.02

7.1811.69

342.00N/R

40.07130.94

44.29307.62

89.45210.20198.36

281.305.122.159.967.832.

42.--

396.129.379.

45.182.

81.87.

Erratic Page Sets Erratic Page Sets

Standard Deviation

Mean +(2 x Standard

Deviation)

97.72%of

Population

The NormalDistribution

Summary of Buffer Configurations Summary of Buffer Configurations

PoolBP0BP1BP2BPXBPYBPZBPX’BPY’

ContentsDB2 Catalog & DirectorySort Work FilesAll Operational DataHigh GP and PADHigh VBP MissesAll Other Cold Page SetsHigh RereadAll Other Cold Page SetsTotals

Original Set-UpVBPSize

HPSize

1,0002,000

10,000

13,000

4,00000

4,000

IFCID 6 IFCID 198VBPSize

HPSize

VBPSize

HPSize

1,0002,000

5,0002,000

10,000

4,0000

30,0000

34,000

1,0002,000

2,5005,5001,000

12,000

4,0000

5,00030,000

0

39,000

“Industrial Strength” Analysis “Industrial Strength” Analysis

- Michael Barnard CGU Life, Kent, U.K. “Reread Minimization”- IBM Watson Research Messers. Dan, Yu, Chung “DBPMAA Algorithm”- IBM Almaden Research Drs. Levy and Morris and Mr. Messinger “Depth Distributions”- IBM and Responsive Systems “Buffer Pool Tool™”

™Trademark of Responsive Systems

FormattedIFCID 6 Start

Read I/Os

1. Create DB2 table to save trace dataCREATE TABLE IFCID6

(POOL_ID CHAR(8) NOT NULL, DBID SMALLINT NOT NULL, OBID SMALLINT NOT NULL, PAGE CHAR(6) NOT NULL, READTYPE CHAR(1) NOT NULL)

2. Run DB2 trace for IFCID 6 records

3. Run DB2PM report to format the IFCID 6 records

DB2Trace

Michael Barnard’s Approach Michael Barnard’s Approach Goal: Minimize Rereads of Pages from Disk

DB2PM

RAW IFCID 6Start Read I/O

Records

RAW IFCID 6Start Read I/O

Records

4. Load report line items into DB2 table using the DB2 LOAD Utility*LOAD DATA INDDN SYSREC00 REPLACE LOG NO

CONTINUEIF(118:120)=‘ACE’INTO TABLE xxxxx.IFCID6 WHEN (80:83)=‘DBID’(POOL_ID POSITION(108:115) CHAR(8), DBID POSITION(88:91) INTEGER EXTERNAL, OBID POSITION(218:221) INTEGER EXTERNAL, PAGE POSITION(239:244) CHAR(6), READTYPE POSITION(260:260) CHAR(1))

5. Run SQL query against table IFCID6 for re-read report

Load and Summarize Load and Summarize

* Based on DB2PM report format with DB2 Version 4. Version 5’s record layout is different to accommodate LARGE table spaces.

DB2 LOAD DB2 TableIFCID6

FormattedIFCID 6 Start

Read I/Os

CouplingFacility Sizing

Estimator

Buffer PoolPerformance

Statistics

The Buffer Pool Tool The Buffer Pool Tool

DB2Instrumentation

Facility

IFI ExitLogic

GETPAGEby Page Set

Analysis

DB2 Subsystem

DB2Trace

IFCID 6,7 & 198Records

PoolConfiguration

Simulator

BPTAnalysis

Buffer Pool Statistics Buffer Pool Statistics Statistics for Buffer Pool: BP0Buffer size is.......................................…...............4K

Number of VP Buffers is....…...........10,000VP sequential threshold is…...............80%Number of HP Buffers is.......….................0HP sequential threshold is......…..........80%Hiper Space Castout is..............….........YesVertical Write Threshold...............….....10%Horizontal Write Threshold............…....50%

Number of GETPAGEs........................….....2,379,499Number of Sequential Access...1,429,246 60.1% of GETPAGEsNumber of Random Access..........709,940 29.8% of GETPAGEsNumber of RID_List.......................240,313 10.1% of GETPAGEsNumber of Random Misses..........277,488 77.1 Misses per Sec.Number of Misses (others)...........149,291 41.5 Misses per Sec. Number of No_Reads...........................604 0.0% of GETPAGEs

Number of Hits............................1,952,116 82.0% of GETPAGES*System HIT RATIO.............................................34.7%Avg. Page Residency......................................1,250 Seconds

* ApplicationHIT RATIO

Report continues on next page

Buffer Pool Statistics Buffer Pool Statistics

Number of Pages Read...........................1,553,174 431.5 Pages per Second Number of Sync Pages Read.................393,989 25.4% of Pages Read Number of SPref Pages Read.................999,889 64.4% of Pages Read Number of LPref Pages Read...................14,853 1.0% of Pages Read Number of DPref Pages Read.................144,443 9.3% of Pages Read

Number of Read I/Os..................................442,966 123.1 I/Os per Second Number of Sync Read I/Os.....................393,989 88.9% of Read I/Os Number of SPref Read I/Os.......................32,127 7.3% of Read I/Os Number of LPref Read I/Os.......................11,853 2.7% of Read I/Os Number of DPref Read I/Os.........................4,997 1.1% of Read I/Os Delay of Sync Read I/Os.................................14 ms. average (Max. 249) Delay of SPref Read I/Os.................................36 ms. average (Max. 250) Delay of LPref Read I/Os.................................43 ms. average (Max. 249) Delay of DPref Read I/Os.................................44 ms. average (Max. 243)

Continued from previous page


Buffer Pool Statistics Buffer Pool Statistics

Number of Set Write Pending (Modifications)............170,158

Number of Pages Written............................73,624 20.5 Pages per Sec Number of Sync Pages Written ...................100 0.1% of Pages Written Number of ASync Pages Written ............73,524 99.9% of Pages Written

Number of Write I/Os..............…...................45,719 12.7 I/Os per Sec Number of Sync Write I/Os...…......................100 0.2% of Write I/Os Number of ASync Write I/Os......................45,619 .99.8% of Write I/Os Delay of Sync Write I/Os....................................7 ms. average (Max. 66) Delay of ASync Write I/Os..................................4 ms. average(Max. 231) Average Number Pages/Async Write..............1.6 pages


Page Set Statistics Page Set Statistics

Statistics for Table Space.......................................DB2xx.TABLEyy

Number of GETPAGEs.................................690,589 29.0% of Pool’s GETPAGEs Number of Sequential Access..................634,019 91.8% of GETPAGEs << Note Number of Random Access........................56,570 8.2% of GETPAGEs Number of RID_List Access.................................0 0.0% of GETPAGEs

Number of Random Misses.........................27,438 7.6 Misses per Sec Number of Misses (others)..........................19,740 5.5 Misses per Sec Number of No_Reads............................................0 0.0% of GETPAGEs Number of Hits............................................643,411 93.2% of GETPAGEs System HIT RATIO.............................................-1.9% Average Page Residency......................................0 Seconds


ApplicationHIT RATIO


Number of Pages Processed..........................704,258 195.6 Pages per Second Number of Sync Pages Read.........................27,938 4.0% of Pages Number of SPref Pages Read.......................654,078 92.9% of Pages Number of LPref Pages Read..................................0 0.0% of Pages Number of DPref Pages Read.........................22,242 3.2% of Pages

Number of Read I/Os.........................................49,414 13.7 Read I/Os per Sec Number of Sync Read I/Os...........................27,938 56.5% of Read I/Os Number of SPref Read I/Os...........................20,735 42.0% of Read I/Os Number of LPref Read I/Os....................................0 0.0% of Read I/Os Number of DPref Read I/Os................................741 1.5% of Read I/Os Delay of Sync Read I/Os.....................................21 ms. average (Max. 239) Delay of SPref Read I/Os.....................................38 ms. average (Max. 250) Delay of LPref Read I/Os.......................................0 ms. average (Max. 0) Delay of DPref Read I/Os.....................................43 ms. average (Max. 183)




Number of Set Write Pendings (Modifications)..14,443 8.5% of pool SWPs

Number of Pages Written.....................................8,055 2.2 Pages per Sec. Number of Sync Pages Written .... ...........................1 0.0% of Pages Written Number of ASync Pages Written ........................8,054 100.0% of Pages Written

Number of Write I/Os.............................................4,833 1.3 I/Os per Sec. Number of Sync Write I/Os........................................1 0.0% of Write I/Os Number of ASync Write I/Os...............................4,832 100.0% of Write I/Os Delay of Sync Write I/Os...........................................6 ms. average (Max. 6) Delay of ASync Write I/Os.........................................4 ms. average (Max. 219)

Average Pages / Write I/O (Asynch).......................1.67


Simulation at the Buffer Pool Level Simulation at the Buffer Pool Level

Results of Simulation for Buffer Pool..................................BP0 Buffer Pool GETPAGE Total.......................................2,379,499

Pages in GETPAGEs Forecast Applic. Page Read Read I/O System Pool Used BP Hits Hit Ratio Rate Rate Hit Ratio 10,000* 2,362,768 1,971,692 83.5 % 438.5/sec. 120.6/sec. 34.0% 18,000 2,349,204 2,049,084 87.3 % 382.8/sec. 95.0/sec. 42.6% 26,000 2,322,343 2,067,504 89.1 % 358.2/sec. 83.1/sec. 46.7% 34,000 2,294,862 2,070,271 90.3 % 338.2/sec. 75.1/sec. 49.9% 42,000 2,267,584 2,064,701 91.1 % 326.4/sec. 69.2/sec. 51.6%

50,000 2,233,613 2,044,257 91.6 % 322.3/sec. 66.3/sec. 52.5% 58,000 2,171,257 1,995,045 91.9 % 322.6/sec. 64.2/sec. 52.6%

66,000 2,160,792 1,991,388 92.2% 315.8/sec. 62.1/sec. 53.6%

* Baseline Verification

ForecastApplicationHit Ratio

ForecastRead

I/O Rate

Simulation at the Page Set Level Simulation at the Page Set Level

Results for Table Space..................................... DB2xx.TABLEyy Page Set GETPAGE Total.................................690,589 ( 29.0% of BP GETPAGEs)Pages in GETPAGEs No. of Applic. Wking Set Size Pages Read System Pool Used BP Hits Hit Ratio Avg. Max. Read I/Os Hit Ratio 10,000 689,218 661,803 95.6 % 3,277 8,607 198.5/sec. 13.8/sec. -2.1% 18,000 688,180 663,431 96.4 % 6,297 15,329 198.8/sec. 13.1/sec. -1.6% 26,000 686,874 664,066 96.7 % 9,537 21,855 201.2/sec. 12.8/sec. -1.2% 34,000 685,796 664,560 96.9 % 13,074 28,390 203.3/sec. 12.5/sec. -0.8% 42,000 685,025 665,323 97.1 % 16,815 34,939 204.8/sec. 12.2/sec. -0.5% 50,000 684,158 665,571 97.3 % 20,804 41,553 208.1/sec. 12.1/sec. -0.2% 58,000 680,175 662,784 97.4 % 25,196 48,298 213.6/sec. 12.0/sec. -0.1% 66,000 672,849 655,905 97.5% 29,344 54,684 211.9/sec. 11.9/sec. 0.0%

Heavy use of sequential prefetch causes high application and low system hit ratio


ForecastRead

I/O Rate

Simulation at the Page Set Level Simulation at the Page Set Level

Results for Table Space......................................DB2xx.TABLEzz Page Set GETPAGE Total................................406,973 ( 17.1% of BP GETPAGEs)Pages in GETPAGE No. of Applic. Wking Set Size Pages Read System Pool Used BP Hits Hit Ratio Avg. Max. Read I/Os Hit Ratio 10,000 405,845 302,693 74.6 % 2,352 6,653 85.3/sec. 31.0/sec. 25.4% 18,000 404,852 349,267 86.3 % 3,512 7,072 51.3/sec. 17.2/sec. 55.4% 26,000 402,225 363,294 90.3 % 4,312 8,501 39.6/sec. 12.5/sec. 66.0% 34,000 395,202 367,227 92.9 % 4,914 8,476 30.0/sec. 9.3/sec. 74.2% 42,000 385,542 364,811 94.6 % 5,341 8,643 24.2/sec. 7.1/sec. 78.9% 50,000 374,082 355,458 95.0 % 5,592 8,712 21.2/sec. 6.5/sec. 81.3%


ForecastRead

I/O Rate

This page set should not be in the same pool as DB2xx.TABLEyy on the preceding page

Simulation - Large Random Object Simulation - Large Random Object

Results for Index Space........... ..............................DB2xx.INDEXxx Page Set GETPAGE Total....................................559,377 ( 17.2% of BP GETPAGEs)Pages in GETPAGE No. of Applic. Wking Set Size Pages Read System Pool Used BP Hits Hit Ratio Avg. Max. Read I/Os Hit Ratio 10,000 558,266 340,542 61.0 % 3,364 6,264 193.5/sec. 20.8/sec. 58.1% 18,000 557,425 345,046 61.8 % 7,249 10,112 194.6/sec. 19.9/sec. 58.5% 26,000 556,357 346,054 62.1 % 9,656 12,314 199.1/sec. 19.5/sec. 59.1% 34,000 555,494 348,850 62.8 % 13,513 15,965 201.6/sec. 18.9/sec. 59.7% 42,000 554,873 349,015 62.9 % 15,223 17,101 201.9/sec. 18.7/sec. 60.1% 50,000 554,159 349,674 63.1 % 16,174 18,096 202.8/sec. 18.4/sec. 60.5%

Very large, random and low rereference causes this profile....


ForecastRead

I/O Rate

Buffer Pool Tool Testimonial Buffer Pool Tool Testimonial

No. of VB Pools 1 7 7 7 7 7No. of Hiperpools 0 0 3 3 3 3Pages in HPs 5k 10k 15k 30k

GETPAGE Calls per secondRead I/Os per second

13.3% Reduction in DB2 Elapsed Time 6.7% Reduction in DB2 CPU TimeOver 2,000,000 I/Os per day eliminated $410,000 savings per years

58%Reduction

Conclusions Conclusions

DB2 buffer management is• Sophisticated but understandable • Designed for high-performance• Keeping up with hardware economics and capabilities • Measurable and controllable

Questions?Contact

Chuck HooverCompuware Corporation(248) 737-7300 x-16249

[email protected] [email protected]

COMPUWARE

Chuck HooverCompuware Corporation

31440 Northwestern HighwayFarmington Hills, Michigan

MI 48334-2564(248) 737-7300 Extension 16249

FAX: (248) 737-7513E-mail: [email protected]

People and software for business applicationssm

www.compuware.com

c o m p u w a r e c h u c k h o o v e r s e r i e s everything you ever wanted to know about db2...

Documents