minn ims haldb-2009_

IMSHigh Availability Large Database

(HALDB)(subset of course CMW46)

The world depends on it

John Harper - IBM IMS Development

[email protected]

Topics

Overview Database StructuresDefinitionPartition SelectionDBRC SupportDatabase CommandsUtilitiesDatabase Creation and MigrationChanging DatabasesSecondary IndexesApplication ConsiderationsMisc. Considerations HALDB Online ReorganizationSummary

HALDB Overview

HALDB (High Availability Large Database)Large Database

Databases are partitionedUp to 1001 partitions per databasePartitions have up to 10 data set groups

High Availability Database

Partition independenceAllocation, authorization, reorganization, and recovery are by partition

Self healing pointersReorganization of partition does not require changes to secondary indexes or logically related databases

HALDB (High Availability Large Database)Benefits

Greater database capacity Without application changes

Increased database availabilityPartitions, not databases, are removed from systemShortened reorganization processBatch window is shortened with concurrent processing

Improved manageabilityData sets may be smaller

Highlights

Hierarchic structure is maintained

A database record resides in one partition

Minimal (or no) application changes required

Cannot initially load logical child segmentsNew status code for load programs

Some secondary index segment formats change May require I/O area changes when secondary index is processed as a database

New database types

PHDAM - partitioned HDAMPHIDAM - partitioned HIDAM

Index is partitionedPSINDEX - partitioned secondary index

Highlights

OSAM and VSAM (ESDS and KSDS) are supported

Partition selection is done by key range or user exit routine

Logical relationships and secondary indexes are supported

Secondary indexes may be partitionedDBRC is required

Databases must be registeredDynamic allocation uses DBRC information

DFSMDA is not used

Definition Process

DBDGEN

Used to define databaseHierarchic structure, data set group boundaries, pointer options, logical relationships, secondary indexes,...

HALDB Partition Definition Utility

ISPF basedUsed to define partitions in database

Partition selection, space characteristics, randomizers,...

Or

DBRC Commands

Indirect Pointers

HALDB uses both direct and indirect pointers for logical relationships and secondary indexes

Direct pointers point to segments

Indirect pointers "point" to Indirect List Entries (ILEs) in Indirect List Data Set (ILDS)

ILEs have token for keyILEs contain direct pointer to segment

ILDS is KSDS associated with a Partition

Extended Pointer Set

Extended Pointer Set (EPS) is used for logical relationships and secondary indexes

EPS is not updated by reorganizations!

EPS contains direct pointer, reorganization number, and ILK

If reorg number is current, direct pointer is usedIf reorg number is not current, ILK is use to find ILE in ILDSILE contains pointer to segment

Direct pointer and reorg number is EPS are updated when ILE is used

Self healing pointers!

HALDB Database Structure

Each partition in a database has a unique partition ID (PID)

A reorganization number is maintained in each partition

Incremented by each reorganization reload

Each segment in PHDAM or PHIDAM database is assigned a unique token when created

Indirect List Entry Key (ILK)8 bytes stored in segment prefix

Self-Healing PointersUsing an Extended Pointer Set (EPS)

Since reorg # in EPS matches reorg # in Partition DBDS, we use EPS RBA pointer

EPSPartition IDReorg # : 1RBAILK...

Segment A

ILEILKSegment CodePartition IDCurrent reorg # : 1Current RBA...

ILDS

ILE

KSDS

Current reorg # :1

Segment B

Partition DBDS

ILK

Indirect List Key (ILK) is stored in segment prefix.

EPS KEY

Self-Healing PointersAfter reorganization of Partition

Reorg moves segment B, updates reorg # in ILE and Partition DBDS, and changes Current RBA in ILE


Segment A


ILE

KSDS

ILDS

Current reorg # :2

Segment B

Partition DBDS

ILK

"broken" pointer


EPS KEY

Self-Healing PointersUsing the EPS after the reorganization

Since reorg # in EPS does not match reorg # in Partition DBDS, we use the ILE RBA pointer


Segment A


ILE

KSDS

ILDS

Segment B

Partition DBDS

Current reorg # :2

ILK

"broken" pointer


EPS KEY

Self-Healing Pointers"Healing" the EPS

When we use the ILE, we update the reorg # and RBA pointer in the EPS


Segment A


ILE

KSDS

ILDS

Segment B

Partition DBDS

ILK

Current reorg # :2

"healed" pointer


EPS KEY

HALDB Database Data Sets

Each HALDB database has up to 1001 partitions

PHIDAM has index, ILDS, and up to 10 data set groups per partition

3 to 12 data sets per partition3 to 12,012 data sets per database

PHDAM has ILDS and up to 10 data set groups per partition

2 to 11 data sets per partition2 to 11,011 data sets per database

PSINDEX has no ILDS or data set groups

1 data set per partition1 to 1001 data sets per secondary index

HALDB Database Data Sets

PHIDAM PHDAM PSINDEX

...

...

...

...

...

...

...

...Index

ILDS

Data ...

1 to 1001 partitions 1 to 1001 partitions

1 to 1001 partitions

1 to 10 data set groups... ... ...

Partition Independence

Commands

Allowed on both databases and partitions

Availability

Partitions are allocated and authorized independently

Scheduling

Based on database availabilityPCBs and INQY calls report database availabilityPartition may be unavailable with available database

Database Utilities

Allowed on individual partitionsConcurrent processing of multiple partitions allowed

HALDB Overview

Database candidates for HALDB

Very large databasesApproaching 4GB limitationTheoretical limit is now over 40 terabytes per database

Medium and large databasesParallel processing to meet time deadlinesLess frequent reorganizations due to more free space

Any databaseMore frequent reorganizationsMaking only parts of the data unavailable for database maintenance

HALDB Database Structures

Database Structures

Logical relationships

Virtual pairing is not allowedLimited to unidirectional or physically paired

Logical child segments cannot be initially loadedMust be added by update

Symbolic pointers are not used

Secondary indexes must have unique keys

/SX or /CK may be used to create uniqueness/SX is increased from 4 to 8 bytes (ILK)

Symbolic pointers are not used

Database Structures

PHIDAM primary indexes are not separately defined

Defined as part of the PHIDAM database

Applies to DBDGEN and system definition

Parent pointers

All segments have physical parent pointers

Database Data SetsPHIDAM PHDAM PSINDEX

The data sets in a partition have generated data set names and DDNAMEs.Letters are used to distinguish them.

X - PHIDAM indexL - ILDSA through J - Data data setsA - PSINDEX

...

...

...

...

...

...

...

...

Index

ILDS

Data ...

1 to 1001 partitions 1 to 1001 partitions

1 to 1001 partitions

1 to 10 data set groups

X

L L

A-J

...

A

A-J

... ...

Database Data Sets

DDNAMEs

Partition name is basis for DDNAMEUp to 7 charactersAssigned in HALDB Partition Definition Utility or DBRC

commandLetter is used as suffix

X for PHIDAM index L for ILDSA for PSINDEXA through J for data

Example:Partition name: FRANCEDDNAME for PHIDAM Index: FRANCEX

Database Data Sets

Data set names

Begin with data set name prefix for the partitionUp to 37 charactersAssigned in HALDB Partition Definition Utility or DBRC

command

Letter and Partition ID are used as suffixX for PHIDAM index L for ILDSA for PSINDEXA through J for data

Example:Partition data set name prefix IMP0.DB.INV23.FRPartition ID: 00004Data set name PHIDAM Index: IMP0.DB.INV23.FR.X00004

Database Data Sets

Data set size limitations

Maximum data set size is 4GBApplies to OSAM and VSAM

OSAM block sizes must be even

Partition DDNAMEs and Data Set Names

Data set DDNAME Data Set Name

Data set group 1 Partition_nameA DSN_prefix.ApartitionID

Data set group 2 Partition_nameB DSN_prefix.BpartitionID

Data set group 3 Partition_nameC DSN_prefix.CpartitionID

... ... ...

Data set group 10 Partition_nameJ DSN_prefix.JpartitionID

ILDS Partition_nameL DSN_prefix.LpartitionID

PHIDAM Index Partition_nameX DSN_prefix.XpartitionID

Secondary Index Partition_nameA DSN_prefix.ApartitionID

Partition_name assigned by user in HALDB Partition Definition Utility or DBRC command

DSN_prefix assigned by user in HALDB Partition Definition Utility or DBRC command

PartitionID assigned by IMS in HALDB Partition Definition Utility or DBRC command


Partition_name of FRANCE

DSN_prefix of IMP0.DB.INV23.FRANCE

PartitionID of 00004

Example: PHIDAM with 10 data set groups, FRANCE partition


Data set group 1 FRANCEA IMP0.DB.INV23.FRANCE.A00004

Data set group 2 FRANCEB IMP0.DB.INV23.FRANCE.B00004

Data set group 3 FRANCEC IMP0.DB.INV23.FRANCE.C00004

... ... ...

Data set group 10 FRANCEJ IMP0.DB.INV23.FRANCE.J00004

ILDS FRANCEL IMP0.DB.INV23.FRANCE.L00004

PHIDAM Index FRANCEX IMP0.DB.INV23.FRANCE.X00004


Partition_name of CANADA

DSN_prefix of IMP0.DB.INV23.CANADA

PartitionID of 00011


Data set group 1 CANADAA IMP0.DB.INV23.CANADA.A00011

Data set group 2 CANADAB IMP0.DB.INV23.CANADA.B00011

Data set group 3 CANADAC IMP0.DB.INV23.CANADA.C00011

... ... ...

Data set group 10 CANADAJ IMP0.DB.INV23.CANADA.J00011

ILDS CANADAL IMP0.DB.INV23.CANADA.L00011

PHIDAM Index CANADAX IMP0.DB.INV23.CANADA.X00011

Example: PHIDAM with 10 data set groups, CANADA partition

Extended Pointer Set

Extended pointer set (EPS)

Used for logical relationships and secondary indexes

Data:

Reorg number of target partitionTarget partition IDPointer to logical parent or sec. index targetPointer to paired logical child for physical pairingDatabase record lock ID of segmentILK

28 bytes

ILDS Data Sets

Indirect List Entries (ILEs)

Created or updated by reorg reload

Reorgs do not update pointers in segments

Not created or updated by non-reload processing

This processing updates pointers in segments

Initial load does not create ILEs

ILDS Data Sets

ILE keys (9 bytes)

ILK (8 bytes)

RBA of segment at its creation time (4 bytes)

Partition id at creation time (2 bytes)

Reorg number at creation time (2 bytes)

Segment code (1 byte)

ILDS Data Sets

ILE data (50 bytes)

Key (ILK and segment code)

Flags

Old and new copies of:Partition IDReorg numberPointer to logical parent or sec. index targetDatabase record lock ID for segmentPointer to paired logical child for physical pairing

HALDB Segment Format

EPS in logical child segments and secondary indexes

ILK in all PHDAM and PHIDAM segments

Target root key in all PSINDEX segments

PHDAM or PHIDAM segment (not a logical child)SC DB Counter and pointers ILK Data

|-----------------------------Prefix-----------------------------------|

PHDAM or PHIDAM logical child segment

|-----------------------------------Prefix-------------------------------------------|

SC DB Pointers EPS ILK LPCK FID

PSINDEX segmentDB EPS Root Key Index Data

|--------------Prefix--------------------|

SC - segment code DB - delete byte LPCK - log. parent concatenated key FID - fixed intersection data

DFSVSMxx and DFSVSAMPand

Dynamic Allocation

DFSVSMxx Member or DFSVSAMP DD

Data sets assigned to subpool with alphabetic characters

Replaces data set numbers

DBD=dbdname(data set identifier,id)

dbdname - partition name or master database name

data set identifier - Letter A-J, L, or XA-J for user data setsA for secondary indexL for Indirect List Data Set (ILDS)X for PHIDAM primary index

id - subpool id

DFSVSMxx member or DFSVSAMP DDPHDAM example

DSG=A

ILDS

PART1 PART2 PART3

DSG=B

LDDN=PART1L

LDDN=PART2L

LDDN=PART3L

ADDN=PART1A

ADDN=PART2A

ADDN=PART3A

B

DDN=PART1B

B

DDN=PART2B

B

DDN=PART3B

SubpoolSP1

SubpoolSP2

SubpoolSP4

SubpoolSP3

DBD=MASTDB(L,SP1)DBD=MASTDB(A,SP2)DBD=PART3(A,SP3)DBD=MASTDB(B,SP4)

DFSVSMxx member:PHDAM databaseDatabase name: MASTDB3 partitions2 data set groups

Dynamic Allocation

Dynamic allocation uses DBRC information

HALDB Partition Definition utility or DBRC commands create this information

DFSMDA is not used with HALDB

Dynamic allocation should always be used

One exception: The Database Recovery utility (DFSURDB0) requires a DD statement for the database data set

DD statements cannot be used to override data set names

HALDB Partition Selection

Order of Segments in a PHDAM DatabasePHDAM with key range partitioning

Keys are sequential between partitions, random within a partition

PHDAM with Partition Selection Exit routineKeys are not sequential between partitions, random within a

partition

874321456

171215321219

241220122567

921090209791

...

201245167718

241278238900

87417123432

654977209791

...

Order of Segments in a PHIDAM DatabasePHIDAM with key range partitioning

Keys are sequential between partitions, sequential within a partition

PHIDAM with Partition Selection Exit routineKeys are not sequential between partitions, sequential

within a partition

...201245167718

241278238900

87417123432

654977209791

121915321712

321456874

201224122567

902092109791

...

Order of Segments in a PSINDEX DatabasePSINDEX with key range partitioning

Keys are sequential between partitions, sequentialwithin a partition

PSINDEX with Partition Selection Exit routineKeys are not sequential between partitions, sequential

within a partition

321456874

121915321712

201224122567

902092109791

...

201245167718

241278238900

87417123432

654977209791

...

Changing HALDB Databases

Database Changes

Changes affecting all partitions of the database

DBD changes

Partition selection exit

Share level

Nonrecoverable attribute

RSR GSG name and/or tracking level

These changes cannot be made if any partition is authorized

Partition Changes

Changes affecting only one partition

DSN prefixRandomizing module nameNumber of root anchor pointsBytes parameterFree spaceBlock size for OSAMCI size for VSAM*

Changes cannot be made if the partition is authorized

VSAM changes by IDCAMS, not PDU

Database Changes

Changing high keys of partitions

Changes not allowed if affected partitions are authorized

Affected partitions must be unloaded before changes

PINIT flag is set for affected partitions

Affected partitions must be reloaded after initialization

"Affected partitions" are those whose key range is changed

Database records may be moved into or out of affected partitions

Example 1: Splitting a Partition

If partition PB with high key 4000000 needs to be split

–Unload partition PB

HD Unload or HP Unload

–Define new partition PE

Sets PINIT flag for PB and PE

–Initialize partitions PB and PE

–Reload partitions PE and PB

HD Reload or HP Load

–Partitions PA, PC, and PD

are not affected

PA2000000

PB4000000

PC6000000

PD8000000

PA2000000

PE3000000

PC6000000

PD8000000

PB4000000

HD Unload

HD Reload

Define new partition

Example 2: Modifying Partition Boundaries

If records need to be moved from partition PB to PC

–Unload partitions PB and PCHD Unload or HP Unload

–Change high key for partition PB from 4000000 to 3000000Sets PINIT flag for PB and PC

–Initialize partitions PB and PC

–Reload partitions PB and PCHD Reload or HP Load

–Partitions PA and PD are not affected

PA2000000

PB4000000

PC6000000

PD8000000

PA2000000

PC6000000

PD8000000

PB3000000

HD Unload

HD Reload

Change high key

See warning in notes for this page

If partitions PB and PC with high keys 4000000 and 6000000 need to be combined–Unload partitions PB and PC

HD Unload or HP Unload–Delete definition of partition PB

Sets PINIT flag for PC–Initialize partition PC–Reload partition PC

HD Reload or HP Load–Partitions PA and PD are not affected

PA2000000

PB4000000

PC6000000

PD8000000

PA2000000

PC6000000

PD8000000

HD Unload

HD Reload

Example 3: Combining Partitions

Delete partition definition

Some databases have dates as the high-order part of the key

To add a partition for a set of dates (higher keys)–Define it and

initialize it

To delete the partition with the lowest dates (keys) and all of its data–Delete the partition

definition–Turn off PINIT flag for PB

Unloads and reloads are not required for these changes

Databases with Dates for Keys

PA2002xx

PB2003xx

PC2004xx

PD2005xx

PA2002xx

PB2003xx

PC2004xx

PD2005xx

PE2006xx

PA2002xx

PB2003xx

PC2004xx

PD2005xx

PE2006xx

Partition Selection Exit Routine Changes

Changing Partition Selection Exit routine

Change not allowed if affected partitions are authorized

User determines which partitions are affected

Must issue CHANGE.DB PINIT command

or

Use specific partition initialization

Affected partitions must be initialized

Partition Selection Exit Changes

Partition Selection Exit

Adding an exit routine

Preserves high keys in definition as key strings

All partitions must be initialized

Deleting an exit routine

Partition strings cleared

High keys must be defined

All partitions must be initialized

Master Database Changes

Changing Partition Selection Exit

Adding a new partition

Partition must be initialized

User must determine other affected partitions

Deleting a partition

User must determine other affected partitions

Changing a partition string or other changes

User must determine affected partitions

Partition Changes

Partition Data DBDS Changes

GENMAXRECOVPDDEFLTJCLICJCLOICJCLRECOVJCLRECVJCL

Partition can be authorized

Does not require partition initialization

Partition Changes

Partition ID numbers are never reusedDeletion and recreation of same partition name results

in new partition IDData set names are not reused because they include partition ID number

1. PARTC exists 2. PARTC deleted 3. PARTC recreated

Partition Name Partition ID

PARTA 00001PARTB 00002PARTC 00003


PARTA 00001PARTB 00002PARTC 00004


PARTA 00001PARTB 00002

Partition Changes

Disabling and enabling partitions was introduced by APAR PQ73858 for IMS V8

Included in IMS V9

Disabling partitionsDefinitions and information remain in RECONs

Includes partition IDs, DSN prefixes, and recovery information

Partitions are not usedPartitions are ignored by IMS processing

Disabled partitions may be enabledEnabled partitions are made activeEnabled partitions are marked ‘recovery needed’

Partition Changes

Use of disabling and enabling partitionsDisabling is normally done prior to deleting a partition

Keeps recovery information, partition ID, DSN names prefix, etc.If testing is successful, partition is deleted

Deletion removes all informationIf testing is not successful, partition is enabled

Partition is recovered and becomes activeOther partitions may require timestamp recovery

PDU support of disabling and enabling‘Partition status’ field on ‘Change Partition’ panel

DBRC commands for disabling and enabling

Partition Changes

ILDS or PHIDAM Prime Index DBDSs

Changes cannot be made by HALDB Partition Definition Utility

Changes are made only by rebuilding the data sets

Secondary Indexes with HALDB

Secondary Indexes

Partition selection exit may be able to partition a secondary index like its target database

US UK FR

Target Database

US UK FR

Secondary Index

Secondary Indexes

Shared indexes are not supported

Must have unique keys

/SX and /CK may be used

Symbolic pointing is not allowed

Extended Pointer Set (EPS) contains direct pointer

Target root key is stored in index segment

Sparse indexing is supported

Secondary Indexes

Migration of secondary index

Must be migrated with target database

Migration uses HD Unload and ReloadIndexed database is readMIGRATX=YES option is preferred

Migration Aid utility provides limited information about secondary indexes

Creation of secondary index

Created with initial load of target databaseAn alternative will be covered later

Work files not createdPrefix Resolution, HISAM Unload and Reload not used

Secondary Indexes

Addition of secondary indexCannot use standard IMS utilities to add an indexIBM IMS Index Builder will add indexesAlternative for adding secondary index:

Add definitions to target databaseDefine secondary indexApplication program replaces source segments with different source dataApplication program re-replaces source segments with the original data

Database could be initially loaded again with secondary index definitions

Secondary Indexes

Reorganization of target database does not affect secondary index

Secondary index does not have ILDS

Partition Selection Exit may be used

Secondary indexes may need to be reorganized

HALDBMiscellaneous Considerations

Extended Pointer Set (EPS) Adjustments

When out of date pointer is found it is corrected on DASD if:

Access intent is update or exclusive

PROCOPT is update

Locking considerations

Read programs with update PROCOPTs may hold many locks

If block level data sharing is used, block locks are held until sync point

Logging

No logging of "after images" for PHIDAM indexes

Rebuilt with DFSPREC0 utility"Before images" are not archived

Only "after images" are logged for EPS adjustments

Adjustments are not backed out

Database change log records include partition name instead of master database name

No logging for ILDS

Only updated by HD Reload utility or Online Reorganization

OSAM Sequential Buffering

HALDB supports OSAM Sequential Buffering (SB)

Same support as for non-HALDB databases

Recommended for batch sequential processing

SB monitors by data set

Sequential buffering is turned on/off due to access patterns

Sequential buffering may be invoked for some, but not all partitions

Well-organized vs. poorly-organized partitions

Data Set Groups

HALDB supports data set groups (DSGs)

DSGs are not needed to address database size limitations

With non-HALDB databases, DSGs were often used to deal with data set size limitations

DSGs remain useful

May improve performance by moving rarely accessed data to another data set

Reevaluate need for DSGs when migrating databases to HALDB

Database Exit Routines

Database Exit Routines

Routines:Segment Edit/Compression Exit RoutinesData Capture Exit RoutinesData Conversion User Exit RoutineSecondary Index Database Maintenance Exit Routines

Defined for entire database

All partitions in database use the same exit routines

Exit routines are not given partition information

Data Set Allocation

Dynamic allocation uses DBRC information

DFSMDA is not used

Users cannot override data set information with DD statements for database data sets

Data Sharing

Data sharing is supported

SHARELVL specified on master database level

Authorization is by partition

Authorization occurs on first call to partition

HALDB Summary

HALDB Database Candidates

Very large databases

Approaching 4GB (VSAM) or 8GB (OSAM) limitations

To allow for growth

To make databases more manageable

Previously partitioned databases

Using IMS/ESA Partition Support Product (PDB)

User partitioning


Medium and large databases

Parallel processing to meet time deadlines

Application programs

Utilities


Any size databaseFaster reorganizations

May be done more frequently

Partition independenceMaking only parts of the data unavailable for database maintenance

Online reorganization in IMS Version 9

Plan migrations of large databases to HALDBLogical relationships may add significant timePay attention to PHDAM randomization parameters

HALDB Migration Aid utility can analyze existing HALDB databases

Useful when planning repartitioning

Deleting a partition definition deletes its recovery information

Disabling a partition keeps its recovery information

Secondary indexes may require reorganizationsThey are not rebuilt when the indexed database is reorganized

Some Things to Remember

Secondary index cannot be rebuilt from database with IMS utilities

Don't make them non-recoverable unless you have a tool like the IBM Index Builder

PHIDAM indexes and ILDSs have a different recovery process

They are rebuilt with Index/ILDS Rebuild Utility (DFSPREC0)

Plan your scheme for creating HALDB test databases

DBRC registration is required for all databases

Some Things to Remember

HALDB Summary

Benefits

Greater database capacity

Without application changes

Increased database availability

Partitions, not databases, are removed from system

Shortened reorganization process

Batch window is shortened with concurrent processing

WWW.IBM.COM/IMS (The IMS webpage!)

WWW.IBM.COM/SUPPORT/TECHDOCS

Search on "HALDB"

Results:IMS HALDB Database Administration

IMS HALDB Reorganization Number Verification

IMS Version 9 HALDB Online Reorganization

Converting IMS Logically Related Database DBDs to HALDB

Application Design and Programming with HALDB

Migrating to IMS HALDB (High Availability Large Database)

More Information on HALDB

minn ims haldb-2009_

Documents

partition id reorg

database partition selection

ilds reorg

partition idreorg

partition minimal

database partitions

updates reorg

epssince reorg