sr ims haldb

IMSHigh Availability Large Database

(HALDB)(subset of course CMW46)

The world depends on it

John Harper - IBM IMS Development

[email protected]

CMW46 - IMS HALDB 1

Topics

Overview Database StructuresDefinitionPartition SelectionDBRC SupportDatabase CommandsUtilitiesDatabase Creation and MigrationChanging DatabasesSecondary IndexesApplication and Misc. Considerations HALDB Online ReorganizationSummary

CMW46 - IMS HALDB 2

HALDB Overview

CMW46 - IMS HALDB 3

Large Database

Databases are partitionedUp to 1001 partitions per databasePartitions have up to 10 data set groups

High Availability Database

Partition independenceAllocation, authorization, reorganization, and recovery are by partition

Self healing pointersReorganization of partition does not require changes to secondary indexes or logically related databases

HALDB (High Availability Large Database)

CMW46 - IMS HALDB 4

Benefits

Greater database capacity Without application changes

Increased database availabilityPartitions, not databases, are removed from systemShortened reorganization processBatch window is shortened with concurrent processing

Improved manageabilityData sets may be smaller

HALDB (High Availability Large Database)

CMW46 - IMS HALDB 5

Highlights

Hierarchic structure is maintainedA database record resides in one partition

Minimal (or no) application changes requiredCannot initially load logical child segments

New status code for load programsSome secondary index segment formats change

May require I/O area changes when secondary index is processed as a database

New database typesPHDAM - partitioned HDAMPHIDAM - partitioned HIDAM

Index is partitionedPSINDEX - partitioned secondary index

CMW46 - IMS HALDB 6

Highlights

OSAM and VSAM (ESDS and KSDS) are supportedPartition selection is done by key range or user exit routineLogical relationships and secondary indexes are supported

Secondary indexes may be partitioned

DBRC is requiredDatabases must be registered

Dynamic allocation uses DBRC informationDFSMDA is not used

CMW46 - IMS HALDB 7

Definition Process

DBDGEN Used to define database

Hierarchic structure, data set group boundaries, pointer options, logical relationships, secondary indexes,...

HALDB Partition Definition UtilityISPF basedUsed to define partitions in database

Partition selection, space characteristics, randomizers,...

Or

DBRC Commands

CMW46 - IMS HALDB 8

Indirect Pointers

HALDB uses both direct and indirect pointers for logical relationships and secondary indexes

Direct pointers point to segments

Indirect pointers "point" to Indirect List Entries (ILEs) in Indirect List Data Set (ILDS)

ILEs have token for keyILEs contain direct pointer to segment

ILDS is KSDS associated with a Partition

CMW46 - IMS HALDB 9

Extended Pointer Set

Extended Pointer Set (EPS) is used for logical relationships and secondary indexes

EPS is not updated by reorganizations! EPS contains direct pointer, reorganization number, and ILK

If reorg number is current, direct pointer is usedIf reorg number is not current, ILK is use to find ILE in ILDSILE contains pointer to segment

Direct pointer and reorg number is EPS are updated when ILE is used

Self healing pointers!

CMW46 - IMS HALDB 10

HALDB Database Structure

Each partition in a database has a unique partition ID (PID)

A reorganization number is maintained in each partition

Incremented by each reorganization reload

Each segment in PHDAM or PHIDAM database is assigned a unique token when created

Indirect List Entry Key (ILK)8 bytes stored in segment prefix


Self-Healing PointersUsing an Extended Pointer Set (EPS)

Since reorg # in EPS matches reorg # in Partition DBDS, we use EPS RBA pointer

EPSPartition IDReorg # : 1RBAILK...

Segment A

ILEILKSegment CodePartition IDCurrent reorg # : 1Current RBA...

ILDS

ILE

KSDS

Current reorg # :1

Segment B

Partition DBDS

ILK

Indirect List Key (ILK) is stored in segment prefix.

EPS KEY


Self-Healing PointersAfter reorganization of Partition

Reorg moves segment B, updates reorg # in ILE and Partition DBDS, and changes Current RBA in ILE


Segment A


ILE

KSDS

ILDS

Current reorg # :2

Segment B

Partition DBDS

ILK

"broken" pointer


EPS KEY


Self-Healing PointersUsing the EPS after the reorganization

Since reorg # in EPS does not match reorg # in Partition DBDS, we use the ILE RBA pointer


Segment A


ILE

KSDS

ILDS

Segment B

Partition DBDS

Current reorg # :2

ILK

"broken" pointer


EPS KEY


Self-Healing Pointers"Healing" the EPS

When we use the ILE, we update the reorg # and RBA pointer in the EPS


Segment A


ILE

KSDS

ILDS

Segment B

Partition DBDS

ILK

Current reorg # :2

"healed" pointer


EPS KEY


HALDB Database Data Sets

Each HALDB database has up to 1001 partitionsPHIDAM has index, ILDS, and up to 10 data set groups per partition

3 to 12 data sets per partition3 to 12,012 data sets per database

PHDAM has ILDS and up to 10 data set groups per partition

2 to 11 data sets per partition2 to 11,011 data sets per database

PSINDEX has no ILDS or data set groups1 data set per partition1 to 1001 data sets per secondary index


HALDB Database Data Sets

...

...

...

...

...

...

...

...Index

ILDS

Data

PHIDAM PHDAM PSINDEX

...

1 to 1001 partitions 1 to 1001 partitions

1 to 1001 partitions

1 to 10 data set groups

... ... ...


Partition Independence

CommandsAllowed on both databases and partitions

AvailabilityPartitions are allocated and authorized independently

SchedulingBased on database availability

PCBs and INQY calls report database availabilityPartition may be unavailable with available database

Database UtilitiesAllowed on individual partitionsConcurrent processing of multiple partitions allowed


HALDB Overview

Database candidates for HALDBVery large databases

Approaching 4GB limitationTheoretical limit is now over 40 terabytes per database

Medium and large databasesParallel processing to meet time deadlinesLess frequent reorganizations due to more free space

Any databaseMore frequent reorganizationsMaking only parts of the data unavailable for database maintenance


HALDB Database Structures


Database Structures

Logical relationships

Virtual pairing is not allowedLimited to unidirectional or physically paired

Logical child segments cannot be initially loadedMust be added by update

Symbolic pointers are not used

Secondary indexes must have unique keys

/SX or /CK may be used to create uniqueness/SX is increased from 4 to 8 bytes (ILK)

Symbolic pointers are not used


Database Structures

PHIDAM primary indexes are not separately defined

Defined as part of the PHIDAM database

Applies to DBDGEN and system definition

Parent pointers

All segments have physical parent pointers


Database Data Sets

The data sets in a partition have generated data set names and DDNAMEs.Letters are used to distinguish them.

X - PHIDAM indexL - ILDSA through J - Data data setsA - PSINDEX

PHIDAM PHDAM PSINDEX

...

...

...

...

...

...

...

...Index

ILDS

Data

...

1 to 1001 partitions 1 to 1001 partitions

1 to 1001 partitions

1 to 10 data set groups

X

L L

A

A-J

A-J

... ... ...


Database Data Sets

DDNAMEsPartition name is basis for DDNAME

Up to 7 charactersAssigned in HALDB Partition Definition Utility or DBRC command

Letter is used as suffixX for PHIDAM index L for ILDSA for PSINDEXA through J for data

Example:Partition name: FRANCEDDNAME for PHIDAM Index: FRANCEX


Database Data Sets

Data set namesBegin with data set name prefix for the partition

Up to 37 charactersAssigned in HALDB Partition Definition Utility or DBRC command

Letter and Partition ID are used as suffixX for PHIDAM index L for ILDSA for PSINDEXA through J for data

Example:Partition data set name prefix IMP0.DB.INV23.FRPartition ID: 00004Data set name PHIDAM Index: IMP0.DB.INV23.FR.X00004


Database Data Sets

Data set size limitations

Maximum data set size is 4GBApplies to OSAM and VSAM

OSAM block sizes must be even


Partition DDNAMEs and Data Set Names

Data set DDNAME Data Set Name

Data set group 1 Partition_nameA DSN_prefix.ApartitionID

Data set group 2 Partition_nameB DSN_prefix.BpartitionID

Data set group 3 Partition_nameC DSN_prefix.CpartitionID

... ... ...

Data set group 10 Partition_nameJ DSN_prefix.JpartitionID

ILDS Partition_nameL DSN_prefix.LpartitionID

PHIDAM Index Partition_nameX DSN_prefix.XpartitionID

Secondary Index Partition_nameA DSN_prefix.ApartitionID

Partition_name assigned by user in HALDB Partition Definition Utility or DBRC command

DSN_prefix assigned by user in HALDB Partition Definition Utility or DBRC command

PartitionID assigned by IMS in HALDB Partition Definition Utility or DBRC command



Partition_name of FRANCE

DSN_prefix of IMP0.DB.INV23.FRANCE

PartitionID of 00004


Data set group 1 FRANCEA IMP0.DB.INV23.FRANCE.A00004

Data set group 2 FRANCEB IMP0.DB.INV23.FRANCE.B00004

Data set group 3 FRANCEC IMP0.DB.INV23.FRANCE.C00004

... ... ...

Data set group 10 FRANCEJ IMP0.DB.INV23.FRANCE.J00004

ILDS FRANCEL IMP0.DB.INV23.FRANCE.L00004

PHIDAM Index FRANCEX IMP0.DB.INV23.FRANCE.X00004

Example: PHIDAM with 10 data set groups, FRANCE partition



Partition_name of CANADA

DSN_prefix of IMP0.DB.INV23.CANADA

PartitionID of 00011


Data set group 1 CANADAA IMP0.DB.INV23.CANADA.A00011

Data set group 2 CANADAB IMP0.DB.INV23.CANADA.B00011

Data set group 3 CANADAC IMP0.DB.INV23.CANADA.C00011

... ... ...

Data set group 10 CANADAJ IMP0.DB.INV23.CANADA.J00011

ILDS CANADAL IMP0.DB.INV23.CANADA.L00011

PHIDAM Index CANADAX IMP0.DB.INV23.CANADA.X00011

Example: PHIDAM with 10 data set groups, CANADA partition


Extended Pointer Set

Extended pointer set (EPS)Used for logical relationships and secondary indexes

Data:

Reorg number of target partitionTarget partition IDPointer to logical parent or sec. index targetPointer to paired logical child for physical pairingDatabase record lock ID of segmentILK

28 bytes


ILDS Data Sets

Indirect List Entries (ILEs)

Created or updated by reorg reload

Reorgs do not update pointers in segments

Not created or updated by non-reload processing

This processing updates pointers in segments

Initial load does not create ILEs


ILDS Data Sets

ILE keys (9 bytes)

ILK (8 bytes)

RBA of segment at its creation time (4 bytes)

Partition id at creation time (2 bytes)

Reorg number at creation time (2 bytes)

Segment code (1 byte)


ILDS Data Sets

ILE data (50 bytes)

Key (ILK and segment code)

Flags

Old and new copies of:Partition IDReorg numberPointer to logical parent or sec. index targetDatabase record lock ID for segmentPointer to paired logical child for physical pairing


HALDB Segment Format

EPS in logical child segments and secondary indexesILK in all PHDAM and PHIDAM segmentsTarget root key in all PSINDEX segments

SC - segment code DB - delete byte LPCK - log. parent concatenated key FID - fixed intersection data

PSINDEX segmentDB EPS Root Key Index Data

|--------------Prefix--------------------|

PHDAM or PHIDAM segment (not a logical child)SC DB Counter and pointers ILK Data|-----------------------------Prefix-----------------------------------|

PHDAM or PHIDAM logical child segment

|-----------------------------------Prefix-------------------------------------------|

SC DB Pointers EPS ILK LPCK FID


DFSVSMxx and DFSVSAMPand

Dynamic Allocation


DFSVSMxx Member or DFSVSAMP DD

Data sets assigned to subpool with alphabetic characters

Replaces data set numbers

dbdname - partition name or master database name

data set identifier - Letter A-J, L, or XA-J for user data setsA for secondary indexL for Indirect List Data Set (ILDS)X for PHIDAM primary index

id - subpool id

DBD=dbdname(data set identifier,id)


DFSVSMxx member or DFSVSAMP DDPHDAM example

DSG=A

ILDS

PART1 PART2 PART3

DSG=B

LDDN=PART1L

LDDN=PART2L

LDDN=PART3L

A

DDN=PART1A

A

DDN=PART2A

A

DDN=PART3A

B

DDN=PART1B

B

DDN=PART2B

B

DDN=PART3B

SubpoolSP1

SubpoolSP2

SubpoolSP4

SubpoolSP3

PHDAM databaseDatabase name: MASTDB3 partitions2 data set groups

DBD=MASTDB(L,SP1)DBD=MASTDB(A,SP2)DBD=PART3(A,SP3)DBD=MASTDB(B,SP4)

DFSVSMxx member:


Dynamic Allocation

Dynamic allocation uses DBRC information

HALDB Partition Definition utility or DBRC commands create this information

DFSMDA is not used with HALDB

Dynamic allocation should always be used

One exception: The Database Recovery utility (DFSURDB0) requires a DD statement for the database data set

DD statements cannot be used to override data set names


HALDB Partition Selection


Order of Segments in a PHDAM DatabasePHDAM with key range partitioning

Keys are sequential between partitions, random within a partition

PHDAM with Partition Selection Exit routineKeys are not sequential between partitions, random within a partition

874321456

171215321219

241220122567

921090209791

...

201245167718

241278238900

87417123432

654977209791

...


Order of Segments in a PHIDAM DatabasePHIDAM with key range partitioning

Keys are sequential between partitions, sequential within a partition

PHIDAM with Partition Selection Exit routineKeys are not sequential between partitions, sequential within a partition

...201245167718

241278238900

87417123432

654977209791

121915321712

321456874

201224122567

902092109791

...


Order of Segments in a PSINDEX DatabasePSINDEX with key range partitioning

Keys are sequential between partitions, sequential within a partition

PSINDEX with Partition Selection Exit routineKeys are not sequential between partitions, sequential within a partition

321456874

121915321712

201224122567

902092109791

...

201245167718

241278238900

87417123432

654977209791

...


Changing HALDB Databases


Database Changes

Changes affecting all partitions of the database

DBD changes

Partition selection exit

Share level

Nonrecoverable attribute

RSR GSG name and/or tracking level

These changes cannot be made if any partition is authorized


Partition Changes

Changes affecting only one partitionDSN prefixRandomizing module nameNumber of root anchor pointsBytes parameterFree spaceBlock size for OSAMCI size for VSAM

These changes cannot be made if the partition is authorized


Database Changes

Changing high keys of partitions

Changes not allowed if affected partitions are authorized

Affected partitions must be unloaded before changes

PINIT flag is set for affected partitions

Affected partitions must be reloaded after initialization

"Affected partitions" are those whose key range is changed

Database records may be moved into or out of affected partitions


If partition PB with high key 4000000 needs to be split

Unload partition PBHD Unload or HP Unload

Define new partition PE Sets PINIT flag for PB and PE

Initialize partitions PB and PE

Reload partitions PE and PBHD Reload or HP Load

Partitions PA, PC, and PD

are not affected

Example 1: Splitting a Partition

PA2000000

PB4000000

PC6000000

PD8000000

PA2000000

PE3000000

PC6000000

PD8000000

PB4000000

HD Unload

HD Reload

Define new partition


Example 2: Modifying Partition Boundaries

If records need to be moved from partition PB to PC

Unload partitions PB and PCHD Unload or HP Unload

Change high key for partition PB from 4000000 to 3000000

Sets PINIT flag for PB and PC

Initialize partitions PB and PC

Reload partitions PB and PCHD Reload or HP Load

Partitions PA and PD are not affected

PA2000000

PB4000000

PC6000000

PD8000000

PA2000000

PC6000000

PD8000000

PB3000000

HD Unload

HD Reload

Change high key

See warning in notes for this pageCMW46 - IMS HALDB 48

If partitions PB and PC with high keys 4000000 and 6000000 need to be combined

Unload partitions PB and PCHD Unload or HP Unload

Delete definition of partition PBSets PINIT flag for PC

Initialize partition PCReload partition PC

HD Reload or HP LoadPartitions PA and PD are not affected

PA2000000

PB4000000

PC6000000

PD8000000

PA2000000

PC6000000

PD8000000

HD Unload

HD Reload

Example 3: Combining Partitions

Delete partition definition


Some databases have dates as the high-order part of the key

To add a partition for a set of dates (higher keys)Define it and

initialize it

To delete the partition with the lowest dates (keys) and all of its data

Delete the partition definition

Turn off PINIT flag for PB

Unloads and reloads are not required for these changes

Databases with Dates for Keys

PA2002xx

PB2003xx

PC2004xx

PD2005xx

PA2002xx

PB2003xx

PC2004xx

PD2005xx

PE2006xx

PA2002xx

PB2003xx

PC2004xx

PD2005xx

PE2006xx


Partition Selection Exit Routine Changes

Changing Partition Selection Exit routine

Change not allowed if affected partitions are authorized

User determines which partitions are affected

Must issue CHANGE.DB PINIT command

or

Use specific partition initialization

Affected partitions must be initialized


Partition Selection Exit Changes

Partition Selection Exit

Adding an exit routine

Preserves high keys in definition as key strings

All partitions must be initialized

Deleting an exit routine

Partition strings cleared

High keys must be defined

All partitions must be initialized


Master Database Changes

Changing Partition Selection Exit

Adding a new partition

Partition must be initialized

User must determine other affected partitions

Deleting a partition

User must determine other affected partitions

Changing a partition string or other changes

User must determine affected partitions


Partition Changes

Partition Data DBDS Changes

GENMAXRECOVPDDEFLTJCLICJCLOICJCLRECOVJCLRECVJCL

Partition can be authorized

Does not require partition initialization


Partition Changes

Partition ID numbers are never reusedDeletion and recreation of same partition name results in new partition ID

Data set names are not reused because they include partition ID number

Partition Name Partition ID

PARTA 00001PARTB 00002PARTC 00003


PARTA 00001PARTB 00002PARTC 00004


PARTA 00001PARTB 00002

1. PARTC exists 2. PARTC deleted 3. PARTC recreated


Disabling and enabling partitions was introduced by APAR PQ73858 for IMS V8

Included in IMS V9

Disabling partitionsDefinitions and information remain in RECONs

Includes partition IDs, DSN prefixes, and recovery informationPartitions are not used

Partitions are ignored by IMS processing

Disabled partitions may be enabledEnabled partitions are made activeEnabled partitions are marked ‘recovery needed’

Partition Changes


Use of disabling and enabling partitionsDisabling is normally done prior to deleting a partition

Keeps recovery information, partition ID, DSN names prefix, etc.If testing is successful, partition is deleted

Deletion removes all informationIf testing is not successful, partition is enabled

Partition is recovered and becomes activeOther partitions may require timestamp recovery

PDU support of disabling and enabling‘Partition status’ field on ‘Change Partition’ panel

DBRC commands for disabling and enabling

Partition Changes

CHANGE.PART DBD(dname) PART(pname) DISABLECHANGE.PART DBD(dname) PART(pname) ENABLE


Partition Changes

ILDS or PHIDAM Prime Index DBDSs

Changes cannot be made by HALDB Partition Definition Utility

Changes are made only by rebuilding the data sets


Secondary Indexes with HALDB


Secondary Indexes

Partition selection exit may be able to partition a secondary index like its target database

US UK FR

Target Database

US UK FR

Secondary Index


Secondary Indexes

Shared indexes are not supported

Must have unique keys/SX and /CK may be used

Symbolic pointing is not allowedExtended Pointer Set (EPS) contains direct pointer

Target root key is stored in index segment

Sparse indexing is supported


Secondary Indexes

Migration of secondary index

Must be migrated with target database

Migration uses HD Unload and ReloadIndexed database is readMIGRATX=YES option is preferred

Migration Aid utility provides limited information about secondary indexes

Creation of secondary indexCreated with initial load of target database

An alternative will be covered later

Work files not createdPrefix Resolution, HISAM Unload and Reload not used


Secondary Indexes

Addition of secondary indexCannot use standard IMS utilities to add an indexIBM IMS Index Builder will add indexesAlternative for adding secondary index:

Add definitions to target databaseDefine secondary indexApplication program replaces source segments with different source dataApplication program re-replaces source segments with the original data

Database could be initially loaded again with secondary index definitions


Secondary Indexes

Reorganization of target database does not affect secondary index

Secondary index does not have ILDS

Partition Selection Exit may be used

Secondary indexes may need to be reorganized


HALDBMiscellaneous Considerations


Extended Pointer Set (EPS) Adjustments

When out of date pointer is found it is corrected on DASD if:

Access intent is update or exclusive

PROCOPT is update

Locking considerationsRead programs with update PROCOPTs may hold many locks

If block level data sharing is used, block locks are held until sync point


Logging

No logging of "after images" for PHIDAM indexesRebuilt with DFSPREC0 utility"Before images" are not archived

Only "after images" are logged for EPS adjustmentsAdjustments are not backed out

Database change log records include partition name instead of master database name

No logging for ILDSOnly updated by HD Reload utility or Online Reorganization


OSAM Sequential Buffering

HALDB supports OSAM Sequential Buffering (SB)Same support as for non-HALDB databases

Recommended for batch sequential processing

SB monitors by data setSequential buffering is turned on/off due to access patterns

Sequential buffering may be invoked for some, but not all partitions

Well-organized vs. poorly-organized partitions


Data Set Groups

HALDB supports data set groups (DSGs)

DSGs are not needed to address database size limitations

With non-HALDB databases, DSGs were often used to deal with data set size limitations

DSGs remain useful May improve performance by moving rarely accessed data to another data set

Reevaluate need for DSGs when migrating databases to HALDB


Database Exit Routines

Database Exit Routines

Routines:Segment Edit/Compression Exit RoutinesData Capture Exit RoutinesData Conversion User Exit RoutineSecondary Index Database Maintenance Exit Routines

Defined for entire database

All partitions in database use the same exit routines

Exit routines are not given partition information


Data Set Allocation

Dynamic allocation uses DBRC information

DFSMDA is not used

Users cannot override data set information with DD statements for database data sets


Data Sharing

Data sharing is supported

SHARELVL specified on master database level

Authorization is by partition

Authorization occurs on first call to partition


HALDB Summary


HALDB Database Candidates

Very large databasesApproaching 4GB (VSAM) or 8GB (OSAM) limitations

To allow for growth

To make databases more manageable

Previously partitioned databases

Using IMS/ESA Partition Support Product (PDB)

User partitioning



Medium and large databases

Parallel processing to meet time deadlines

Application programs

Utilities



Any size databaseFaster reorganizations

May be done more frequentlyPartition independence

Making only parts of the data unavailable for database maintenance

Online reorganization in IMS Version 9


Plan migrations of large databases to HALDBLogical relationships may add significant timePay attention to PHDAM randomization parameters

HALDB Migration Aid utility can analyze existing HALDB databasesUseful when planning repartitioning

Deleting a partition definition deletes its recovery informationDisabling a partition keeps its recovery information

Secondary indexes may require reorganizationsThey are not rebuilt when the indexed database is reorganized

Some Things to Remember


Secondary index cannot be rebuilt from database with IMS utilitiesDon't make them non-recoverable unless you have a tool like the IBM Index Builder

PHIDAM indexes and ILDSs have a different recovery processThey are rebuilt with Index/ILDS Rebuild Utility (DFSPREC0)

Plan your scheme for creating HALDB test databasesDBRC registration is required for all databases

Some Things to Remember


BenefitsGreater database capacity

Without application changesIncreased database availability

Partitions, not databases, are removed from systemShortened reorganization processBatch window is shortened with concurrent processing

HALDB Summary


WWW.IBM.COM/IMS (The IMS webpage!)

WWW.IBM.COM/SUPPORT/TECHDOCS

Search on "HALDB"

Results:IMS HALDB Database Administration IMS HALDB Reorganization Number Verification IMS Version 9 HALDB Online Reorganization Converting IMS Logically Related Database DBDs to HALDB Application Design and Programming with HALDB Migrating to IMS HALDB (High Availability Large Database)

More Information on HALDB


sr ims haldb

Documents

partition id reorg

partition cmw46 ims

partition minimal

epssince reorg

updates reorg

haldb database structure

cmw46 ims haldb12

cmw46 ims haldb7