haldb toolkit present - ims ug may 2013 helsinki

®

IMS Tools 2013

© 2013 IBM Corporation 1

HALDB ToolkitNordics IMS Technical Update May 2013

Krister PetterssonClient Technical Professional [email protected]

© 2013 IBM Corporation2

Agenda

HALDB Toolkit Overview

Application Support

HALDB and DBRC

Applications processing partitions

Maintaining HALDB

Converting to HALDB

Summary



HALDB Database Types are defined by using a single DBD that creates the same structure for each partition.The partitions are defined for access in the DBRC Recons.

Each HALDB Partition is treated as a single dataset and may be allocated and reorganized individually, in sequential groups or all together as a single database. HALDB allows for a maximum of 1001 partitions per database.

A reorganization of the partitions does not required rebuilding of a secondary index or logical relations. These can be self healed during regular processing.

Partitioned HDAM (PHDAM)Partitioned HIDAM (PHIDAM)Partitioned Secondary Index (PSINDEX)


HALDB Toolkit OverviewDBRC and HALDB

Cloning a DBRC definition of HALDB to a different Recon datasetCopy of a HALDB and its DBRC information to a different IMS systemBypassing DBRC requirements

Analysis of HALDB structures, partitions and datasets

HALDB Toolkit UtilitiesSplitting unload files to separate partition filesRebuilding the Indirect List Keys Adding or removing partitions

Convert an existing database to HALDB

ISPF interface


HALDB and DBRC


Testing HALDB Databases

Non-HALDB test databasesOften, test databases are not registered in DBRC RECONsEach application programmer may have one or more versions of a database or set of databases

All HALDB databases must be registered in the DBRC RECON datasets. This leads to considerations regarding testing environments.

DBRC does not allow multiple databases with the same name to be in the same Recon.When using multiple versions of the same database the database must be defined in different RECONsMultiple test versions of a database require multiple RECONs


Defining Test EnvironmentsUsing non-HALDB databases for testing can mean using the same DBD as a production database. Placing the DBD in a different DBDLIB or ACBLIB is sufficient for data separation between IMS environments and allowing testing to not effect a production IMSdatabase.

Replicating a HALDB into a test environment requires establishing the same partition definitions as defined in the Recons of the originating system.

Define the partitions in a test environment using the same names as the source database in a different Recon.

orUse the Partition Definition Utility EXPORT and IMPORT functions to move the definitions to the test environment Recon datasets.


HALDB and DBRC

Transporting a HALDB to a different IMS can be performed several ways.

By using an Unload/Reload using IMS utilities and a different Recon dataset with the DBD and partitions definitions.

Using an application load program into a new set of partitions and the DBD and partition definitions in a different Recon dataset.

By Using Image Copy Recovery process using the HP Image Copy product from one IMS system to another with a new set of Recon definitions.

DFDSS system services for dataset copying.


HALDB and DBRCTransport a DB to a different IMS

Using the IMS Unload and Load toolsRedefine the partition definitions in target RECONReset the Independent List Keys as required

HALDB Toolkit steps• Can create a copy of the DBD and the partition definitions in the

target Recon based on the definitions in the source RECON.• Invoke the IMS utilities to unload the source database

datasets and load the newly defined datasets.• The ILK reset utility during unload can be used to assure the ILK

is current and correct.

COPYDBRC DBD(ddddd) INCLIND(Y)INCLIC(Y)TODD(OUTDBRC)

IHCHALDBRESETILK DBD(ddddd)

IHCHALDBIHCHALDB


Copy via Image CopyThe Partition definitions must be identical in the target Recon as the source partitions.

The partition registration must be added to the target RECON.The Image Copy record must be copied from the source RECON and added to the target Recon.Change the dataset naming to be uniqueAllocate the new target datasets.

Recover the database using the target RECON.

HALDB TOOLKITWill create the partition definition for the target RECON based on the source database definition.Copy the Image Copy entry from the source RECON to the target RECON.Create the IDCAMS statements to allocate the new partition datasets.


Copy a HALDB using DFDSS

Via DFDSSThis process is similar to the Image Copy process.The target RECON information must reflect the same partition structure as defined in the source RECON.The Database must be /DBRed.

HALDB Toolkit• Create the partition definition control statements for the

target RECON from the originals in the source RECON.Create the DFDSS commands to copy the database datasets.


HALDB ToolkitBypassing DBRC


HALDB and DBRCTest DatabasesWhen multiple test databases are needed a separate RECON dataset is required for everybody needing a separate test database. This means that DBRC knowledge is needed to create and maintain the definitions.

TOOLKIT AssistedThe cloning feature of the HALDB Toolkit will copy the partition definition from the original Recon to a newly created Recon. The procedure on the left shows a new recon is created with the partitions for a single HALDB.

//ALLO EXEC PGM=IDCAMS //SYSPRINT DD SYSOUT=* //SYSIN DD * DELETE my.test.RECON1 DELETE my.test.RECON2 DELETE my.test.RECON3 SET MAXCC=0 DEFINE CLUSTER(NAME(my.test.RECON1) …DEFINE CLUSTER(NAME(my.test.RECON2) ….DEFINE CLUSTER(NAME(my.test.RECON3) ….//CRE EXEC PGM=IHCHALDB, REGION=60M //STEPLIB DD DISP=SHR,DSN=IHC320.SIHCLOAD // DD DISP=SHR,DSN=IMS910.SDFSRESL //DFSRESLB DD DISP=SHR,DSN=IMS910.SDFSRESL //IMS DD DISP=SHR,DSN=my.DBDLIB //RECON1 DD DISP=SHR,DSN=SOURCE.RECON1 //RECON2 DD DISP=SHR,DSN=SOURCE.RECON2 //RECON3 DD DISP=SHR,DSN=SOURCE.RECON3 //MSGPRINT DD SYSOUT=* //SYSUDUMP DD SYSOUT=* //SYSOUT DD SYSOUT=* //IHCSYSIN DD * COPYDBRC DBD(nnnnn) –DSNPREF(new.hlq) –DSNDBD(DBD) - INCLIND(YES) –TODD(DBRCOUT) /* //DBRCOUT DD DISP=(,PASS),SPACE=(TRK,(1,1)),UNIT=SYSALLDA

//DBRC EXEC PGM=DSPURX00,REGION=50M,COND=(4,LE) //STEPLIB DD DISP=SHR,DSN=IMS910.SDFSRESL //RECON1 DD DISP=SHR,DSN=my.test.RECON1 //RECON2 DD DISP=SHR,DSN=my.test.RECON2 //RECON3 DD DISP=SHR,DSN=my.test.RECON3 //DFSRESLB DD DISP=SHR,DSN=IMS910.SDFSRESL //IMS DD DISP=SHR,DSN=your.DBDLIB //SYSPRINT DD SYSOUT=* //SYSIN DD * INIT.RECON NOFORCER CATDS TAPEUNIT(3480) // DD DISP=(OLD,DELETE),DSN=*.CRE.DBRCOUT


HALDB and DBRC – Bypassing DBRCThere may be a need to have several HALDBs of the same name for testing or quality assurance purposes. In order to reduce the DBRC maintenance HALDB Toolkit provides a method to simulate a DBRC=N condition in order to have multiple HALDBs of the same name without multiple Recon datasets and their associated maintenance overhead.

This uses a single Recon that holds the HALDB master DBD and partition definitions. The partition names, numbers and key ranges, etc.

A DSNPREFX is substituted as part of the dataset name to avoid naming conflicts. The DBD and partition structure is used from the Recon definitions

DBRC is accessed for the definitions, however, all other DBRC authorization functions are bypassed. As an example, an Image Copy needed flag in the Recon will not stop an application job from accessing the database.


HALDB and DBRCAllocate the HALDB repository dataset. This is a KSDS and a sample allocation job is in member the IHCXDBRI member of the SIHCSAMP library defined during installation.

Specify the bypass parameter values and compile the IHCXMAC macro into the HALDB Toolkit loadlib.

Link a DSPCRTR0 module into the HALDB Toolkit loadlib. This module will be used to access the RECON for the partition structure and will not return any DBRC authorization information.

Execute the database job. The DD statements of the database datasets being bypassed must be included in the execution JCL.

Bypassing DBRC completely for a job use the card//$$DBRCNO DD DUMMY


Backup of DBRC definitions of HALDBBackup of the HALDB definitions. The Physical description of a HALDB is kept in the DBRC RECON datasets. This consists of the DBD and the partition definitions.

A DELETE.DB of the HALDB effectively makes the database unusable. The database no longer exists. There is no clue of the physical file definitions.Even if Image Copies are still available they cannot be used without the partition definitions in a RECON.

HALDB Toolkit allows the DBRC definitions to be copied to a flat file as DBRC statements that may be re-input to the Recon at a later time if needed.

//IHCSYSIN DD * BACKUPDBRC DBD(PHDO2)INCLIND(Y) TODD(outfile)

/*


Applications & HALDB


Parallel Loads of HALDB Partitions

The HALDB Toolkit allows an unload file to be split into single unload files per partition. This allows a load of the HALDB partitions to be run in parallel reducing elapsed time significantly.

The unload files are in HD format and contain only the records for their particular partition. Each file has its own header and trailer record.


Parallel Loads


Splitting an unload file//S2 EXEC PGM=IHCHALDB, // REGION=80M //STEPLIB DD DISP=SHR,DSN=IHC320.SIHCLOAD // DD DISP=SHR,DSN=IMS910.SDFSRESL //RECON1 DD DISP=SHR,DSN=your.RECON1 //RECON2 DD DISP=SHR,DSN=your.RECON2 //RECON3 DD DISP=SHR,DSN=your.RECON3 //IMS DD DISP=SHR,DSN=your.DBDLIB //TRACE DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SYSABENDDD SYSOUT=* //MSGPRINT DD SYSOUT=* //IHCSYSIN DD *

RUN PGM(IHCUSPLT) –DBD(nnnnn)

//DFSUINPT DD DISP=SHR,DSN=your.hd.unload.dataset //PART01Z DD DISP=(,PASS),UNIT=SYSDA,SPACE=(CYL,(50,20)), // DSN=&&PART01 //PART02Z DD DISP=(,PASS),UNIT=SYSDA,SPACE=(CYL,(50,20)), // DSN=&&PART02


Partition Selection API

Partition Selection API is a callable interface that may be usedto split input on a partition boundary. An application program does not know what the key boundaries are in the partitions.

The Partition Selection API allows a database root segment key to be input and returns the partition name and number that the key belongs in.

Selection call:

CALL IHCUAPI using

(″SEL″,HANDLE,RETCODE,RSNCODE,yourkey,PARTNAME,PARTNUM)

The caller provides yourkey.

The API returns PARTNAME and PARTNUM.


Applications & HALDB PartitioningSelective Partition ProcessingBatch application programs may be set to process individual partitions or a set of partitions. The partitions to process areidentified by specifying a DFSHALDB DD statement.

DFSHALDB DD *HALDB PCB=(4,POHIDKA) HALDB PCB=(PCBNUM2,POHIDJA)

DFSHALDB DD *HALDB PCB=(3,PVHDJ5A,NUM=4) HALDB PCB=(PCBNUM7,PVHDJ5B,NUM=3)


Application Support

DFSHALDB statement generator can be run a step prior to the application job.

Provide starting key and ending key

In subsequent application step, you specify

The DFSHALDB DD is generated based on the key range input and passed to the next step.

//S2 EXEC PGM=IHCHALDB,TIME=(1,20), // REGION=80M //STEPLIB DD DISP=SHR,DSN=IHC320.SIHCLOAD// DD DISP=SHR,DSN=IMS910.SDFSRESL //RECON1 DD DISP=SHR,DSN=IMSPLXA.IMS91.RECON1 //RECON2 DD DISP=SHR,DSN=IMSPLXA.IMS91.RECON2 //RECON3 DD DISP=SHR,DSN=IMSPLXA.IMS91.RECON3 //IMS DD DISP=SHR,DSN=IMSA.DBDLIB //$$DYNTR DD * //TRACE DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SYSABEND DD SYSOUT=* //MSGPRINT DD SYSOUT=* //IHCSYSIN DD *

RUN PGM(IHCUDFSH) DBD(PHDO2) PCB(2) //KEYS DD * 08001000 98008000 //DFSHALDB DD UNIT=SYDSA,DISP=(,PASS),// DSN=&&HAL,SPACE=(TRK,1)

low key

high key

//DFSHALDB DD DSN=&&HAL,DISP=(OLD,DELETE)


Maintaining HALDB


Production Maintenance

Deleting a single partition

Merging two DBDs into one DBD

Changing partition boundaries

Healing Index Pointers

Reorganizing a PSINDEX

Maintaining databases online


Deleting a partition

//IHCSYSIN DD * REORGIND DBD(PHDO2) DROP(PHDO22)

/*

21 22 23 nn

21 23 nn


Splitting a partition

//IHCSYSIN DD * RUN PGM(IHCSPLT) DBD(PHDO2)

/*

21

22

23 nn

21 23 nn



//IHCSYSIN DD * MAINTAIN DBD(PHDO2) PARTITION(name) PARTSIZE(4096)

/*

21

22

23 nn

21 23 nn

22



//IHCSYSIN DD * MAINTAIN DBD(PHDO2) PARTITION(name) KEYS(keysdd)

/*

21

22

23 nn

21 23 nn

22


HALDB - ILDS

There is an Indirect List Data Set (ILDS) is associated with each partition of a HALDB. The ILDS is a VSAM KSDS.

The ILDS contains an RBA pointer to each secondary index or logical relationship target in the HALDB. An entry in the ILDS consists of an Independent List Entry (ILE), Independent List Key (ILK), segment code, partition ID, RBA and reorg number.

ILDS entries may become “out of date” during HALDB maintenance. The “out of date” condition is noted when the reorganization number in logical relationship or secondary index pointer does not match the reorganization number of the partition.


HALDB Overview – The EPS and ILE

EPS

Partition ID

Reorg #:1

RBA

EPS KEY

Segment A

If Partition ID in EPS is correct andreorg # in EPS matches reorg # inPartition DBDS, we use EPS RBApointer, else we use the ILK to findILE and use ILE's RBA

ILDS

ILE

Partition DBDS

Current reorg #:1

ILKSegment B

ILE

ILK

Segment Code

Partition ID

Current reorg #: 1

Current RBA

. . .Current reorg #:1


HALDB Overview – The EPS and ILE

EPS

Partition ID

Reorg #:1

RBA

EPS KEY

Segment A

If Partition ID in EPS is correct andreorg # in EPS matches reorg # inPartition DBDS, we use EPS RBApointer, else we use the ILK to findILE and use ILE's RBA

ILDS

ILE

Partition DBDS

Current reorg #:2

ILKSegment B

ILE

ILK

Segment Code

Partition ID

Current reorg #: 2

Current RBA

. . .Current reorg #:2


Database Pointer HealingHeal a secondary index pointer

EPS pointer of the entry in the ILDS is not “healed” after REORG. An example may be that a single partition was reorganized. The entries for that partition are now “out of date” in the sense that they point to where the target segments used to be before the partition is reorganized.HALDB has a pointer healing process that updates the ILDS pointer with the correct RBA information during database access.Pointer healing during regular database access may not be considered directly related to the access and might be referred to as extra overhead.HALDB Toolkit provides batch pointer healing. The primary database must not be open for updateThe secondary indexes must be offline.

34

//IHCSYSIN DD * PTRHEAL DBD(PHDO2)

/*


HALDB AnalyzerA HALDB is like any other database. It requires monitoring and maintenance. The HALDB analyzer can assist by verifying HALDB Structures. The analyzer can validate ILKs, root key location, EPS pointers against the ILE and performance information.The analyzer information regarding the perfomance related information is set by thresholds in the parameter of the resource being checked.The parameters are specified to give a low and high value. These are the thresholds that the analyzer uses to provide feedback.• PTRHEAL(50,80)

The first value is the Low threshold: a warning about an upcoming problemThe second value is the High threshold: problem is now more urgent and may need attention

35


HALDB Analyzer

36


HALDB Analyzer

37


Health CheckerRuns very fast

Up to 8 datasets at the same timeVerification of ILK and ILE can be turned off

Identifies potential performance impactsILK/ILE missing or invalidPartition Selection exit errorsRandomizer errors

Provides decision support on partition maintenanceCombining/splitting/rearranging segmentsAdjusting randomizing parameters for partitions

User set warning thresholdsNormal and Criticalparm(norm,crit)

38


Converting to HALDB


Migration to HALDB

HDAM Database

PHDAM Database


Converting to HALDBThe HALDB Toolkit provides the functions to convert an IMS fullfunction database to HALDB:

Find partition boundariesConvert the non-HALDB DBD to a HALDB DBDCreate the control card information for DBRC Update the Recon datasetCalculate appropriate size and allocate the new files for the

database datasets of each partitionInvoke the appropriate utilities to unload, allocate, load and update

the Recon information needed to convert the database.

Can convert SHISAM, HISAM, DEDBs and HDAM databases.

The database can remain open for update during the conversion process when IMS Online Reorg Facility is available and used forthe conversion. This option can help maximize the availability of the database being converted.

41


CONVERT

//ICHSYSIN DD *CONVERT DBD(dbdname) Primary DBD nameDBDPATT(*****…) to create a partition name

ONLINE(N) to create a batch processDSNPREF(your.hlq) appended with DBD namePARTSIZE(2048)|PARTNUM(nn)|KEYS(ddname)

Takeover conceptNo renames, all dataset names are newActivated only if successfulUse of temporary RECONsRECON notification delayedActivate changed DBD


HALDB ISPF Interface


HALDB Toolkit - ISPF Interface

• All functions previous addressed can be invoked through an ISPF interface for JCL creation and tracking projects that are in progress.

– Environment set up– Project Tracker for Database conversion management– DBD Conversions – HALDB Maintenance Functions– DBRC for HALDB functions– Database Utilities



• Environment setup

A VSAM cluster - IHCXKSDS is used as a repository for the toolkit information.

→IMS utilities used for maintenance and conversion steps

→Identification of databases in maintenance

→Identification of database in conversion

This allows projects to be more easily tracked and controlled over a period of time.


IMS HALDB Toolkit - Benefits

HALDB maintenance assistance

Splitting and combining partitionsSingle step pointer healing for indexes

Full Function to HALDB single step conversion

Application testing enhancements

DBRC information replicationDBRC authorization bypass


IMS HALDB Toolkit - Benefits

Invokes Standard IMS HD Reorganization utilities.

Invokes High Performance IMS Tools.

Interfaces with IMS Online Reorganization Facility.

May use the High Performance I/O capabilities of the operating system.

Provides a single point of control and management for on-going HALDB conversion and maintenance projects.

haldb toolkit present - ims ug may 2013 helsinki

Technology

haldb partition

dbrc definition of haldb

thetarget recon

target recon information

different recon dataset

partitions definitions

haldb toolkitwill

single database