ims to db2 migration: how a fortune 500 company made the move in record time with minimal disruption

Damon Anderson and Bill BostridgeJuly 2016

IMS to DB2 Migration: How a Fortune 500 Company Made the Move in

Record Time With Minimal Disruption.

Meet Today’s Presenters

2

Bill BostridgeDirector of Business Development

Syncsort

Damon AndersonManager of Data Services

Leading Fortune 500 Global Distribution Company

Syncsort is the Global Leader in Big Iron to Big Data Solutions

3

Syncsort Confidential and Proprietary - do not copy or distribute

• We are a leading data management software company and the global leader in Big Iron to Big Data solutions for large enterprises

• Our innovative and high-performance software enables customers to tap into the value of their data assets while dramatically reducing the cost of managing mainframe and legacy systems

• Our Big Iron to Big Data solutions address the complex set of requirements for delivering rich data from mainframe and legacy sources to Big Data environments, powering business and operational analytics

• A trusted industry leader for nearly 50 years, we deliver a unique focus on customer value through cost-effective solutions and unparalleled support

Marquee global customer base of leaders and emerging businesses across all major industries

WOODCLIFF LAKE, NJ

JAPAN

SINGAPORE

Deep engineering and go-to-market relationships with strategic partners in Big Iron and Big Data ecosystems

Marquee global customer base of leaders and emerging businesses across all major industries

Syncsort DL/2 - Transparent Data Migration to DB2®

Migrate IMS Segments to DB2 tables without making anyapplication changes

Eliminate IMS DB and 3rd party IMS support tools

Move application maintenance to SQL and DB2

Syncsort DL2 is not a replication tool, it is a replacement for IMS DB

Combination of software and services for a low risk migration from IMS to DB2


4

What Do We Mean by Transparent Data Migration?

• No modifications, compiles, recompiles of existing application programs

• Migrate at your own pace – one database at a time or multiple databases

• Environmental changes for batch JCL and CICS can be made in non-

disruptively in advance

• Simple switch between IMS and DB2 for application testing

• Fallback capability

• Easiest and fastest route to DB2 and value delivery

LOWEST RISK MIGRATION APPROACH – PROVEN TIME AND TIME AGAIN

5


DL/2: Transparent Migration Process Overview

6

Before During After

IMS

NO

DB2

YES

DL2 Stub

Application Program

Static SQL

Datain

DB2?

IMS Stub

Application Program

DL2 Stub

Application Program

IMSDB2



7

Before During After

IMS

NO

DB2

YES

DL2 Stub

Application Program

Static SQL

Datain

DB2?

IMS Stub

Application Program

DL2 Stub

Application Program

IMSDB2



8

Before During After

IMS

NO

DB2

YES

DL2 Stub

Application Program

Static SQL

Datain

DB2?

IMS Stub

Application Program

DL2 Stub

Application Program

IMSDB2


Accessing IMS Databases

9

DBCTL

DLISAS DBRC

IMS/DC

CICS/TS

BMP

DLI/DBB

BATCH

Supported by DL/2:

• xxxTDLI Assembler IMS database calls

• EXEC DLI COBOL IMS system calls

• AIBTDLI PL/I and LE/370 IMS command codes


July, 2016

10

Review of a

Migration from

IMS to DB2

with DL/2

Damon Anderson

Anixter, Inc.

11

Why Migrate from IMS to DB2

Save Money

Cost of IMS Software

Cost of Software to monitor IMS

Cost of Software to Backup, Recover,

Reorganize, Validate

Personnel / Consultants

Staffing Shortages

Fewer and fewer developers and DBA’s

with IMS Skills

Consolidate to one z/OS DBMS

Simplify

12

More Why Migrate off of IMS

Forced to upgrade IMS to stay on supported

release.

IMS 11 – GA 2009 – EOS 2014

IMS 12 – GA 2011 – EOS 2016

IMS 13 – GA 2013

IMS 14 – GA 2015

Similarly forced remain current on maintenance

Databases were functionally stable.

No Changes for several years.

The Question is: Is the new release providing

any value to you and your organization?

13

Ways to Migrate from IMS to DB2

Recode the application.

Replace all IMS calls with DB2 calls.

Risk of inconsistent results.

Replicate IMS to DB2.

Great for Readers, but what application is read-

only. I want to update!

Replication delays – “Near Real Time”

Use a Transparency Layer – AKA: DL/2

Intercepts all IMS Calls

Conditionally passes them to IMS

Smoke & Mirrors

14

Environmental Overview

IMS V11 running DBCTL

32 Full Function Database, 28 Primary Indexes,

85 Secondary Indexes

About 2,500 application PSB’s

About1600 Application JCL & PROC members

to change.

DB2 V10 - No Upgrade to DB2 V11 during DL/2

Migration ( Simplify the project )

CICS 4.1 & CICS 4.2

15

Additional Constraints Present

in Our Environment

IMS Data Sharing.

Not a constraint, simply a complexity.

We had Replication from IMS to DB2 running

since 2001

Large number of applications as readers of the

DB2 targets of replication.

Apps had to “facilitate” a delay

There was overhead in replication: IMS User

Exit, Agents, Repositories, Additional Logging

Tasks, Post Log Merge Tasks, Apply Tasks

16

Convert Replication Targets into

Views on DL/2’s DB2 Tables

Replication was one segment to one table.

At some point the light bulb came on and we

realized the targets could become views.

We were not replicating every segment or even

every field ( column ).

Some “occurs” in segments that were replicated

1 segment occurrence to 15 DB2 rows.

Occurs rows are done through a view with 14

“UNION ALL” references to same table.

Must use the key, but predicate pushes into

query and it performs great.

17

Risk / Reward Tradeoff

Risks

Product won’t work ( Proof of Concept )

Implementation will take forever. ( 1 Year )

The Organizational “Naysayers”

Craziest Move: Our biggest Naysayer was assigned

as the Application Lead.

Rewards

Remove IMS from our inventory ( MLC )

Remove additional IMS Tooling ( $$$ )

Remove replication – DBA Stress Level Reduction

Simplification ( Three Major Components to One )

Position our data in a familiar relational model.

18

Syncsort Defined Phases of a

DL/2 Migration Project

PHASE 1: IMS analysis and migration unit definition

PHASE 2: Define migration and test plans

PHASE 3: Map IMS databases and secondary

indexes

PHASE 4: Migrate IMS data to DB2

PHASE 5: Application testing

PHASE 6: Production cutover

19

DL/2 – Project Preface & Phase 1

“Short Period of Contemplation”

December 2004 until March 2013

April 2013: IMS analysis – Done on Syncsort site

FTP them DBDLIB/PSBLIB

FTP them COBOL Copybooks for Segments

FTP code for all exit routines ( Sparse Indexes )

April 2013: Defined POC databases

20

Definition – Mapping

21

Definition – Migration Options

Option 1 – Big chunks of CHAR with key columns.

Option 2 – Most fields made CHAR columns.

Option 3 – Most fields made appropriate data type.

Data cleansing is likely.

Option 4 – Similar to Option 3, but includes mapping

segment types to different tables.

NOTE: It is possible to have any combination of these

options for your various databases to be converted.

Decision on which option to use determines the “ease

of use” later on.

22

DL/2 – Project Phase 2

July 2013: Definition of Migration plan

Opted for 2 groups = 2 “steps” in each environment

Second group would also include IMS Elimination.

Two passes is similar project plan to DB2 migration

plan with CM and NFM steps.

About 50% of data ( Not an even split on databases )

Couldn’t consider all at once, and didn’t want small

groups or individual database steps to drag on

forever.

Test plans were “business as usual” in Dev/Sys/QA

Thorough test as part of DR Test.

23

DL/2 – Project Phase 3

July 2013 – Syncsort prepares mapping for POC

databases and testing migration.

Lots of “Data Quality” questions

Lots of data analysis discoveries

Effects on “Migration Option Decisions”

August 2013 – On-Site POC

Several DBA’s now involved.

Minimal work by CICS system programmer.

Application specialist with extensive

IMS knowledge

A week of long days as we learned DL/2.

POC Complete – Green light for the project !

24

DL/2 – Start Your Engines

Divide and Conquer

Re-link all programs to get DL/2 Stub included.

Change SCLM Build to link DL/2 Stub.

JCL Changes for BMP JCL and Procs.

Edit macro for JCL changes: about 1,600 members.

Progress Tracked Weekly.

CICS Programs Installed across the board.

DB2 Plans Changed.

PSB’s become “driver” with Static SQL in a Package.

Plans needed collid of DL/2 packages in the packlist.

Create new Bufferpools ( GBP ) for DL/2 Tablespaces

and Indexes. ( DFSVSAMP )

25

DL/2 – Project Phase 4, 5 & 6

Syncsort delivers mapping.

Generate drivers.

Any issues in testing may be mapping related.

Test migration ( IMS Unload / DB2 Load ) process.

Convert Development, System Test & Quality.

Head to Production.

Quality mimicked production when production cutover

occurred. Did this to facilitate problem recreation.

Syncsort on site during both Production Cutovers.

Excellent support from Syncsort project players.

26

Group Cutover Timeline

Group 1 Migration

January 29, 2014 – Test / Development

March 9, 2014 – System Test

April 30, 2014 – Quality Assurance

September 7, 2014 – Production

Group 2 & IMS Elimination Mode Migration

July 2, 2014 – Test / Development

July 16, 2014 – System Test

September 17, 2014 – Quality Assurance

October 5, 2014 – Production

27

Special Processing and Issues

Data Warehouse – HD Unload processing became

DSNTIAUL Unloads.

Issue with IMS code having “SQL COMMIT” rather

that CHKP.

Ignored by IMS

Issue in IMS Elimination mode as it closes DL/2 Cursors

Unconverted developer JCL – U3303 abends.

Alignment Errors on PCB’s

DB Call against I/O PCB

Code had been wrong for years.

28

Issues continued…

2 or 3 Programs with IMS call pattern not conducive to

performance.

Didn’t perform great in IMS

Performed worse under DL/2

Added relational call to replace IMS calls.

New diagnostic processes for developers.

Abend-AID, Dumpmaster

Xpeditor setup.

Maintained all doc for project on Wiki, including how-to’s

for developers.

29

Reaping the rewards…

Removed IMS from our inventory ( MLC )

Removed additional IMS Tooling ( $$$ )

Removed replication – DBA Stress Level Reduction

Removed the workload of maintaining those products.

Simplification ( Three Major Components to One )

Positioned our data in a familiar relational model.

30

Thanks for Attending!

That concludes my portion of the presentation.

Syncsort has some additional information to share.

Following that, there will be a time for questions.

I will gladly answer any questions related to our

implementation project.

Technical questions about the product can be

answered by Syncsort.

Thanks again for attending.

Typical Migration Project

Syncsort Confidential and Proprietary - do not copy or distribute31

• Databases grouped by application/size/complexity for optimal testing

• Primarily 3 main project phases:

Database Group 1

Database Group 2

Database Group 3

TIME

Database Group n

Eliminate IMS

Database mapping usually performed by Syncsort

Map/migrate Application testing Production cutover

Syncsort Migration Services


32

• Free PSB and DBD analysis

• Proof of concept

• Mapping and migration services

• Onsite support for production cutover

Why the Transparency Approach?


33

• NO application program changes are required

• LOWEST RISK migration strategy – bite-sized chunks

• REDUCED COSTS through elimination of IMS DB and BMC tools licenses

• SIMPLIFICATION of DBA support through DBMS convergence

• CHOICE of migration methods to meet differing requirements

• Simple, rapid method to reduce costs very quickly

• More measured method to meet DB2 design objectives

• A combination of both

• Simple method first, measured method later

• MINOR JCL changes (which can be made in advance)

DL/2 Transparent Data Migration


34

• Eliminate IMS DB and IMS tools License costs with NO changes to

application code

• Support Legacy Modernization initiatives

• Reduce Risk as IMS DBA and Programming skills head for retirement

35

Questions and Answers

ims to db2 migration: how a fortune 500 company made the move in record time with minimal disruption

Technology