data conversions - scgmisdata conversions the good, bad and ugly. ... tools for data conversion...

DATA

CONVERSIONS

The Good, Bad

and UglyCarolyn Yon, PMI-ACP

Richland County Government

DATA

CONVERSIONS

The Good, Bad

and Ugly

Lazlo – Real Genius (1985)

Practices for

Successful Data

Conversions

Complex

and

Difficult

Difficult

to

Estimate • Time

• Budget

Expect

Perfection

90-95%

Other Considerations

• Delivery Methods

• XP, Lean

• Scrumban

• Team Composition

• IT

• Users/Stakeholders

• Issue Tracking

• Information Radiation

Data to Convert

Reference Tables System/Setup/Configuration

Entity / Child Data to LoadNeeds little/no pre-processing before loading

Entity / Child Data to ConvertNeeds re-processing/validation before loading

Data Mapping Workbook

Target Table

Target Field

Source Table

Source Field

Notes, Formulas

Tools for Data Conversion

Reference Tables SQL Server Integration Services (SSIS)

Entity / Child Data to Load SQL Server Integration Services (SSIS)

Entity / Child Data to ConvertMicrosoft Visual Studio .Net

UniSys(Legacy)

Interim SQL Server

Conversion DB

Assess SQL

InterimConversion

SQL

New CAMASQL

Flat Files(Weekly)

SSIS(Weekly)

Stored Procedures

(On Demand)

Internet Apps

Conversion or SSIS

Tools for Issue Tracking

Tools for Issue Tracking

Status

• Central Body of Knowledge• Fast Import wherever possible• Enforced Data Integrity• Increased Application Integrity• Facilitate product growth

Data Documentation & Tools

• Manual documentation• Time to learn, architect SSIS

Data Documentation & Tools

• Time Consuming Administrivia• Manual SSIS Runs

Data Documentation & Tools

SQL Server Agent

SSIS Packages

Entity Pkgs

Parcel DEV Job

SQL Server Agent

Delivery Time

Environment

Scripts

SQL Server Agent

Delivery Time

Delivery Cycles

Delivery Cycles

Conversion

Effort

Demo to

Users

Evaluate

Feedback

Entity & Issue

Analysis

Retro &

Planning

• Involved user’s in selecting data subset;

• Early detection on accuracy of data;

• Early connection with users;

• Early considerations of business process changes;

• Identification of desired future enhancements;

• Early scheduling helped to ensure user’s presence;

• The further along we got, users began to see some of the benefits of the new system.

Delivering

• Inconsistent business process knowledge;

• Progress of user acceptance testing;

• Live legacy system is a moving target;

• User’s lack of understanding of the new system;

• User’s lack of experience with conversions;

• Dependencies among the entities.

Delivering

• Inconsistent data integrity;

• Users reluctant buy-in to new system;

• Users reluctant to consider business process changes;

• Changes in conversion inadvertently broke something else;

• Integration testing laborious;

• Features not present/ not identified back during development.

Delivering

SUMMARY

Data Conversion

is only going to be as easy

as the quality of the data.

SUMMARY

People Collaboration

can be just as complex and

difficult.

Questions?

Carolyn Yon, PMI-ACP

Richland County Government

cell: 803.309.0353

yonc@rcgov.us