db2 information integrator classic federation (db2iicf ... · pdf filedb2 information...
TRANSCRIPT
Lightyear Consulting Ltd
Steve Hunn, Brian Lee
BusinessPartner
L I G H T Y E A R
DB2 Information Integrator Classic Federation (DB2IICF) Overview
Feb. 2004
Steve HunnDirector of [email protected]
Brian LeeSenior Consultant
Lightyear Consulting Ltd.Palo Alto, Calgary, Scottsdale, Laguna Beach, Austin, Chicago
Tel: 1-800-989-6060www.lightyr.com
IBM Software Group | DB2 Information Management Software
2
How do you leverage your mainframe data assets?
Enterprise Environment
The Typical Large IT Data Integration EnvironmentDecades of heterogeneous technology investment
60% of data resides on mainframe and is growing 20% per year
High performance and scalability is mandatory
Legacy programming is becoming very costly
Lightyear Consulting Ltd
Steve Hunn, Brian Lee
IBM Software Group | DB2 Information Management Software
3
Options for modernizing legacy data
Manual rewriting and migrationDisruptive, risky (high)Requires extensive investment
Automatic (semi-automatic) conversion and migrationUtilizing tools – DL/2, ManTech, …Risky (medium to low)Requires fair amount of investment
Wrapper technologyCICS Transaction Gateway, IMS Connector, Information Integrator,…Non-disruptive, no riskNot an expensive solutionQuick implementation, Quick return on investment
IBM Software Group | DB2 Information Management Software
4
Positioning of Information Integrator Classic Federation
IICF
MS SQL
InformationIntegrator
WAS
z/OS, OS/390
NT/UNIXNT/UNIX
NT/UNIX
DB2
IMS
DATACOM
IDMS
ADABAS
VSAM
R/W
R/W
R/W
R/W
R/W
R/W
Oracle
Wrapper
ODBCWrapper
JDBC ODBC
Crystal Report,MS Excel, …
Crystal Report,MS Excel, …
WAS,Custom Appl.
WAS z/OS
Lightyear Consulting Ltd
Steve Hunn, Brian Lee
IBM Software Group | DB2 Information Management Software
5
Operational Processing Flow
DataData Meta DataCatalog
Meta DataCatalog
IICF Client(ODBC, JDBC)
IICF Client(ODBC, JDBC)
DB2 II Classic Federation
Server
DB2 II Classic Federation
Server
Data Management
System
Data Management
System
SQL Query Result RowsNative Dialect
Native Result
IICF is a thin layer of software translates a query into the native language and materializes the result set based on the meta data definition.
IICF is a thin layer of software thattranslates a query into the native language and materializes the result set based on the meta data definition.
IBM Software Group | DB2 Information Management Software
6
DB2 II CF Product Suite
CF Data ServerCommon server infrastructure that supports all CF multi-threaded, multi-user, services.
CF RDBMS ServicesFederated RDBMS whose data stores are non-relational database systems
CF Transaction ServicesIMS and CICS Business Rules Reuse
Lightyear Consulting Ltd
Steve Hunn, Brian Lee
IBM Software Group | DB2 Information Management Software
7
CF Data Server
Application hosting using named service architectureMemory Management
Master Pool and Sub-PoolsDynamic Growth
Task ManagementPre-started, Dynamically startedManaged termination and notification
Communications ManagementCommon APITCP/IP, APPC, Cross Memory, MQ-Series, Local Queues, etc.
File ManagementFlat, btree, avl tree, etc.
Platform specificHiperspace, DataSpace, etc.
Common Miscellaneous support routines
IBM Software Group | DB2 Information Management Software
8
CF Data Server
Data Savants
Data Savants
Data Savants
Data Savants
Transaction Savant
Transaction Savant
TransactionSavant
TransactionSavant
MetaData
Catalog
DataMapperDataMapperTCP/IP
IMSDB2 VSAMIDMS IMS/TMCICS
Architecture
Query Processor
JDBC,JCA,ODBC,CLIJDBC,JCA,ODBC,CLI
LinuxUSS
AIXSolaris
HP-UX
95/98/NT2000/XP
Applications& Tools
0S/390, z/OS
Data Savants
Data Savants
Data Savants
Data Savants
Lightyear Consulting Ltd
Steve Hunn, Brian Lee
IBM Software Group | DB2 Information Management Software
9
CF RDBMS Services (XDi)
SQL 92 / DB2 Dialect
Level-ized “savants” access legacy data sources
Advanced Query Processor engine aggregates multiple data sources
Read and Write support including 1PC and 2PC
Standards-based clients, ODBC, JDBC, and CLI
Metadata driven
IBM Software Group | DB2 Information Management Software
10
SQL Support, Views & Joins
SQL 92 supportSELECT/INSERT/UPDATE/DELETE all supported
Complex tables can span segments, records, etc.JOIN support for heterogeneous multi-table access
Embedded JOIN optimization
Multi-threaded Data Savants for maximum throughputComplexity in the Meta data simplifies development
Legacy data types automatically translated
Meta data defined "where" filtering through Views
Multiple automated ways to normalize recurring data
Full support for catalog queries
Optional DB2-like security
Lightyear Consulting Ltd
Steve Hunn, Brian Lee
IBM Software Group | DB2 Information Management Software
11
Reporting, Auditing, Errors
Standard SQL error handlingSQL error and response codes returned as part of result
Embedded SAF exit supportSAF exit for charge-back and other accounting purposes
Leverages existing mainframe accounting practicesEmbedded SMF accounting
Type 99 recordsTracing and Logging by level
Remote and local monitoring
Triggered tracing
Post-Mortem tracing
IBM Software Group | DB2 Information Management Software
12
PowerConsistent SQL interface … Instant integration
Connects mainframe data with any eBusiness platform
Insulates application from proprietary back-end data
Non-intrusive – NO impact on mainframe transactionsPowerful data access … non-relational & RDBMS
Direct SELECT-INSERT-UPDATE-DELETE
Distributed join, data normalization, view-driven filteringEase-of-Use
No mainframe programming
Automated meta data configurationIntegrates with your mainframe environment
RACF - ACF2 - TopSecret
Manage and monitor with existing mainframe utilities
Security and accounting exits for site customization
Lightyear Consulting Ltd
Steve Hunn, Brian Lee
IBM Software Group | DB2 Information Management Software
13
Manage & Monitor
GovernorsLimit resource utilization
Set at Installation, Server and/or User levels
SecurityInterface for RACF, ACF and TopSecret
Optional embedded DB2-like security
Operator ConsoleMonitor usage
Manage users
IBM Software Group | DB2 Information Management Software
14
Components
Connectors JDBC, JCA , ODBC and CLI (call level interface)
ServerCore mainframe-based SQL engine
Data Savants Database specific read/write access engines
DataMapperConfiguration and administration
Lightyear Consulting Ltd
Steve Hunn, Brian Lee
IBM Software Group | DB2 Information Management Software
15
DataMapper
Non-RelationalDataMapper
(Windows application)
MS-AccessRepository
COBOLCopybooks
DBDSource
USEGrammar
IDMSSchema/Subschema
Data Catalog Types•DataCom•IDMS•IMS•Sequential•VSAM
Objects not supported by DataMapper•ADABAS Tables•DB2 Tables•Views•Stored Procedure definitions•Security
DataMapper supports redefinitions and occurrences
IBM Software Group | DB2 Information Management Software
16
DataMapper – sample screenshot
Lightyear Consulting Ltd
Steve Hunn, Brian Lee
IBM Software Group | DB2 Information Management Software
17
VSAM access overview
X Direct (a.k.a batch) access - limited function, slow, not scalable.- no configuration required - systems side folks usually not involved.
CICS/VSAM - full-function, fast, scalable, recommended- configuration required - MVS, CICS & VTAM systems programmers needed.
Import Copybook and generate the use of grammar syntax
IBM Software Group | DB2 Information Management Software
18
CICS/VSAM access
VSAMCUSTOMER
File
CICS
EXV1
transactionEXC1/EXS1
CICSCICSDAPPL
LocalCACCICS1
APPL
VSAMSavant
QueryProcessor SQL Request
Lightyear Consulting Ltd
Steve Hunn, Brian Lee
IBM Software Group | DB2 Information Management Software
19
IMS access overviewII CF supports three methods to access/update IMS data:
DBB/BMP InterfaceDRA Interface: RecommendedODBA Interface: Supports 2PC (RRS)
Data access will be converted into the native DL/I call
Supports all IMS database types except GSAM, PHDAM, PHIDAM, PSINDEX, MSDB
Supports mapping of physical, logical and SIX
IMS mappingImport DBD and copybook into the DataMapper and generate the use of grammar syntaxStarting from the root segment in the physical, logical or secondary data structure hierarchy. This root segment is referred to as the Index Root SegmentEnding at a specified child segment which is referred to as the Leaf SegmentIMS Logical Table can only map a single path in the physical, logical or SIX data structure hierarchy. For example, at least two IMS Logical Tables must be defined for database E4. One IMS Logical Table to map the Root->Child1 path and another to map the Root->Child2 path (See next chart for a diagram)
IBM Software Group | DB2 Information Management Software
20
IMS mapping concepts
Root
Child1 Child2
E4 Example
IMS Logical Table1Index Root=RootLeaf Segment-Child1
Root Child1
IMS Logical Table2Index Root=RootLeaf Segment-Child2
Root Child2
Lightyear Consulting Ltd
Steve Hunn, Brian Lee
IBM Software Group | DB2 Information Management Software
21
ADABAS access
If Predict is availableExtract layout information by using sample utility
Then generate the meta data grammar (USE grammar) from the extracted layout
If Predict is not availableManual creation of the meta data grammar is required
During the installation, it is needed to link a module with the Adabasmodule for the Adabas access
DB2 access
Extract the table layout information by using sample utility
Uses CAF interface to access DB2
IBM Software Group | DB2 Information Management Software
22
CA-DATACOM/DB access
Get the COBOL copybook information by punching out the layout of the data
Download the copybook into the Data Mapper and create the use of grammar syntax
Upload onto the mainframe and generate the system catalog
Database connection will be made through the MUF (multi user facility)
Lightyear Consulting Ltd
Steve Hunn, Brian Lee
IBM Software Group | DB2 Information Management Software
23
CA-IDMS access
Punch and get the Schema and Subschema
Mapping the Schema and Subschema using Data Mapper
Import the copybook into the Data Mapper
IICF supplies batch access module (default – central version mode)
Data access will be converted into the native DML calls
IBM Software Group | DB2 Information Management Software
24
Other objects
VIEWS - Define manually and use the Meta Data Utility to load the definitions into the System Catalog.
Stored Procedure definitions - Define manually and use the Meta Data Utility to load the definitions into the System Catalog.
SQL Security - Define manually and use the Meta Data Utility to perform the grants/revokes against the System Catalog.
Lightyear Consulting Ltd
Steve Hunn, Brian Lee
IBM Software Group | DB2 Information Management Software
25
CF Configuration
DB2 II CFServer
UNIX, Windows, z/OSJVM Platforms
IDMS
IMS
MetaData
Adabas
SQL
ServletSQL
Servlet
ClientClass
SQL DB2 II CFJDBC/JCAConnector
SQL
SQL
R/W DataSavant
DataSavant
R/W
DataSavant
R/W
DataSavant
R/W
DB2 II CFODBC
Connector
Datacom
DataSavant
R/W
z/OSOS/390
DB2
DataSavant
R/W
Portal
BI Tool
EJB
VSAM
SQL
IBM Software Group | DB2 Information Management Software
26
Default business mappings generated• Imports existing physical definitions- Copybooks, IDMS schemas, IMS DBDs, Predict
• Generates logical relational table definitionsVisual customization
• Many-to-Many Model• Exports meta data to mainframe utilities
Meta data driven features• Automatic translation of legacy data types• Meta-Data driven filtering
MetaData
Catalog
MetaData
Catalog
DataMapperDataMapper
IMSDB2 VSAMCA-IDMSArchitecture
Data Integration ServerData Integration ServerSQL Engine
AdabasCADatacom
Data Savants
Data Savants
Data Savants
Data Savants
Data Savants
Data Savants
Powerful SQL access - Performance of Native Databases - Portability of industry standards
eXadas
JDBC/JCA or ODBC Connector
TCP/IP or
MQ Series
LinuxUSSAIX
Solaris HP-UX
95/98/NT2000/XP
Applications& Tools
0S/390, z/OS
Lightyear Consulting Ltd
Steve Hunn, Brian Lee
IBM Software Group | DB2 Information Management Software
27
BenefitsSimplifies Operational Environment
Eliminates redundant copies of mainframe data
Reduces stress on batch processing windows Simplifies Development Environment
Single-platform development
Empowers development using ubiquitous SQL skills
Leverages scarce mainframe database skillsBetter Solution for the Business
Eliminates data latency issues
Reduces errors created by multiple copies of data
Empowers rapid delivery of information rich solutionsSaves Money
Deliver more with fewer and less costly resources
Reduces hardware and software for duplicate data
IBM Software Group | DB2 Information Management Software
28
Speed, Scale & Footprint
Architectural Features Hyperspace-enabled
Multi-threaded at all levels
Reentrant and reusableData Access Features
Leverages native access tools: indexes, keys (e.g. Calc Key)
Embedded query optimizer
Native IDMS support (online or batch) for functionality and speedSample customer benchmark results
Keyed queries from Java. . . .60 Milliseconds
Two table Join . . . . . . . . . . . .80 Milliseconds
Access through USS . . . . . . .40 Milliseconds
Transaction throughput . . . . .500-1,000+ per second
Lightyear Consulting Ltd
Steve Hunn, Brian Lee
IBM Software Group | DB2 Information Management Software
29
Integration in Action – Major US Airline
Provide frequent flyers with up-to-the-minute informationDynamically connect Web site to IMS & VSAM flight data
Eliminate copies of flight data reducing cost, data latency and errors
NT
CFTCP/IP
MVSBoardingPasses
Frequent Flyer
Web Site
Up to
sec
ond
mile
age
cred
it
CF
IMS
Flight Operations
VSAM
MemberMember
MemberMember
IBM Software Group | DB2 Information Management Software
30
Integration in Action – Major US Insurance Carrier
Provide up-to-the-minute policy, claims and accounting informationDynamically connect interactive voice response system to IMS, VSAM & DB2
Connect agent, provider and insured self-service Web sites with live operational data
NT
CF
TCP/IP
MVS
IVRCF
IMS
PolicyManagement
VSAM
AgentAgent
CustomerCustomer
DB2
Claims
Billing
SelfService
WebSites
CF
Solaris DoctorDoctor
AgentsAgents
Lightyear Consulting Ltd
Steve Hunn, Brian Lee
IBM Software Group | DB2 Information Management Software
31
CF and DB2 Information Integrator
BiologicalBiologicalData and Data and
AlgorithmsAlgorithms TextText
DB2 Family
Sybase
Informix
SQL Server
Oracle
• CF extends reach of DB2 Information Integrator with reach to mainframe data sources
TeradataWebSphere MQ ODBC
IBM Extended
Search
Excel WWW, email,…XML
DB2 Information IntegratorSQL, SQL/XML
Federation Engine
Wrappers and functions
Integrated SQLView
DB2II
CF
IBM Software Group | DB2 Information Management Software
32
CF – Transaction Services
DB2 II CF Transaction Services
CICS and IMS through same infrastructure as data access
"Call" mechanism is GA
Web Services mechanism on horizon
Lightyear Consulting Ltd
Steve Hunn, Brian Lee
IBM Software Group | DB2 Information Management Software
33
CF Data Server
Transaction Savant
Transaction Savant
TransactionSavant
TransactionSavant
MetaData
Catalog
DataMapperDataMapper
TCP/IP
IMS/TM CICS
CF T.S. Architecture
JDBC/JCA or ODBC ConnectorJDBC/JCA or ODBC Connector
LinuxUSS
AIXSolaris
HP-UX
95/98/NT2000/XP
Applications& Tools
0S/390, z/OS
Query Processor
IBM Software Group | DB2 Information Management Software
34
Ordering information
OTC (One time charge) product under the category of DB2 ToolsPrice based upon Value Unit (VU)Base product covers IBM data sources only
DB2, IMS, VSAMThree optional features for the other data sources
DB2 Information Integrator Connector for Software AG Adabas
DB2 Information integrator Connector for CA-Datacom
DB2 Information Integrator Connector for CA-IDMSMinimum release level requirements
IDMS V14IDMSDB2 V4DB2
Datacom V9DatacomIMS V3IMS
Adabas level 5.3AdabasOS/390 V2R6Base System
Min. Rel. levelComponentMin. Rel. levelComponent
Lightyear Consulting Ltd
Steve Hunn, Brian Lee
IBM Software Group | DB2 Information Management Software
35
Summary
Quick return on investmentEasy to implement – rapidly deliver information-rich business solutions
Based on the wrapper technology – utilizes existing infrastructure
Minimizing dependence on legacy data skills
Cost efficient solution
Small footprint
Can utilize mainframe spare MIPS
Provide SQL interface to legacy data
IBM Software Group | DB2 Information Management Software
36
Legacy data modernization services
ProductInformation Integrator & Information Integrator Classic Federation
CTG, IMS Connector, DB2 Connector
DL/2, VS/2
ManTechServices – Classic
Installation
Database migration, Data replication
Tuning servicesServices – Modern
Java development
Wireless connectivity to legacy system
Rewrite legacy application
BusinessPartner
L I G H T Y E A R
Lightyear Consulting Ltd.Palo Alto, Calgary, Scottsdale, Laguna
Beach, Austin, ChicagoTel: 1-800-989-6060
www.lightyr.com