luncheon webinar series july 1, 2010 - datastagedsxchange.net/uploads/2010julydsx-metadex.pdf ·...

© 2010 IBM Corporation

Information Management

1

Luncheon Webinar SeriesJuly 1, 2010

MetaData Workbench on Steroids - MetaDexTM - Unparalleled Cross Platform Access

Sponsored By:




Questions and suggestions regarding presentation topics? - send to [email protected]

Downloading the presentation– http://www.dsxchange.net/MetaDex.html – Replay will be available within one day with email with details

Pricing and configuration - send to [email protected]

Bonus Offer – Free premium membership for your DataStage Management! Submit your management’s email address and we will offer him access on your behalf.

– Email [email protected] subject line “Managers special”.– Join us all at Linkedin http://tinyurl.com/DSXmembers

Visit Ernie Ostic’s Blog: http://www.dsrealtime.com/

2

mailto:[email protected]�



http://tinyurl.com/DSXmembers�

http://www.dsrealtime.com/�



33

Challenges of Every Data Integration Project

Lack of communication and collaboration across teams

Inability to find information quickly

Information ambiguity and lack of full understanding

Lack of confidence and trust in information

Poor data governance



44

Customer Pain Points

FRONT LINE APPLICATIONS

OLAP

DATA INTEGRATION / DATA QUALITY / ETL

SOURCE SYSTEMS, DATA MARTS, SILOS

DATA WAREHOUSE

“If often takes hours even days to find the information I need to

do my work.”

“Where is my at risk data? Can I audit the sources of

information?”

“We have a lot of information,

but I don’t always know which is the

trusted data to use to make my

decision.”

”We often struggle to understand the

business needs, as IT isn’t using the

same definitions as the business.”

“We are overloaded with requests from the

business on what information means.”

Warehouse Manager

Subject Matter Expert

Business Analyst

Data Steward

Project Executive

Data Analyst

“Project costs always exceed targets due to

the lack of proper communications.”



55

Metadata Requirements in Data Integration Projects

Metadata RequirementsBusiness Needs

Trust in information

Better understanding of information

Easy search and find-ability

Better data governance

Business and IT collaboration

End to end data lineage

Business context and meaning for IT assets

Catalog of information assets

Risk data analysis and dependency management

Shared metadata repository for Business and Technical users



6

Data Integration Landscape

FRONT LINE APPLICATIONS

OLAP

DATA INTEGRATION / DATA QUALITY / ETL

DATA WAREHOUSE

BI Tools

(e.g. Cognos, Business

Objects, etc.)

ETL Tools

(e.g. DataStage,Informatica,

etc.)

ETL Tools


etc.)

ETL Tools


etc.)

Data Warehouse Platforms

(e.g. IBM SQW,Teradata Bteq/Mload,Oracle PL/SQL, etc.)

SOURCE SYSTEMS, DATA MARTS, MASTER DATA

End to end data lineage

Business context and meaning for IT assets

Catalog of information assets

Risk data analysis and dependency management

Shared metadata repository for Business and Technical users



7

Common Metadata in Data Integration Projects

Category: Costs

Term: Tax Expense

Full Name: Tax to be paid on Gross Income

Status: Current

Data Steward: Roger West

Related Term: Tax Liability

Source: tbl_IncomeExp (Oracle Table)

Expression: SUM(TAXAMT)

Data Rule: TAXEXP > $1

Business Term: Tax Expense

CommonMetadata

Repository

Collaborationbetween Business & Technical Teams

Business Metadata Technical Metadata

•Business terminology•Business rules definitions•KPI and metric definitions•Link to supporting technical assets •Business lineage

•Source and target data definitions•BI model and report definitions•ETL jobs, stored procedures•Related terms•Data lineage

Business Glossary Metadata Workbench



88

Empower the “business user” to make critical decisions with trust and confidence in information

Understand quickly where information originates

Reduce cycles spent across teams validating data accuracy

Includes only the critical path of information which the business user cares about.

Simple to understand and

use with notraining required!

Business Lineage – Trusted Sources of Information



99

Starting point for designing information governance projects

A “Business Users” gateway into Heterogeneous IT environmentData Sources ETL JobsStored ProceduresApplicationsBI ReportsBusiness ProcessesData ModelsWeb ServicesMore…

Business Context for Every Heterogeneous IT Asset



1010 10

Avoid system disruptions

Provide audit information for data governance

Build trust with LOB users

Understand the impact of making changes to information environment

Visualize and trace information flows across enterprise landscape

Access and report on operational metadata

Requirements

Benefits

Metadata Workbench

Support information governance with traceability on data movement,

modeling & BI applications

InfoSphere Metadata Workbench



1111

Catalog and Report on All Metadata Relationships

Business RulesQuality RulesBusiness Terms and DefinitionsData StewardsRuntime and Operational Statistics

Source and Target SystemsTable and Fields, Legacy FilesBI Reports, Marts, & Warehouse CubesData Models, Logical ModelsETL Jobs, FTP Processes, Scripts

Window into Your Data Integration World



1212

Extended visibility to Heterogeneous Data Integration Processes – including Information Server and 3rd Party

Popular business use cases 3rd party ETL tools and applications Mainframe COBOL programs External scripts, Java programs, or web services Stored procedures Custom transformations

Information Lineage Visibility to Trusted Sources of Information

Technical and Detailed View into All Data

Transformations



1313

Multi-Vendor Metadata Integration

Metadata Workbench

Business Glossary

BI Tools

(e.g. Cognos, Business

Objects, etc.)

ETL Tools


etc.)

Data WarehousePlatforms

(e.g. IBM,Teradata,

Oracle, etc.)

Data Lineage Extender Feature

for Non-Information Server Tools!!!

Com

mon

Met

adat

a



1414

Compact MetaDexEmbraces and Extends IBM InfoSphere Metadata Workbench

A tool-accelerated services solution

Integrates metadata from diverse data management processes

Uses InfoSphere Metadata Workbench to consolidate heterogeneous metadata into IBM InfoSphere Platform

The bridge to bring a multi-vendor data integration environment into InfoSphere’s single master registry.

BI Tools

(e.g. Cognos , Business

Objects, etc.)

ETL Tools

(e.g. DataStage ,Informatica ,

etc.)


(e.g. IBM,Teradata,

Oracle, etc.)

Metadata Workbench

Business Glossary



1515

Compact MetaDex Solution Vision

BI ToolsETL Tools


Metadata Workbench

Business Glossary



1616

CognosReportField

TeradataBteq Script

Data Lineage – Graphical View

End to end data lineage showing Teradata Script, Informatica, Oracle and Cognos

CognosReportQuery

TeradataEDWTable

InformaticaMapping

OracleData martTable

Enabled by Compact MetaDex



1717

BusinessRule insideTeradataBteq script





1818



BusinessRules insideInformaticaMapping



1919

Data Lineage – Detailed View of Informatica Metadata

Output FieldName

Source FieldName

BusinessRuleApplied

ComponentsModifyingThe Field

MappingName



2020

Business Lineage – Graphical View Business Lineage view within Metadata Workbench (Department sales comes

from Quantity Sold and Item Price fields via a set of Teradata and Oracle tables)

CognosReport Field

OracleDatamartTablesTeradata

EDWTables

Source Fields For TeradataScripts



2121

Enable better data governance Data lineage across multiple BI and ETL applications, running on

different platforms Rich, automated documentation of data flows and processes Powerful audit capability: identify exactly where a measure is calculated

in the ETL chain – which process, what transformation logic

Accelerate business decision making Better understanding of information at hand Trust and confidence in information accuracy and consistency Closer business and IT collaboration

Reduce costs of data integration processes Identify redundancies in processes: different processes updating same

attributes or tables, unused attributes Streamline ETL processes: fewer processes, better data management Rapid and efficient impact analysis reduces development time

Business Benefits Metadata Workbench, Business Glossary, and Compact MetaDex Solution



22 22

Better Results

Respond Quickly to Compliance DemandsUse end-to-end lineage

on Sarbanes-Oxley, Basel II, and other

regulatory mandates.

Increase Revenue OpportunitiesMake confident

decisions. Understand your customers and

market place.

Rescue Runaway Costs

Cut project costs by reducing development

cycles, process redundancies and risks.



Thank you




Questions and suggestions regarding presentation topics? - send to [email protected]

Downloading the presentation– http://www.dsxchange.net/MetaDex.html – Replay will be available within one day with email with details

Pricing and configuration - send to [email protected]

Bonus Offer – Free premium membership for your DataStage Management! Submit your management’s email address and we will offer him access on your behalf.

– Email [email protected] subject line “Managers special”.– Join us all at Linkedin http://tinyurl.com/DSXmembers

Visit Ernie Ostic’s Blog: http://www.dsrealtime.com/

24




http://tinyurl.com/DSXmembers�

http://www.dsrealtime.com/�

luncheon webinar series july 1, 2010 - datastagedsxchange.net/uploads/2010julydsx-metadex.pdf ·...

Documents