governance and architecture in data integration

www.analytixds.com

Governance and Architecture in Data Integration An Agile approach to Governance & Regulatory Compliance

WHITE PAPER

www.analytixds.com

Table of Contents

What is Data Governance?..................................................................................... ................... 3

Needs........................................................................................................................................... 3

Benefits of a data governance program...................................................................................... 6

How to start up a data governance program in an agile way?.................................................... 6

AnalytiX™ Mapping Manager™.....................................................................................................7

Addressing the Metadata Management Gap............................................................................... 9

AnalytiX™ Mapping Manager™ – a brief introduction to the best mapping management......... .9

Benefits of jump-starting a Data Governance program with AnalytiX™ Mapping Manager™ ......9

www.analytixds.com

What is Data Governance? Data Governance is said to be the organization and implementation of policies, procedures,

structures, roles, and responsibilities which outline and enforce rules of engagement, decision rights,

and accountabilities for the effective management of information assets.

While this may be perfectly appropriate from an academic standpoint, it is still hard for people in the

trenches to define to their audience what the real meaning of Data Governance is. When people talk

about governance, they may be talking about:

Organizational bodies that govern data and information,

Rules (business rules, data quality rules, etc.),

Decision rights (how we "decide how to decide")

Roles and responsibilities, or

Monitoring, controls, and other data and information enforcement methods.

So, which one is it? The answer to this question is relatively easy: yes – It is all of the above.

Data Governance (DG) becomes critical when the need to support a major data integration effort

arises, as data governance refers to the overall management of the availability, usability, integrity, and

security of the data and information employed in an enterprise.

These efforts typically include the Enterprise opts for the deployment of an Enterprise Data

Warehouse (EDW), a Master Data Management (MDM) program or the roll out of new Enterprise

Resource Planning (ERP) application.

A sound data governance program includes a governing body or council, a defined set of procedures,

and a plan to execute those procedures. In practical terms, this means putting personnel, policies,

procedures, and organizational structures in place to make data accurate, consistent, secure, and

available to accomplish the integration mission. It takes on special importance of the legal

requirements the Enterprise must fulfill.

Effective data governance makes the Enterprise more efficient by saving money, allowing reuse of

data, fulfilling the legal and compliance requirements of the Enterprise and by supporting Enterprise

Analytics.

www.analytixds.com

www.analytixds.com

Needs The primary mission of a Data Governance Program is to enable various strategic initiatives,

satisfying the needs of the Enterprise. These initiatives can usually be characterized by three major

areas:

1) Data Integration, Data Warehousing and Analytics

While every data warehousing program or initiative is based upon the premise of providing the end user (the ultimate consumer of data in that initiative) with a single version of the truth, how many programs can actually say they have a single interpretation of customer, or product, or hierarchy; or can, for sure, say what the lineage of a report data element is?

While data integration initiatives typically instantiate additional projects that converge upon the same point: the data -its availability (i.e. Data profiling) and fitness for use (i.e. data and information quality management) by either analysis or reporting tools; and, by definition, a data warehouse does not create new data: it only combines and repackages data created elsewhere for analysis and reporting. The ability to manage information about the data is critical to the success of every data integration or data warehouse initiative.

2) Enterprise (Application) Integration

Enterprise Application Integration (EAI) is defined as the use of software and computer systems architectural principles to integrate a set of enterprise computer applications. Enterprise Application Integration (EAI) is basically an integration framework comprised of a collection of technologies and services which form a middleware (applications that connects

software components and applications) to enable integration of systems and applications across the Enterprise.

Supply Chain Management (SCM) applications for managing inventory and shipping, Customer Relationship Management (CRM) applications for managing current and potential customers, Business Intelligence (BI) applications for identifying patterns from existing data from operations, and other types of applications (for managing data such as human resources data, health care, internal communications, etc) typically cannot communicate with one another in order to share data

or business rules. For this reason, such applications are sometimes referred to as islands of automation or information silos.

This lack of communication leads to inefficiencies, wherein identical data are stored in multiple locations, or straightforward processes are unable to be automated. The ability to identify, classify, match and publish the information contained within these islands of information is critical to the success of any enterprise application integration initiative.

www.analytixds.com

3) Compliance

There are multiple compliance requirements that must be satisfied by all organizations. These take the form of various laws governing the aspect of most organizations. They vary from information privacy laws that cover the protection of information on private individuals from intentional or unintentional disclosure or misuse (such as HIPAA, the Health Insurance Portability and Accountability Act of 1996) to accuracy and transparency laws that govern financial institutions and publicly traded companies (such as the Sarbanes-Oxley Act of 2002, and Basel II and III).

Take a publicly traded company for example – disclosure and transparency requirements bring new challenges that are clearly outlined in Sarbanes-Oxley’s Mandatory regulatory controls:

Sections 302, 906

Section 404

Section 409

The Sarbanes-Oxley act introduced major changes to the regulation of corporate governance and financial practice and affects corporate financial reporting in a variety of ways. Let’s take a quick look at some of the impact from these requirements into various areas of the data environment: Meeting the data demands presented by the Sarbanes-Oxley act of 2002, is a challenge that many

302,906

Data Integration

Im pa ct on Da ta en vir on m en t

• Convert currencies, Reporting periods • Consolidate chart of accounts • Track record adjustments • Track Off-balance transactions • Keep historical data available for audit

• Reconcile G/L & transactional systems that feed G/L • Measure and improve accuracy, completeness, and understanding of G/L data • Identify and eliminate duplicates and all data inconsistencies

• Document data mappings, aggregations and transformation rules

Section

404

• Integrate internal transactional systems logs and audit trails with external market and analytical data • Provide a search mechanism across structured and unstructured data

• Measure and improve Metadata quality (data definitions, specifications and metadata standards) • Ensure data security • Measure and improve data accessibility and ease of use • Check customer and vendor base against watch lists

• Define, document and create a repository of business terms, rules, data elements, KPI’s and metrics • Establish data and information stewardship • Establish roles for data access and manipulation • Establish approval hierarchies

409

• Integrate information across systems to monitor exceptions or process failures

Data Quality • Measure & improve data availability, timeliness and relevance

Metadata Management

• Maintain repository of triggering events • Define and check against thresholds and limits

www.analytixds.com

Institutions have struggled with because their data environments were not properly built to be flexible and responsive to change.

Benefits of a data governance program The success of implementing a data governance program and the associated structure around the data governance program is critically dependent upon a) The organizational structure of the data governance board, b) The policies defined by the data governance board, c) The tools and processes used to operationalize the decisions of the data governance board, and d) The techniques and methods supporting the data governance program. Companies that focus on the data and work forward to the reporting and analytic layers of the data integration architecture are far more likely to succeed in terms of reduced implementation costs and in exceeding the expectations of users and management as well as being compliant with regulations. It would appear to the casual observer, that the key requirements for success in data governance are relatively clear:

The Enterprise must have the ability to manage information about the data in every data integration or data warehouse initiative,

The Enterprise must have the ability to identify, classify, match and publish the information contained about the data and its data processes to all users,

The Enterprise needs to be able to support the demands made by regulators, legislators and new business opportunities. Although this all seems relatively easy and straight-forward, how do you achieve success in data

governance in a non-intrusive and agile way?

How to start up a data governance program in an agile way? As we mentioned before, the success of any data governance program and of the associated structure around the data governance program is critically dependent upon, among other things, the tools and processes used to operationalize the decisions of the data governance board. Perhaps the best and easiest way to start a data governance program and achieve quick return on the Data Governance Program investment is to implement a data-focused Data Governance Group. For this purpose we will use the data requirements to satisfy sections 302, 906, 404 and 409 of the Sarbanes-Oxley act of 2002.

www.analytixds.com

You have requirements that point to a well defined set of data requirements related to data:

1. The need to provide data traceability and auditability

2. The need to measure and improve data and metadata quality (data definitions, specifications and metadata standards)

3. The need to document and publish data mappings, aggregations and transformation rules

For that you will need a set of reference models. Take the Data Governance model used by AnalytiX Data Services for instance, on where you have, central to the data governance initiative, a formal data and information integration architecture comprised of an architecture model that deals with all domains of data and information integration architecture. Perhaps the easiest way to understand this model is to understand what the focus of a data governance program should be: data and meeting the various requirements centered around data: On this initial white paper we will focus around the use of AnalytiX™ Mapping Manager™ as an enabler for the data governance program.

AnalytiX™ Mapping Manager™ is much more than what the name implies: a data mapping tool or a mapping management tool. AnalytiX™ Mapping Manager™ is an enterprise-level source to target mapping tool that allows the management of all metadata related to sources, targets and business rules associated with the data and information reporting needs of the enterprise as well as a centralized metadata repository capable of providing even the most demanding organization with answers to just about every compliance need imaginable.

AnalytiX™ Mapping Manager™ What is AnalytiX™ Mapping Manager™ and how can it help the fledging Data Governance program? By bringing standards, control, auditability, traceability and versioning across all integration projects at the enterprise level.

www.analytixds.com

We will use the Sarbanes-Oxley example above to outline how AnalytiX™ Mapping Manager™ can jump start a robust and flexible Data Governance program that satisfies the act’s regulatory controls requirements. In this initial White Paper we will focus on metadata management and information tracking. The Sarbanes-Oxley act defines that, in order to be compliant, a reporting organization must

Document data mappings, aggregations and transformation rules;

Define, document and create a repository of business terms, rules, data elements, KPI’s and

metrics;

Establish data and information stewardship;

Establish roles for data access and manipulation;

Establish well documented approval hierarchies;

Maintain repository of triggering events for changes;

Define and check against thresholds and limits AnalytiX™ Mapping Manager™ can easily provide, by virtue of its architecture, significant improvement over traditional methods that use spreadsheets, documents or decoupled metadata management solutions. AnalytiX™ Mapping Manager™ improves efficiency in processes, people, policy and rights management -not only in the analysis, design and development phases of integration, but also in downstream work flow management and regulatory reporting. Typically, all metadata management tools provide some sort of repository and can import and export data definitions from sources and targets. What they typically lack is a set of well defined processes and the ability to manage change. AnalytiX™ Mapping Manager™ bridges the gap between leveraging metadata, and making the data definition and acquisition processes automated, disciplined, predictable, giving insight into the data lineage to all data enterprise stakeholders via its web enabled portal and by bringing standardization, collaboration, versioning, traceability, impact analysis, management visibility, and programmatic control to what otherwise would be a set of either weak or non-existing processes. One of the primary directives of any Data Governance organization is to implement rigorous, repeatable design and integration processes with the larger overreaching objective of reducing errors and the amount of rework normally associated with integration projects and addressing the gaps normally found in these (metadata, version control, security and configuration management among others). AnalytiX™ Mapping Manager™ provides this discipline and rigor through its dedicated data mapping methodology as well as its metadata management processes and powerful patented mapping technology. AnalytiX™ Mapping Manager™ was designed and developed to not only fill the gap of having the ability to manage and version mapping specifications, but to also streamline and improve current process and drive standards around the entire process and across the enterprise for all integration and governance processes.

www.analytixds.com

Addressing the Metadata Management Gap A Metadata Integration capability provides a basic ability to build metadata flows into and out of a managed metadata environment. Metadata is produced and consumed by a variety of components in the Information Architecture. To achieve consistency, quality and reuse utility, metadata must be integrated between the sources of record for metadata. Metadata Integration provides the foundation for active metadata management in a model driven architecture. Metadata Integration itself should be model driven by interfacing with AnalytiX™ Mapping Manager™ capabilities in the product architecture to provide a framework for repeatable development processes and reusable components to integrate data and metadata seamlessly. AnalytiX™ Mapping Manager™ provided data lineage metadata, allowing full discovery of information between systems, including the operations that are performed upon the data. The need to establish a comprehensive view of data lineage has grown in importance over the past few years, particularly with renewed compliance requirements. The ability to trace lineage of data from producers to consumers is an important feature of the AnalytiX™ Mapping Manager™ architecture.

AnalytiX™ Mapping Manager™ – a brief introduction to the best mapping management AnalytiX™ Mapping Manager™ is 100% metadata-driven and is used to define and drive standards across integration projects within an enterprise, enable data & process audits, improve data quality, streamline downstream workflows, increase productivity (especially over geographically dispersed teams) and give project teams, IT leadership, and management visibility into the 'real' status of integration projects across the enterprise. AnalytiX™ Mapping Manager™ is a complement to existing data integration products and allows the business analyst to dynamically build mapping specifications (via drag and dropping of metadata from the metadata browser) which become clean, approved requirements/ inputs to data integration developers. By using AnalytiX™ Mapping Manager’s fully integrated metadata management capabilities, these specifications are fully governed and are then versionable, trackable, auditable, and repeatable throughout the lifecycle of data integration and Master Data Management (MDM) projects.

www.analytixds.com

Figure: Agile Data Governance Framework

The Benefits of jump-starting a Data Governance program with AnalytiX™ Mapping Manager™: There are varied benefits from using AnalytiX™ Mapping Manager™ as the foundation of any Data Governance program and to facilitate cross-team collaboration and governance, by providing the following:

the ability to run data lineage reports across the enterprise,

the ability to create, maintain and consolidate data dictionaries for all enterprise systems,

the ability to monitor and review enterprise standards for naming conventions and federation of data

the ability to assign data stewards to promote accountability for information quality

the ability to manage and improve data security

the ability to comply and track on compliance with regulatory demands

the ability to manage and promote data consistency and user confidence in data quality

www.analytixds.com

www.analytixds.com

For further information, please contact Analytix Data Services

Corporate HQ

14175 Sullyfield Circle, Suite # 400 Chantilly, VA 20151 USA

Tel: 1+ (800)-656-9860

Email: [email protected]

SALES

For Sales - Please call (800)-603-4790 (9am-5pm EST) or Email: [email protected]

SUPPORT

For Product Support - Please call (800)-617-9620 or Email: [email protected]

http://www.analytixds.com/

mailto:[email protected]



governance and architecture in data integration

Technology