A S G W H I T E PA P E R

DATA LINEAGE: MAKE NO MISTAKE ABOUT IT!

by Giovanni Butera, CEO and Managing Director, Head of Data Management and Innovation at Nixora Group

A S G W H I T E PA P E R

DATA LINEAGE: Make No Mistake About It!

Data lineage is the ability to trace the flow of data as it passes through diverse

processes from one end of an organisation to the other. It provides an understanding

of how data is transformed and aggregated across the enterprise or in individual lines

of business. In this article, we take a close look at the revolutionary approach of the

Nixora-ASG data lineage solution, and what it actually entails for the banking and

finance industry; the needs it satisfies, how it works, what are its advantages.

INTRODUCTIONWhen we consider the millions upon millions of structured and unstructured data collected every day in any bank

or large financial institution, as well as the number and complexity of digital processes and systems the data

passes through, data lineage is a vital tool within the overall IT landscape for confirming the reliability of the data.

Data lineage is the ability to track and trace the flow of data as it passes through diverse processes and systems

from one end of the organisation to the other. From the data’s origin all the way through to its final destination in

a report, data lineage provides a clear understanding of how data has been transformed and aggregated across

the organisation or in individual lines of business.

Yet such a description is really the tip of the iceberg. Data lineage is more than simply a tool for increasing a bank’s

ability to report more accurately. So let’s now look at the other 90 percent of the iceberg and take a more detailed

appraisal of data lineage -- its drivers, the requirements it satisfies, the benefits, and the distinctive approach to

tracking data lineage taken by Nixora Group using ASG’s Enterprise Data Intelligence Solution.

A S G W H I T E PA P E R

DATA LINEAGE: Make No Mistake About It!

KEY DRIVERSThere are three key interrelated drivers for data lineage: regulatory compliance, operational efficiency, and

business efficiency.

REGULATORY COMPLIANCEThe first driver is about ensuring that reported data

complies with regulatory requirements, wherein two

essential questions need to be answered: Where

did the data come from? and How did the data get

here? Complexity arises not only with the number

of the data items collected and documented, but

also because data can flow from one system to

another, even between processes within the same

system, or can arrive from multiple systems and be

transformed into new data. Data lineage that is well

executed eliminates ambiguity about where the

data originated, when was it collected, who has had

access to it and for what reason.

OPERATIONAL EFFICIENCYCompliance risk is one of the biggest ongoing

concerns for banks. Let’s suppose a bank has an

issue with reported data, it just doesn’t make sense,

and so what the bank needs to do is dedicate

time and labour to building a process for locating

the source of the data issue and remediating the

problem. This could take weeks or months if data

lineage is not in place. Data lineage provides an

effective way to identify any issues that may take

place and affect data during its journey from

origin to final destination. That said, operational

efficiency achieved by tracking data lineage leads

to substantial savings on remediation activities, and

higher focus on business efficiency (our third driver)

for the simple reason that the bank can focus more

on doing business rather than fixing data problems.

BUSINESS EFFICIENCYNow let’s suppose that same bank is offering new

products or services and would like to cross-sell

life insurance or income protection to a particular

demographic of its current account holders. What

the bank needs is to gain insight about which

customers are likely to take life insurance or income

protection. But the bank’s information about its

existing customers is not only contained within

the one system but across several of the bank’s IT

systems. Data lineage is also an effective tool for

bringing together all of the information that can be

known about the bank’s customers in order for it to

respond quickly and competitively as and when new

business opportunities arise.

A S G W H I T E PA P E R

DATA LINEAGE: Make No Mistake About It!

THE CHALLENGEThe question, Where did the data come from?,

is not as simple as it appears. Nixora and ASG

Technologies have identified several challenging

yet essential requirements if data lineage is to be

comprehensive, accurate and up-to-date. These

include:

• Tracking the movement of data in Data

Warehouse, Master Data and Big Data

environments.

• Quickly identifying the source and quality of

data.

• Recognising how data was calculated and

aggregated as it moved from source systems to

target reports and inquiries.

• Understanding data transformations, and

tracing lineage implemented in ETL tools,

PLSQL and other SQL variants, and Java and

.Net environments to provide the complete

“End-To-End” understanding that is vital for

many compliance projects.

• Navigating backward or forwards in data

transformation processes to understand data

lineage and change impact.

• Supporting understanding of lineage as data

travels across multiple lines of business.

• Providing a business context to lineage by

linking it to business assets, data models,

reference data and data quality information.

• Managing discrepancies in lineage to

resolution with issue management and

tracking for data stewards.

DATA LINEAGE BENEFITSBanks or large financial institutions that implement

data lineage can experience several competitive

advantages.

DATA GOVERNANCEData governance is in constant need of information

relating to existing data, such as who works with the

data, how data is shared between different users

or lines of business, and who owns the data and

the systems it resides in. Data lineage provides this

information and effortlessly identifies policies, roles

and responsibilities for specific data.

QUALITY OF DATAData lineage improves data quality in several ways.

It can help identify data incompleteness, eliminate

redundant data flows, prevent exposure of sensitive

data, and easily trace data discrepancies back to their

point of origin and facilitate root-cause analysis.

IMPACT ANALYSISData lineage enables users to explore the

dependencies between IT systems and business

processes, and how data in one part of the IT

environment can have an impact on data and

reports in the ‘downstream’ end of the data flow.

RISK CONTROLData lineage allows resolution of data quality

issues more quickly and effectively, providing risk

calculators with the fuel for better risk reporting

on whether controls are adequate or need

improvement.

REGULATORY COMPLIANCEData lineage provides proof that reports accurately

reflect the data and gives all stakeholders, including

customers and auditors, the assurance that the data

meets certain reporting standards.

IMPROVED ANALYTICSData lineage provides for agile system architecture

and data model that help analysts and managers to

better “slice and dice” data, thereby empowering

them to reduce business risks and exploit

opportunities.

DATA LINEAGE: Make No Mistake About It!

A S G W H I T E PA P E R

ASG’S ENTERPRISE DATA INTELLIGENCE – A MASTERSTROKENow let’s talk about the masterstroke of ASG’s Enterprise Data Intelligence solution for data lineage. ASG

Technologies, an IT provider to many of global Fortune 500 companies, has developed a suite of systems and

platforms that allow different technologies to talk to each other.

The extraordinary characteristic of this solution is that data capture and data lineage maintenance is fully

automated no matter where the data exists: mainframe, applications, database or business intelligence systems.

HOW TO TRACK DATA LINEAGE WITH NIXORA AND ASG DATA INTELLIGENCE SOLUTIONS The core of tracking data lineage with ASG Technology is populating ASG-DI repository in automation. This is

done in three subsequent phases:

1 - The first phase is discovery of applications and this is done slightly differently depending on whether it is a

database, ETL, BI tool, or an application.

The discovery phase’s objective is to identify all technologies involved and the related ASG’s scanner, to populate

the repository. This is not as straightforward as it sounds. IT systems and applications involved may be many,

varied and often do not communicate.

FIGURE 1: The importance of Application discovery and the complexity of the task

Complexity of the Task: Original - High Level Scope (1)

Mainframe Customer Master System

Enterprise CRM

Mainframe Customer

Information System

Other Product Systems Departmental

Warehouse

Enterprise Data Warehouse

Real-time, middleware interfacesReal-time and Batch

middleware and ETL interfaces

Data provide via real-time and daily batch interfaces

Data provide via Batch interfaces

Unknown

Batch InterfacesBatch ETL interfaces

A S G W H I T E PA P E R

DATA LINEAGE: Make No Mistake About It!

Medium Level Scope (2) Detailed Level Scope (3)

ASG’s Enterprise Data Intelligence is a tool-agnostic solution that captures metadata from more than 220

sources. An example of source categories is represented in the figure below.

FIGURE 2: Sample of ASG scanners and interfaces

The range of scanners and interfaces delivers unique flexibility as the solution supports metadata captured and

combined across mainframe platforms, distributed databases, ETL processes and IT applications.

DATA LINEAGE: Make No Mistake About It!

A S G W H I T E PA P E R

2 - The second phase is the reconciliation of the information that has been scanned into ASG’s repositary using

ASG’s reconciliation toolkit. The toolkit, for each area of reconciliation (namely models, RD, data structure,

transformation, BI, and Big Data), can drill down to visualise sub-areas/details of the reconciliation process, as

illustrated in the Figure 3 below.

FIGURE 3: Levels of reconciliation for different type of technologyLevels of reconciliation for Models, Data Structure, Relationship Databases and Transformation, Business

Intelligence and Big Data:

3 – The third phase is to visualise the final results, i.e. visualise data lineage. This can be done with ASG

proprietary reporting solution as well as with any other solution given the full portability of the results.

FIGURE 4: Sample of Data Lineage visualisation using ASG’s solution

DATA LINEAGE: Make No Mistake About It!

A S G W H I T E PA P E R

UNIQUE FEATURES OF NIXORA-ASG APPROACHScanners are employed to capture metadata and data attributes existing across mainframes, platforms,

interfaces and systems. To say that the benefits of this solution for banking and finance industry are revolutionary

is by no means overstating the point. Its unique features are:

• Coverage of more than 220 metadata sources.

• Macro-level and micro-level analysis of data lineage between data stores and within applications.

• Simple configuration and customization of metadata collection.

• Simple scheduling and management of metadata collection processes.

• Real-time monitoring of the scanning process.

• Integration with Issue Management capability to manage the resolution of lineage problems and track the

history of issue resolution.

• Portable deployment of the results in preferred data management enterprise architectures.

SUMMARY-TO-DETAIL VIEWS OF DATA LINEAGEOne of the most significant capabilities provided by the solution is “out-of-the-box” flexibility in visualizing data

lineage. The visualisation provided by the Metadata Repository provides lineage at a “table-to-report” level. It

answers the question, What data stores contribute to a given report? The summarized visualisation provided by the

table level view is ideal for clarifying system flow.

There is often also a requirement to understand the derivation of a particular data element, either to investigate

apparent data issues or to provide verification for compliance purposes.

ZERO GAP LINEAGEThe solution provides zero gap lineage for business traceability for all Critical Data Elements (CDEs), including

policies and rules. Knowing where data came from, how it moves through systems, and how it changes, are the

most critical tasks in any data management project. When the solution can deliver zero gap lineage then there is

absolute confidence in the authenticity of the data supporting business operations and management.

DATA LINEAGE ANYWHEREAnother important capability of this solution is its portable E2E lineage visualisation. This means ASG’s solution

can be utilized for data lineage and still have the results exported into a preferred data management system for

visualisation and reporting.

DATA LINEAGE: Make No Mistake About It!

A S G W H I T E PA P E R

NIXORA AND ASG APPROACH – THE PIÈCE DE RÉSISTANCEIntroducing new technology is usually a major headache for most banks and large corporations. Setting aside

the nail-biting uncertainty that it could be incompatible with an organisation’s existing technology, such an

undertaking requires an enormous investment in time, money and human resources.

Nixora Group is the implementation partner for ASG’s Enterprise Data Intelligence solution in Australia and New

Zealand. What this means essentially is that we reduce costs by deploying qualified local resources, create value

by sharing our knowledge and capabilities, and can offer peace of mind because we offer data lineage as a service

(LaaS).

NIXORA’S DOMAIN EXPERIENCE AND DATA LINEAGE AS A SERVICE (LAAS)

Data lineage as a service is a major benefit because banks or other financial institutions need not make a major

investment in new technology. They need not go through all the checks, approvals and authorizations to make an

investment in new technology. Nixora provides ASG technology as an operating expense for our customers during

the course of the data lineage project.

While on the one hand ASG Technologies is the leader in data intelligence technology, on the other hand Nixora

is the ideal implementation partner because of our extensive banking and finance domain knowledge. We are the

subject matter experts in banking, we are the ones who understand the specific needs of finance, and we are the

ones who will install the technology and provide specialized user and executive training.

The Nixora-ASG data lineage solution reduces costs and creates value while at the same time enhances

operational efficiency, minimizes risks, increases trust, improves business insights and gives a competitive edge.

Make no mistake about it!

Nixora Group is the implementation partner for ASG Technologies in Australia and New Zealand. If you

would like to know how to take full advantage of ASG Data Intelligence Solution, call us today for a

consultation and for a proof of concept of Nixora-ASG proposition, a “fit-for-purpose exercise” on a small

set of technologies with the objective to demonstrate the value, capability and effectiveness of Nixora-

ASG approach.

You can email us at giovanni.butera@nixoragroup.com.au, connect with us on LinkedIn, follow us on

Facebook or visit our web site nixoragroup.com.au.

ASG Technologies brings peace of mind to every enterprise with information access, management and control for our customers. These solutions empower businesses to enhance workforce productivity, gain an accurate and timely understanding of the information that underpins business decisions and address compliance needs with improved visibility of cross-platform data from legacy to leading edge environments. ASG is a global provider of technology solutions with more than 1,000 people supporting more than 4,000 midmarket and enterprise customers around the world. For more information, visit www.asg.com.

ASG Technologies | 1.239.435.2200 or 1.800.932.5536 | 708 Goodlette Road North, Naples, Florida USA 34102 | www.asg.com

© 2018 ASG Technologies Group, Inc. All products mentioned are trademarks or registered trademarks of their respective holders. ASG-WP-Data-Lineage-make-no-mistakes-about-it_20180329en