peter benson fima 2009
DESCRIPTION
TRANSCRIPT
Managing Your Enterprise–Wide Data To Minimize RiskManaging Your Enterprise–Wide Data To Minimize Risk
Day 1 - Data Quality and MetricsDay 1 - Data Quality and Metrics
4:15-4:504:15-4:50
Bringing your data up to Bringing your data up to
ISO 8000 Data Quality StandardsISO 8000 Data Quality Standards
FIMA 2009 Monday March 16, 2009
New York, NY
Mr. Peter BensonISO 8000 Project Leader Executive Director and Chief Technical OfficerElectronic Commerce Code Management Association (ECCMA)
Standards Hierarchy
STRATEGIC
TACTICAL
Market Acceptance
Market Access
Social Responsibility
Standards of Convenience
Customer Requirements
Government Requirements
Societal Expectations
Industry and CompanyPreferences
Mandatory Voluntary
FIMA 2009, March 16, 2009, New York, New York
If
Quality information comes from Quality Data
and
Enterprise-wide data is Master Data
What is ISO 8000-110:2008 Master Data Quality? and
What are the costs and benefits of bringing data up to the new international standard?
FIMA 2009, March 16, 2009, New York, New York
Wisdom
Knowledge
Information
Data
Information vs. Data
“… the quality of the data element values is a major determinant of the quality of the information and consequently the accuracy and reliability of the knowledge that can be derived from the information.” ISO 8000-110:2008
FIMA 2009, March 16, 2009, New York, New York
Information* representation of something and what can be logically inferred from it NOTE 1 The term "representation" is used in its most general sense to include not only symbolic representations, but also images and what is held in the mind.
NOTE 2 The phrase "logically inferred" means statements that can be deduced by applying the laws of logic.
Data* symbolic representation of something that depends, in part, on its metadata for its meaningNOTE An image could be a representation, but not a symbolic representation.
EXAMPLE The written description of a chair is a symbolic representation. This is contrasted with a picture of a chair, which is a non-symbolic representation.
* Represents international consensus of ISO TC 184/SC 4 subject to ballot
ISO 8000 Definitions
FIMA 2009, March 16, 2009, New York, New York
ISO 8000 – Data Quality
• ISO 8000 addresses data quality. ISO 8000 is concerned with:
– the principles of data quality;
– the characteristics of data that determine its quality;
– the processes to ensure data quality.
FIMA 2009, March 16, 2009, New York, New York
data
transactional data
master data describes entities that are independent and fundamental for the organization;
needs to be referenced in order to perform transactions
identification datadefined by reference to another organization’s
master data
descriptive datadefined by the
characteristics of the entity being described
Master Data
Data that identifies and describes individuals, organizations, locations, goods, services, rules
and regulations.
• Vendor master
• Customer master
• HR master
• Asset master• Material master• Service master• etc.
FIMA 2009, March 16, 2009, New York, New York
ISO 8000 Data Quality
100 Series covers Master Data QualityISO 8000-100:2009* Master Data Quality – Introduction and terminologyProvides an overview of master data and the terms and definitions used to define it.
ISO 8000-110:2008 Master Data Quality - Syntax, semantics, and conformance to data requirementsSpecifies requirements that can be checked by computer for the exchange, between organizations and systems, of master data that consists of characteristic data.
ISO 8000-120:2009* Master Data Quality - ProvenanceSpecifies requirements for representation and exchange of information about provenance of property value pairs.
ISO 8000-130:2009* Master Data Quality - AccuracySpecifies requirements for representation and exchange of information about accuracy of property value pairs.
* Completed ISO ballot cycles, awaiting publication
FIMA 2009, March 16, 2009, New York, New York
ISO 8000-110:2008
Syntax
Each data set shall contain a reference to the syntax to which the data set complies....The reference shall be resolvable to the specification of the syntax through a mechanism that is publicly available.
Semantic encoding
Each data element value shall reference all concepts necessary to unambiguously define its meaning. Each reference shall be to a concept dictionary entry contained in a concept dictionary that supports an interface for resolution of a concept identifier.
Syntax and semantic resolution shall be available at no charge unless the data carries a “fee based encoding” warning label.
Conformance to requirements
Each data set shall contain a reference to the data requirements statement to which the data set complies. The reference shall be a globally unambiguous identifier that was used to encode the data set. The reference shall be resolvable to the data requirements statement. The data requirements statement shall be publicly available.
FIMA 2009, March 16, 2009, New York, New York
All identifiers are copyright. They belong to the organization that issued them and their use is subject to the terms and conditions imposed by the issuer.
Unless identifiers have been declared available for public use without a licence, they should never be used to retrieve data that was not supplied by the owner of the identifier unless you have specific permission to do so.
In order to protect your data from claims of “joint work” you should not use proprietary identifiers as metadata.
Data quality and Intellectual Property (IP)
FIMA 2009, March 16, 2009, New York, New York
A syntax must exist
Semantic encoding must be explicit
“The great thing about standards is that there are so many to chose from.” Anon.
ISO 8000-110:2008 is standards neutral
ISO 9362 Bank Identifier Code (BIC / SWIFT Code / )
ISO 10383:2003 Codes for exchanges and market identification (MIC)
ISO 6166:2001 International Securities Identification Number (ISIN)
ISO 9735 Electronic Data Interchange
ISO 15022 Securities -- Scheme for messages
ISO 20022 Universal financial industry message scheme
ISO 19312 Market Data Model - under development
Country code + Tax IDor DNA?
Ultimate Personal Identification Number
Country code + Tax ID International Organization Identification Number
GPS? International real estate parcel identification number
FIMA 2009, March 16, 2009, New York, New York
Structured Data
Data requestor
Data
Application
Application
Graphics Documents
Data provider
Data
Traditional data transfer
FIMA 2009, March 16, 2009, New York, New York
ISO 8000-110 is about automating the data supply chain
Data requestor
Data provider
Sub
eOTD-i-xml(data requirements statement) ISO 22745-30
eOTD-q-xml(query)
ISO 22745-35
Sub-TiereOTD-q-xml
Sub-TiereOTD-r-xml
eOTD-r-xml(data exchange)ISO 22745-40
FIMA 2009, March 16, 2009, New York, New York
Meeting the requirements of ISO 8000-110:2008 Master Data Quality
• Certified Software applications and data cleaning services
– Know how to use an open technical dictionary for metadata encoding and mapping
– Know how to read and write master data requirement specifications
– Know how to generate requests for master data
– Know how to read and write encoded master data messages
• Certified Master Data Quality Managers (data requestors)
– Know how to specify their master data requirements
– Know how to ask for they data they need to validate or complete their master data
• Certified Quality Master Data Providers
– Know how to respond to a request for master data
FIMA 2009, March 16, 2009, New York, New York
NEXT STEPS1. ISO 8000-110:2008 Master data quality; Syntax, semantic
encoding and conformance to data specification.
a) Develop your metadata registry (map it to the eOTD*)
b) Develop your data requirement statements (as eOTD-i-xml Identification guides*)
c) Measure the quality of your data - perform a gap analysis
d) Develop a plan to source missing data
e) Develop a plan to validate data
2. ISO 800-120:2009 Master data quality; Provenance
a) Add provenance data to imported and exported data
* optional