A Global Open StandardFor Building

Your Controlled Vocabulary

Ron Schuldt, ChairRon Schuldt, Chair

The Open Group UDEF ProjectThe Open Group UDEF ProjectJuly 13, 2009July 13, 2009

UU DD EE FF

AgendaAgenda

• The Problem

• The Missing Metadata Layer

• A Global Metadata Managed Architecture

• ISO/IEC 11179 – Metadata Registry Standard

• Universal Data Element Framework (UDEF) Overview

• Leveraging Online Gap Analysis

• Sample Controlled Vocabularies

• The Need for Distributed UDEF

• Example Distributed UDEF Use Cases

• UDEF Features Summary

• Some UDEF Tools

Enterprises Want ThisEnterprises Want This

External “In” Space

Internal Space

External “Out” Space

Procuring

ManufacturingLegalFinanceAssembling

Customer Support

ProcurementSystems

DesignSystems

Online Systems

ERPSystems

RequirementsSystems

Systems

Processes

But Experience This*

* Extract from The Open Group SOA White Paper

At present, this problem is typically addressed within the services, and by use as far as possible of enterprise-standard and industry-standard vocabularies. In the future, it may be addressed by semantic technology that is incorporated in the infrastructure, which will be a more scalable way to deliver the ideal of Boundaryless Information Flow.

Problem – Global PerspectiveProblem – Global PerspectiveEach organization is attempting to set its own semantics standardEach organization is attempting to set its own semantics standard

Each must interface with organizations they do not controlEach must interface with organizations they do not control

The problem is the The problem is the lack of a controlled vocabularylack of a controlled vocabulary

Customers

Suppliers

Partners

Other Retail

Local

State

TransUtilities

Banks

Federal

Organization

Problem – Internal PerspectiveProblem – Internal Perspective

<PARTNUMBER>111-222-333</PARTNUMBER>

<partNumber>111-222-333</partNumber>

<PartNumber>111-222-333</PartNumber>

<partnumber>111-222-333</partnumber>

Though semantically equal, the following are 4 different XML tag names

App B App C

App A

Other Apps

Legacy Data

Conflicting semantic overlaps between back-office systems

The Goal

Current Point-to-Point Approach --- n(n-1)

Global Semantics Standard Approach --- 2n

$$

Savings

GlobalSemanticsStandard

0

50

100

150

200

250

300

350

400

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Reduce Requirements and Design-Time Phase Semantics Analysis Time and Cost

Data-Information-Knowledge-WisdomData-Information-Knowledge-Wisdom

Data - Information Reference Model

•  Databases •  Data Formats

• Information discovery • Information retrieval

•  File Servers •  Repositories

•  Naming Conventions •  Indexing Methods•  Registry

• Ontologies• Taxonomies • Vocabularies

•  Search engines •  Inference engines•  File retrieval

•  Data standards •  Middleware•  DBMS

•  Vaults •  Format standards•  Change control

Structured DataStores

Un-Structured DataFiles

Data - Metadata Management

Data Management

Information - Metadata Management

Information Management

•  Logical Data Model •  File Labels

Analogous to the OSI 7 layer reference model

Example functions and technologies for each layer

Data - Information Reference Model

•  Databases •  Data Formats

• Information discovery • Information retrieval

•  File Servers •  Repositories

•  Naming Conventions •  Indexing Methods•  Registry

• Ontologies• Taxonomies • Vocabularies

•  Search engines •  Inference engines•  File retrieval

•  Data standards •  Middleware•  DBMS

•  Vaults •  Format standards•  Change control

Structured DataStores

Un-Structured DataFiles

Data - Metadata Management

Data Management

Information - Metadata Management

Information Management

•  Logical Data Model •  File Labels

Analogous to the OSI 7 layer reference model

Example functions and technologies for each layer

The layer typically overlooked

Wiki - Controlled Vocabularies

“Controlled vocabularies provide a way to organize knowledge for subsequent retrieval. They are used in subject indexing schemes, subject headings, thesauri and taxonomies. Controlled vocabulary schemes mandate the use of predefined, authorized terms that have been preselected by the designer of the vocabulary, in contrast to natural language vocabularies, where there is no restriction on the vocabulary.”

The above provides a good description of the function that the UDEF offers to any enterprise on the planet since the UDEF is open and is infinitely extensible and can transcend any domain or function of any enterprise.

The Open Group Architecture Framework (TOGAF)The Open Group Architecture Framework (TOGAF)

The Open Group Vision Statement

The Open Group is a vendor-neutral and technology-neutral consortium,

whose vision of Boundaryless Information Flow™ will enable access to integrated information, within and among enterprises, based on open standards and global interoperability.

TOGAF does a great job with Enterprise Architecture but does not address Boundaryless Information Flow

TOGAF v8.1 Principle 10

Statement

“Users have access to the data necessary to perform their duties; therefore, data is shared across enterprise functions and organizations.”

Rationale

“… The speed of data collection, creation, transfer, and assimilation is driven by the ability of the organization to efficiently share these islands of data across the organization. …”

Implications

“… need to develop standard data models, data elements, and other metadata that defines this shared environment and develop a repository system for storing this metadata to make it accessible. …”

Chapter 29, Data is Shared

Metadata Management

Metadata describes the characteristics of an Metadata describes the characteristics of an artifact, such as its name, type of content, artifact, such as its name, type of content, sensitivity, lifecycle phase, perceived sensitivity, lifecycle phase, perceived importance to specific audiences, quality or importance to specific audiences, quality or value to the enterprise, and its relationships to value to the enterprise, and its relationships to other artifacts. other artifacts.

Managed Metadata with a Managed Metadata with a Controlled Vocabulary Controlled Vocabulary … Standard Metadata… Standard Metadata Will enable organization to reduce the costs Will enable organization to reduce the costs

of building and maintaining application of building and maintaining application interfacesinterfaces

Is a necessary enabler of a Service Oriented Is a necessary enabler of a Service Oriented Architecture (SOA)Architecture (SOA)

Will improve data and information assets Will improve data and information assets discovery and reuse … data miningdiscovery and reuse … data mining

Will improve knowledge worker productivityWill improve knowledge worker productivity

To achieve enterprise IT agility requires effective metadata management

Invoice No.

Qty

Total Price

Order Date

Buyer

1234

10

9/20/07

Metadata Data

John Doe

$210.00

Metadata Registry ISO/IEC 11179 Standards

Part 1: Metadata Registries - Framework

Part 2: Metadata Registries - Classification

Part 3: Metadata Registries - Registry Metamodel and Basic Attributes

Part 4: Metadata Registries - Formulation of Data Definitions

Part 5: Metadata Registries - Naming and Identification Principles

Part 6: Metadata Registries - Registration

http://standards.iso.org/ittf/PubliclyAvailableStandards/index.html/

ISO/IEC 11179-5 Summary*

• Provides guidance for the naming or identification of data constructs administered in a metadata registry.

• Names are applied to data constructs through the use of naming conventions. There are semantic, syntactic and lexical rules used to form a naming convention.

• Identifiers are designations meant for dereferencing data constructs for metadata management and exchange.

*Extracts from ISO/IEC 11179-1:2004(E) – section 8.1.5ISO/IEC 11179-1:2004(E) – section 8.1.5

 

ISO/IEC 11179 – Terminology

DataElementConcept

DataElement

ValueDomain

ObjectObjectClassClass

PropertyProperty RepresentationRepresentation

CoreData

Element

ApplicationData

Element

The “Key” for a Controlled Vocabulary of Structured Data

A Metadata Managed Architecture

EAI

Transformation Engines

Interfaces to Back-Office

Systems

• Data Dictionary

• Mapping Matrices

• Std XML Schema

UDEF-Indexed Metadata Registry/Repository

InterfaceDevelopers

Run Time

Data ModelersAnd Apps Developers

Design Time

Internet

UDEFExtension Process

Global UDEF Registry

Vendors with Canonical Models

Software Vendors

with UDEF IDAPIs Web

Public

Extend Matrices

UseMatrices

Std Schema

UDEF-Indexed Metadata Registries

Build/Extend Schema

Centralized metadata registry/repository

• Enables reuse to reduce costs

• Encourages standardization

UDEF Thumbnail Sketch

The Open Group’s Universal Data Element Framework (UDEF)

The only truly open standard that will simplify building an enterprise-wide controlled vocabulary for structured data and facilitate normalization across data models while enabling SOA I/O interoperability across all services that adopt UDEF through global indexing keys

Oracle’s Daniel Serain recommends using UDEF global

indexing keys for SOA services I/O

UDEF is an Open Standard that Enables Interoperability

Universal Data Element FrameworkUniversal Data Element Framework

UDEF names follow the rules of English – qualifiers precede the word they modify

Data Element Concept Name

Object Class Term

0...n qualifiers +1 or more required

Object Class

+

Example UDEF-Based Data Element Concept Names

Document Abstract Text

Enterprise Name

Product Price Amount

Product Scheduled Delivery Date

Engineering Design Process Cost Amount

Employee Person Family Name

UDEF ObjectClass List• Entity• Document• Enterprise• Place• Program• Product• Process• Person• Asset• Law-Rule• Environment• Condition• Liability• Animal• Plant• Substance• Event

Property Term

0..n qualifiers +1 required Property

Property List*• Amount• Code• Date• Date Time• Graphic• Identifier• Indicator• Measure• Name• Percent• Picture• Quantity• Rate• Text• Time• Value• Sound• Video

ISO/IEC 11179-5 Naming Convention

UDEF is a proposed universal instantiation of ISO/IEC 11179-5

* Based on Tables 8-1 and 8-3 in ISO 15000-5

Creating UDEF IDs

UDEF Trees

17 Object Class Trees 18 Property Trees

Entity Asset Person Amount Name… …

Patientau

50 1

Family Alias…Country …

101…

1 11 40

Patient Person_Family Name has UDEF ID = au.5_11.10

See http://www.opengroup.org/udefinfo/defs.htm

Employee …

a

…

UDEF Provides The Indexing Key

External

Conceptual

Physical

System A

External

Conceptual

Physical

System B

Registry Derived Name

Emp_Start Empl-Hire-Dt

EmpStr EmplHr

a.5_49.6

Registry Derived IDRegistry Derived ID

Open Group UDEF Registry

Map to registry

Assign UDEF ID alias

UDEF Object Term identifies the table

UDEF Property Term identifies the column

Employee PERSON

Hired DATE

UDEF Object Term

UDEF Property Term

UDEF – Current Status

All enterprisesAll enterprises

use the UDEFuse the UDEF

Controlledextensionprocess

Baseline UDEFin usable form

Support by keyvendors (IBM,

SAP, Oracle . .)

Early adoptersuse the UDEF

Training

Demonstratedbenefits

Promotion

Pilot – keyvendor(s) &end user(s)

Global Registry

HomelandSecurity

& Electronic HealthRecord

Launch

Ready

Started

UDEF in other languages

Disaster Response

Initial

UDEF ID Object Type or Role Object Class Property Type Property

a.5_51.6 Birth

HR ExampleEmployee

PersonEmployee

Number a.5_12.35.8

Date

Hire Date

Last Name

First Name

Birth Date

a.5_49.6

a.5_51.6

a.5_12.10

a.5_11.10

6. Derive a structured ID based on the UDEF taxonomy that carries the UDEF inherited indexing scheme. For example <CustomerPersonFamilyName UDEFID=“as.5_11.10”>

Create UDEF Mappings

d.t.2_8

OAG Purchase Order: xCBL Purchase Order:

d.t.2_8

Map Both Systems and/or Both Standards Map Both Systems and/or Both Standards Independently to the UDEFIndependently to the UDEF

Source and Target Files

OAGIS StandardOAGIS Standard xCBL StandardxCBL Standard

Leveraging Online Tool

Vocabulary GapSoftware

On-the-Wire Java and XSLT

SourceSystem

TargetSystem

Source XML Instance Doc with UDEF IDs

Target XML Instance Doc with UDEF IDs

SOAP SOAPDHTML

On-line Gap Analysis Report

Online Gap Analysis Tool – Online Gap Analysis Tool – https://jserver.opengroup.org/UDEF/UdefReport1

Gap Analysis Results

OAGIS StandardOAGIS Standard xCBL StandardxCBL Standard

Gap Analysis Results - EnhancedOAGIS StandardOAGIS Standard xCBL StandardxCBL Standard

HR Controlled Vocabulary Examples

For employee identification, the UDEF based data element concepts’ names and IDs are:

UDEF Name = Controlled VocabularyUDEF Name = Controlled Vocabulary UDEF IDUDEF ID

Employee Person Employer Assigned Identifier a.5_12.35.8

Employee Person Hired Date a.5_49.6

Employee Person Family Name a.5_11.10

Employee Person Given Name a.5_12.10

Employee Person Gender Code a.5_3.4

Employee Person Birth Date a.5_51.6

All HR applications on the globe could use “a.5” as the indexing ID for identifying the application table that contains employee data. The portion of the UDEF ID that starts with the underscore (i.e., _12.35.8, _11.10, _3.4, etc.) provides the indexing ID to identify the associated column in the HR table.

Procurement Controlled Vocabulary

For employee identification, the UDEF based data element concepts’ names and IDs are:

UDEF Name = Controlled VocabularyUDEF Name = Controlled Vocabulary UDEF IDUDEF ID

Purchase Order Document IdentifierPurchase Order Document Identifier d.t.2_8d.t.2_8

Purchase Order Document DatePurchase Order Document Date d.t.2_6d.t.2_6

Purchase Order Document Line Item IdentifierPurchase Order Document Line Item Identifier d.t.2_17.8d.t.2_17.8

Ordered Product IdentifierOrdered Product Identifier z.9_8z.9_8

Ordered Product Ordered QuantityOrdered Product Ordered Quantity z.9_13.11z.9_13.11

Ordered Product Total Price AmountOrdered Product Total Price Amount z.9_6.2.1z.9_6.2.1

All Procurement applications on the globe could use “d.t.2” as the indexing ID for identifying the application table that contains purchase order data. The portion of the UDEF ID that starts with the underscore (i.e., _8, _17.8, _6, etc.) provides the indexing ID to identify the associated column in the purchase order table.

The Need for Distributed UDEFThe Need for Distributed UDEF

Distributed UDEFDistributed UDEF

UDEF-tree

Descendant tree

Descendant tree

connectors

Global UDEFIndustry UDEF

Private UDEF

Example Industry SDEF MappingsExample Industry SDEF Mappings

Example Private UDEF ScenarioExample Private UDEF Scenario

UCAV

MH-60

AWACSF/A-18E/F

SOF

F-22

CAOC

SGU

U S TSDisplays

RC-135

Global Hawk - SAM

Global Hawk - SCUD

Global Hawk - CSAR

PDEF Scenario – Mission AssignedPDEF Scenario – Mission Assigned

UCAV

F/A-18E/F

Global Hawk - SAM

CAOC

TST Cell

SGU

U S TSDisplays

RC-135

(2)TransmitSAM sitetarget info

SAM

(1) Picks up

SAM siteson RWR

(5)Provide image

(6)CAOC displays SAM site image

(7) Task F/A-18E/F to

destroy SAM sites (ATO)

(8)ATO Ack

(4)ATO Ack.

(9) Task UCAV to

destroy SAMsites (ATO)

(10)ATO Ack

(3)Task Global Hawk

to providetarget area

images (ATO)

PDEF Scenario – Accomplish MissionPDEF Scenario – Accomplish Mission

UCAVCAOC

TST Cell

SGU

U S TSDisplays

SAM

(15)Provide image/video

(16)CAOC displays

SAM siteimage/video

(11)UCAV drops

bombs onSAM site

(13)Task Global Hawk

to performbomb damageassessment

(14)ATO Ack.

Global Hawk - SAM

F/A-18E/FRC-135

(12) Bomb dropped on SAM site

(12) Bomb dropped on SAM site

Example PDEF MappingsExample PDEF MappingsSee the UDEF Asset tree and UDEF Measure tree, respectively.

1) Tail-ID-1234.RC-135.UDEF-P.Military.Aircraft.Asset_Latitude.Coordinate.Measure with

UDEF ID = a.a.UDEF-P.a.o.1_2.11.13

The UDEF ID “a.a.UDEF-P.a.o.1_2.11.13” would be transmitted with the applicable latitude value to the applicable command center receiver and would be interpreted as

Tail-ID-1234.RC-135.Military.Aircraft.Asset_Latitude.Coordinate.Measure with the associated latitude value included within the message. The private portion of the UDEF name “Tail-ID-1234.RC-135” and associated private ID “a.a.” would only be known by those military systems controlled by the private registry that assigned the UDEF extensions below the UDEF-P tag under Military.Aircraft.Asset.

Other examples include the following:

2) Tail-ID-1234.RC-135.UDEF-P.Military.Aircraft.Asset_Longitude.Coordinate.Measure with

UDEF ID = a.a.UDEF-P.a.o.1_3.11.13

3) Tail-ID-1234.RC-135.UDEF-P.Military.Aircraft.Asset_Altitude.Coordinate.Measure with

UDEF ID = a.a.UDEF-P.a.o.1_5.11.13

4) Tail-ID-9876.FA-18.UDEF-P.Military.Aircraft.Asset_Latitude.Coordinate.Measure with

UDEF ID = f.b.UDEF-P.a.o.1_2.11.13

Example PDEF MappingsExample PDEF Mappings5) Tail-ID-9876.FA-18.UDEF-P.Military.Aircraft.Asset_Longitude.Coordinate.Measure with

UDEF ID = f.b.UDEF-P.a.o.1_3.11.13

6) Tail-ID-9876.FA-18.UDEF-P.Military.Aircraft.Asset_JDAM.Bomb.Available.Quantity with

UDEF ID = f.b.UDEF-P.a.o.1_1.1.UDEF-P.2.11

7) Tail-ID-9876.FA-18.UDEF-P.Military.Aircraft.Asset_Available-Fuel.Weight.Measure with

UDEF ID = f.b.UDEF-P.a.o.1_1.UDEF-P.8.13

The private portion of the UDEF name “JDAM.Bomb” and associated private ID “1.1.” would only be known by those military systems controlled by the private registry that assigned the UDEF extensions below the UDEF-P tag under Available.Quantity. Similarly, the private portion of the UDEF name “Available-Fuel” and associated private ID “1.” would only be known by those military systems controlled by the private registry that assigned the UDEF extensions below the UDEF-P tag under Weight.Measure.

8) Enemy.Surface-To-Air-Missile-Launcher.UDEF-P.Vehicles.Asset_Latitude.Coordinate.Measure with UDEF ID = a.a.UDEF-P.i.1_2.11.13

9) Enemy.Surface-To-Air-Missile-Launcher.UDEF-P.Vehicles.Asset_Longitude.Coordinate.Measure with UDEF ID = a.a.UDEF-P.i.1_3.11.13

The private portion of the UDEF name “Enemy.Surface-To-Air-Missile-Launcher” and associated private ID “a.a.” would only be known by those military systems controlled by the private registry that assigned the UDEF extensions below the UDEF-P tag under Vehicles.Asset.

10 )Enemy.MIG-29.UDEF-P.Military.Aircraft.Asset_Latitude.Coordinate.Measure with

UDEF ID = a.x.UDEF-P.a.o.1_2.11.13

Example PDEF MappingsExample PDEF Mappings11) Enemy.MIG-29.UDEF-P.Military.Aircraft.Asset_Longitude.Coordinate.Measure with

UDEF ID = a.x.UDEF-P.a.o.1_3.11.13

12) Enemy.MIG-29.UDEF-P.Military.Aircraft.Asset_Altitude.Coordinate.Measure with

UDEF ID = a.x.UDEF-P.a.o.1_5.11.13

The private portion of the UDEF name “Enemy.MIG-29” and associated private ID “a.x.” would only be known by those military systems controlled by the private registry that assigned the UDEF extensions below the UDEF-P tag under Military.Aircraft.Asset

13) Tail-ID-9876.FA-18.UDEF-P.Military.Aircraft.Asset_Target-Engagement.UDEF-P.Type.Code with

UDEF ID = f.b.UDEF-P.a.o.1_2.UDEF-P.33.4

Example target-engagement code values might be:

1 = Air-To-Surface

2 = Air-to-Air

The private portion of the UDEF name “Target-Engagement” and associated private ID “2.” would only be known by those military systems controlled by the private registry that assigned the UDEF extensions below the UDEF-P tag under Type.Code.

14) Enemy.Surface-To-Air-Missile-Launcher.UDEF-P.Vehicles.Asset_Destroyed.Photo.Picture with

UDEF ID = a.a.UDEF-P.i.1_1.UDEF-P.1.3

15) Enemy.MIG-29.UDEF-P.Military.Aircraft.Asset_ Destroyed.Photo.Picture with

UDEF ID = a.x.UDEF-P.a.o.1 _1.UDEF-P.1.3

Some UDEF FeaturesSome UDEF Features

• Conforms to ISO/IEC 11179-5

• Establishes global indexing scheme for data element concepts analogous to Dewey Decimal System used in libraries to index books

• Enables normalization of data models on global scale

• Viewable for anyone with a browser

• Online gap analysis utility to anyone with a browser

• Multi-lingual (German, French, Chinese soon) in addition to English baseline and Dutch equivalent

• Unlimited extensibility

• Anyone with an email address can propose extensions

• Downloadable in XML and RDF formats to anyone with an email address

• Supports industry or organization unique extensions in a manner analogous to Internet DNS

Free UDEF ToolsFree UDEF Tools

• Free Tools

• Online Gap Analysis Tool on The Open Group Web Site

• Download sample source and target XML files for purchase order based on two different standards

See Gap Analysis section of http://www.opengroup.org/udefinfo/

• Obtain gap analysis reporthttps://jserver.opengroup.org/UDEF/UdefReport1

• UDEF Explorer Mapping Tool by Knotionhttp://www.knotion.net/index.php?option=com_content&task=view&id=68&Itemid=76

For Additional Information

OPENGROUP UDEF Web Site – to download the UDEF

http://www.opengroup.org/udefinfo/

The UDEF On-Linehttp://www.opengroup.org/udefinfo/defs.htm

Dr. Chris Harding – c.harding@opengroup.org

Ron Schuldt – ron.l.schuldt@lmco.com, 720-842-6414