1 schema registries steven hughes, lou reich, dan crichton nasa 21 october 2015
TRANSCRIPT
3
Definitions
Registry - Formal record of information, or the place where such records are kept.
Repository - A repository is a place where data is stored and maintained.
Registry and Repository are referenced in the Information Architecture and OAIS RM papers but they are not defined.
4
Registry vs Repository
Repository holds data Provides “get” and “put” functions to retrieve and store data
holdings using a “unique identifier”.
Some additional “operative” metadata may exist primarily for managing the data in the repository. (dates, checksums, etc).
Repository may hold “archive” metadata, but it is co-located with the data. In the repository context the metadata is not considered operative.
Repository has a “register function”. (i.e. A repository is a register in the sense that a formal record of its contents is kept by registering or “making or securing an official entry”)
Repository is similar to the “OAIS Archival Storage” functional entity
5
Registry vs Repository
Registry assigns an identifier to object
Registry uses metadata to build a catalog (or index) Makes the metadata searchable for identifying and locating stored
objects
Supports two phased data access (i.e. search and then retrieve)
Registry is similar to the OAIS functional entity “Data Management”. It supports “OAIS Access”.
Registry is considered separately from a repository since the two functional entities may be deployed separately.
Registry and Repository functions are often collocated.
6
Types of Registries
Metadata registry provides an interface to search for data about metadata. Manage meta-metadata models. (e.g. XML Schemas, Data
Dictionaries, Ontologies
Resource registry provides an interface to search for resources. Manage descriptions of any resource (i.e. object), typically
information objects such as science data products and data sets.
Service registry provides an interface to search for functional services that perform a needed action specified by a user. Manage descriptions of service interfaces (called service
descriptions), including their respective locations, methods and method parameters.
7
Schema Registry
Repository/registry that accepts, stores, and maintains XML schemas Functionality is shared across a distributed environment
Support schema design, development, validation, deployment, evolution, and maintance
8
XFDUProducer Application
XFDUConsumerApplication
XMLToolkitLibrary
PackagingPlug-ins
TransformationPlug-ins
Validation Plug-ins
PluginRegistry/ Repository
XMLRegistry/ Repository
System Programmer
Producer Consumer
CCSDS XFDU Administration
Registration Dissemination Notification Validation Query Subscription Reconstitution
Application Domain Programmer
9
Functional Requirements
Full Standards Compliance Import and validate W3C XML Schema or DTD
Version Control Schema level Object level Configuration management with audit trails Check-In, Check Out Release Workflow – local updates, checkin, integration, create
release, deploy release Subscribe/Notification
Impact Analysis Track dependencies between objects Determine impact of changes
10
Functional Requirements
Object Management Manages schemas at their object level, not at the file or file
fragment level (i.e. To the attribute or element level)
Any redundancy and duplication is resolved at import/update time
Reuse happens through reference links
Assemble schemas dynamically from the objects
Common Repository Developers work in their own personal workspaces
Shared object database provides for team-based development
11
Functional Requirements
Documentation Object annotation Documentation production on request
Multiple namespaces Relationships between elements Automatic transformations
Schema artifact respository
CortexML - Collaborative XML Schema Management
http://www.digitalml.com/en/index.php
12
Functional Requirements
Schema Validation Validate well formed XML and log errors.
XML Instance Validation Validate an XML file against an internally referenced DTD or XSD
13
Plan
Determine interest
Consider working group for schema registries
Gather requirements for schema registries
Define standard information model and services for schema registries
Identify role of schema registry in information infrastructure
15
DSMS NeedsRegistry Service Components
Data Element/Data Type Registry: Provides common data dictionary information that contains element definitions and data type definitions
XML Schema Registry: Provides a versioned repository for XML schemas that describe messages, file formats, and data components.
XML Stylesheet Registry: Provides a versioned repository for XML stylesheets that perform common data translations.
Namespace/Domain Registry: Provides a controlled, hierarchical namespace/domain registry.
Service Registry: Provides a dynamic Registry for Web Services.
Model Registry: Provides a repository for information models, data relationships, and other ontological information.
16
DSMS Needs CIO/Standards Efforts
Schema Registry A component of the Metadata Service.
Currently deployed in the DSMS development environment.
Data Element Registry A component of the Metadata Service.
Currently deployed in the DSMS development environment.
Service Registry Evaluating COTS and Open Source products.
17
Schema RegistryOverview
This registry provides a central repository for developers and organizations to publicly submit, publish and exchange XML-based schemas.
A component of the Metadata Service. An effort currently funded by the CIO.
Interfaces Application Program Interface (API)
Support distributed access using CORBA/RMI
Support Java and HTTP application interfaces
Command-Line
Web-Based GUI
Planned Features for Future Releases Support for Authentication and Authorization
18
Schema RegistryCurrent Deployment
Deployed with Metadata Service release 2.0.0 http://metadata.jpl.nasa.gov/ on February 9
http://bayamo.jpl.nasa.gov:8080/ on March 5 (DSMS Deployment)
Features Add, remove and modify schemas and their supporting
metadata in the registry.
Metadata capture based on the Dublin Core Element Set.
Allow for versioning and promotion of schemas.
Capability to locate and access registered schemas.
Provide for URL access to registered schemas.
Provide for a web directory hierarchy for organizing schemas.
19
Data Element RegistryOverview
This registry provides the capability to register and manage a data element and associate it with a data dictionary.
A component of the Metadata Service. An effort currently funded by the CIO.
Interfaces Application Program Interface (API)
Support distributed access using CORBA/RMI
Support Java and HTTP application interfaces
Command-Line
Web-Based GUI
Planned Features for Future Releases Support for value domain
Support for element relationships
Support for Authentication and Authorization
20
Data Element RegistryCurrent Deployment
Deployed with Metadata Service release 2.0.0 http://metadata.jpl.nasa.gov/ on February 9
http://bayamo.jpl.nasa.gov:8080/ on March 5 (DSMS Deployment)
Features Based on the ISO/IEC 11179 “Information technology --
Specification and standardization of data elements”.
Add, remove and modify data elements and their supporting attributes in the registry.
Allow for versioning of data elements.
Associated data elements to a data dictionary.
Extract data elements from an imported data dictionary via the Dictionary Registry component.
Capability to locate and access registered data elements.