noaa metadata update ted habermann. noaa edmc documentation directive this procedural directive...
TRANSCRIPT
NOAA Metadata UpdateTed Habermann
NOAA EDMC Documentation Directive
This Procedural Directive establishes
1) a metadata content standard (International Standards Organization [ISO] 19115 Parts 1 and 2) and
2) a recommended representation standard (ISO 19139)
for documenting NOAA’s environmental data and information.
Roles and Responsibilities
Background Information
Building the Foundation
Develop and implement common metadata management tools
Support training specifically targeted at improving NOAA’s data documentation
Initiate teams to work on “special documentation problems” that cross Line and Staff Offices
Promote and highlight good examples
Encourage and support participation in the ISO and Open Geospatial Consortium (OGC)
Use rubrics to establish a baseline and monitor progress
NOAA Line/Staff Offices and Programs must build on
a strong foundation of support across all of NOAA
Line / Staff Office & Program Responsibilities
Data Collectors/Providers
Data Users
Data Stewards
Standards Experts
1. Identify Documentation Expertise
3. Create and Improve Metadata
2. Assess current state of documentation
4. Publish/Preserve Metadata
Metadata for DiscoveryMetadata for UseMetadata and Documentation for UnderstandingDocumentation of CollectionsDocumentation of DatasetsDocumentation of Services
NOAA Data Centers / Archives
Building the Foundation
The GOES-R Program recently expanded the scope of their metadata model from just ISO to ISO plus netCDF.
This creates an integrated metadata framework that includes metadata in granules (files) as well as the ISO standards.
This framework is important across NOAA and for many of our partners (NASA).
Initiate teams to work on “special documentation problems” that cross Line and Staff Offices
Building the Foundation
The ISO 19115 metadata standard adopted by NOAA is currently undergoing a significant revision that addresses many requirements from NOAA and the netCDF-CF community that includes many NOAA data providers and partners.
These changes will be adopted as part of the Draft International Standard which is about to be approved by ISO (Nov. 2011)
Encourage and support participation in the ISO and Open Geospatial Consortium (OGC)
Building the Foundation
ISO metadata standard trainings were held during June in Silver Spring as part of the annual EDMC Data Management Workshop and during October as part of the FGDC ISO Metadata Summit
The GEO-IDE Wiki includes over 200 pages of information about the ISO Standards.
Support training specifically targeted at improving NOAA’s data documentation
Building the Foundation
A set of tools for translating metadata collections from FGDC to ISO have been developed and tested by NODC and NGDC.
Tools for evaluating the completeness of ISO metadata that include links to the GEO-IDE wiki have been developed and tested at NGDC.
Develop and implement common metadata management tools
Use rubrics to establish a baseline and monitor progress
What Goes Where?
Spiral 1:Initial ContentSpiral 1:Initial Content
Metadata ContentIndependent of standardMetadata ContentIndependent of standard
StandardGuidance /
Implementation
StandardGuidance /
Implementation
Check BackWith UsersCheck BackWith Users
Spiral 2-N: Scientific QuestionsNew RequirementsNew Use Cases
Check BackWith Data Collectors/
Providers
Check BackWith Data Collectors/
Providers
Spiral Development / Training
IdentificationIdTitleAbstractResource DateTopic CategoryTheme KeywordMetadata ContactScience Contact
IdentificationIdTitleAbstractResource DateTopic CategoryTheme KeywordMetadata ContactScience Contact
ExtentGeospatial Bounding BoxTemporal Start/EndVertical Min/MaxPlace Keywords
ExtentGeospatial Bounding BoxTemporal Start/EndVertical Min/MaxPlace Keywords
DistributionDistributor ContactOnline ResourceDistribution FormatData Center KeywordsBrowse Graphic
DistributionDistributor ContactOnline ResourceDistribution FormatData Center KeywordsBrowse Graphic
Text SearchesPurposeExtent DescriptionLineage StatementProject Keywords
Text SearchesPurposeExtent DescriptionLineage StatementProject Keywords
Quality/LineageSourcesProcess StepsQuality Reports / Coverages
Quality/LineageSourcesProcess StepsQuality Reports / Coverages
Acquisition InformationInstrumentPlatformInstrument KeywordsPlatform Keywords
Acquisition InformationInstrumentPlatformInstrument KeywordsPlatform Keywords
Potential Spirals
Content InformationAttribute TypeAttribute NamesAttribute DefinitionsAttribute Units
Content InformationAttribute TypeAttribute NamesAttribute DefinitionsAttribute Units
DiscoveryDiscovery UnderstandingUnderstanding
ConnectionOnlineResource:NameDescriptionFunctionApplication Schema
ConnectionOnlineResource:NameDescriptionFunctionApplication Schema
Spiral None 0-33% 34-66% 67-99% All
Identification
Connection
Extent
Distribution
Description
Content
Lineage
AcquisitionInformation
Metadata Evaluation for Dataset XDate: 2010/10/24
Spirals and Rubrics
Progress
GOALS
Each record has a score that can be calculated using a consistent, standard tool
Spiral Development / Training: Rubrics
Questions?
Identify documentation expertise1. Establish Data Stewardship Teams to facilitate documentation creation and
improvement for appropriate organizational units or around programmatic needs.
2. Data Stewardship Teams should include the following expertise/skills: • Data Collectors/Providers• Data Users• Data Stewards• Standards Experts
Line/Staff Office Responsibilities
Assess current state of documentation1. Identify existing sources of documentation (Data Collectors/Providers)2. Classify existing documentation into following categories (Data Stewards and
Standards Experts): Metadata for Discovery Metadata for Use Metadata and Documentation for Understanding Documentation of Collections Documentation of Datasets Documentation of Services
3. Identify high-priority targets for improvement (All members of Data Stewardship Team)
4. Highlight best practices, successful teams and individuals (Line and Staff Office Management)
Line/Staff Office Responsibilities
Create and Improve Metadata
1. Translate/transform existing metadata into the recommended representation (ISO 19139)
2. Create metadata for undocumented data and information3. Use spiral approach for improving metadata
Line/Staff Office Responsibilities
Publish/Preserve Metadata (Data Stewards)
1. Publish metadata record in new or existing Web-Accessible Folders or using a standard catalog service. This will make it possible to connect metadata records to various discovery portals using standard services
2. Work with NOAA Archives to ensure that documentation and metadata will be preserved for the long-term
Line/Staff Office Responsibilities
Cross-NOAA Responsibilities1. Encourage and support participation in the ISO and Open Geospatial Consortium
(OGC) standards development and evolution processes 2. Develop and implement common metadata management tools including
mechanisms for evaluating the completeness and quality of data documentation• Utilize rubrics to establish the baseline and monitor progress.• Engage users in providing feedback on data documentation efforts and
opportunities.3. Promote and highlight good examples of documentation and the individuals
involved in their creation. 4. Support training specifically targeted at improving NOAA’s data documentation.5. Initiate teams to work on “special documentation problems” that cross Line and
Staff Offices
EDMC Responsibilities1. Review this Procedural Directive twice a year to evaluate effectiveness and
monitor progress.2. Work with the CIO Council and the NOAA Observing Systems Council to implement
and monitor progress on the Cross-NOAA responsibilities listed above.3. Encourage and support partnerships with external organizations in the process of
migration of metadata from FGDC to ISO Standards.
Measure Progress
1. An increase in the amount of NOAA environmental data and information that is well documented and discoverable via national and international discovery portals.
2. Improvement in the quality of Line and Staff Office and Program data documentation processes.