exeter metadata workshop may 9-12, 2006 l. steenman-clark, k. a. bouton, r. hatcher
DESCRIPTION
Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher NCAS Centre for Global Atmospheric Modelling, University of Reading, UK. NMM Overview. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/1.jpg)
Company
LOGO
Exeter Metadata WorkshopMay 9-12, 2006
L. Steenman-Clark, K. A. Bouton, R. HatcherNCAS Centre for Global Atmospheric Modelling, University of Reading, UK
![Page 2: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/2.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
NMM OverviewNMM Overview
• is an evolving metadata standard intended for the exchange of information about numerical models or codebases, and the simulations done using them.
• provides a metadata standard to describe the numerical codebase and its associated simulations, extending and refining the researcher's ability to understand how a resulting output data was produced.
• built and based on XML and associated technologies.
![Page 3: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/3.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
NMM GoalNMM Goal
• to provide a metadata standard and structure which is clear, well-defined and flexible.
• able to describe climate and forecast numerical codebases (e.g. Met Office Unified Model).
• able to describe the experiments/runs (e.g. xaabc) enabling scientists to search, retrieve, difference and compare.
• provide a better understanding of the provenance of the model data output.
• Note – presentation concentrates on NMM for an model/exp.
![Page 4: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/4.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
NMM Goals cont’dNMM Goals cont’d
• It is envisioned that the scientist would create the NMM XML file, most likely using a tool to automatically create it while, or after, the run is in process. The automatic process would 'capture' the required elements of the codebase and components being used and all the settings and values used to create the model.
• Ideally the NMM Model XML file would be submitted to the data centre with the model output data as part of the data description.
![Page 5: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/5.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
For this MeetingFor this Meeting
Interoperabilibity between frameworks will be determined by the metadata available.
• Semantic metadata describes what it really *means*. This is the type of metadata required for initial discovery and are often more descriptive, and use a standard/controlled vocabulary.
• Syntactic metadata describes what the data *looks* like. This is the type of metadata required for coupling frameworks
• From: http://marinemetadata.org/guides/metadatatypes/
![Page 6: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/6.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
NMM StructureNMM Structure
• Information Properties• Technical Properties• Numerical Properties• Science Properties• Input/Output Properties
![Page 7: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/7.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
NMM Model AttributesNMM Model Attributes
• Which NMM Version was used to create the file?
• What was the NMM creation date?
• Who was the NMM author?
• Assign a unique NMMModel_ID
![Page 8: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/8.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
Information PropertiesInformation Properties
• Purpose: To give some general descriptive information about the model such as name, description of why it was run, contact, documentation, history, references etc.
• These properties will be common for many other metadata standards and should be able to be culled directly from a NMM file.
![Page 9: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/9.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
Technical PropertiesTechnical Properties
• Purpose: To provide some minimal details of the machine properties it was run on, with an eye to creating performance statistics, or re-creating the run. What machine was it run on, type of compiler used, what libraries, optimization, code parallelisation etc
• Many of the properties are shared with coupler metadata requirements, while they have limited use for data metadata requirements.
![Page 10: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/10.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
Numerical PropertiesNumerical Properties• Purpose: To provide high level descriptive
information on the horizontal and vertical resolutions and extent of a model, and time resolution of the model.
• Semantic field in NMM• e.g “regular lat long grid”
• Semantic fields in framework metadata• PRISM: PRISM_reglonlatvrt• ESMF: ESMF_GRID_TYPE_LATLON• Gridspec : logically rectangular grid
![Page 11: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/11.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006http://www.met.reading.ac.uk/~mike/APE/
Semantic Example from APE Semantic Example from APEGroup Location Model Resoln Features
AGU for APE Japan (consortium) AFES v.1.15 T39 L48 Spectral, eulerian
CGAM Reading, UK HadAM3 N48 L30 3.75º x 2.5º grid
CSIRO Aspendale, Australia CCAM-4-12 C48 L18 ~220km conformal cubic grid
DWD Mainz, Germany GME 29.1.1 ni=64 L31 ~1º icosahedral-hexagonal grid
ECMWF Reading, UK IFS Cycle 29r2 TL159 L60 Spectral, semi-lagrangian
FRCGC Yokohama, Japan NICAM 7km L54 icosahedral grid, non-hydro.
GFDL Princeton, USA AM2p14 N72 L24 2.5º x 2º grid (IPCC)
GSFC Maryland, USA NSIPP-1 N48 L34 3.75º x 3º grid
K1-Japan Japan (collaboration) CCSR/NIES 5.7 T42 L20 s-l moisture and cloud
LASG Beijing, China SAMIL R42 L9 Spectral, eulerian
MGO St. Petersburg, Russia MGO-gcm T30 L14 Spectral
MIT Cambridge, USA MIT-gcm C32 L40 ~280km cubed sphere
NCAR Boulder, USA CCSM-CAM3 T42 L26 Spectral, eulerian
UKMO Exeter, UK pre-HadGAM1 N96 L38 1.875º x 1.25º grid, s-lagrangian
Reg lat long grid
![Page 12: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/12.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
Science PropertiesScience Properties
• Purpose: To provide details on the science parameters and setting used to create the model
![Page 13: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/13.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
Input/Output PropertiesInput/Output Properties
• Purpose: To provide details on the initial conditions, boundary conditions, physical constants used to create the model, and point to the output data files (which have their own metadata e.g. CF)
![Page 14: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/14.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
NMM and the IPCC ModelsNMM and the IPCC Models
• IPCC model documentation extremely varied. Was it standardizable?
• Created a NMM XML file for each IPCC Climate Model Documentation and recreated table based on the standard
• Table enabled cross comparison
![Page 15: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/15.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
IPCC ExampleIPCC Example
View Table create by XSL from the many model NMM XML files
![Page 16: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/16.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
To DoTo Do
• Numerical Properties – describing grid specs a la wondergrid
• Can we create a PRISM XML file from NMM XML – what would be missing?
• Apply to other models
• Tools such as a personal notebook
• Get the word out
![Page 17: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/17.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
Information Properties DetailsInformation Properties Details
localNamelongNamedescriptionchangeHistorycontactsreferencesresourceConstraintsdescriptiveKeywordsbrowseGraphics
![Page 18: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/18.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
Technical Properties DetailsTechnical Properties Details
machine - machineName, descriptioncompiler - name, version, descriptionoptimizationlibrariesdateCompiledchangeHistoryenvironmentVariablescode_parallelisation – processorsUsed,
haloSize
![Page 19: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/19.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
Numerical Properties DetailsNumerical Properties Details
HorizontalRepresentationcoverageschemeType spectral, grid,
mixed, isohedral etcresolution longitude (longitudeDegrees,
extent, description) latitude (latitudeDegrees,
extent, description)northPolechangeHistory
VerticalRepresentationresolution- numberOfLevelscoverageextentformulaValueschangeHistory
TimeIntegrationResolution - timeStepsPerDaychangeHistory
![Page 20: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/20.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
Science Properties DetailsScience Properties Details
standardNamelocalNamedefinitionmodifiedFromStandardreferencesparameter localName value perturbed modifiedforcing internal (or) external (or)componentchangeHistory
![Page 21: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/21.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
Input/Output Properties DetailsInput/Output Properties Details
input(s) initialCondition mode – internal,
external or component
description initialConditionType perturbed changeHistory references
physicalConstant(s) standardName localName value references
boundaryCondition description
boundaryConditionType
references boundaryDataset
CouplingRequirements - PMIOD
output outputFile_ID outputFileFormat description changeHistory references
![Page 22: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/22.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
PRISM AD.dtdPRISM AD.dtdPRISM/OASIS NMM
application NMMModel
local_name
long_name
start_mode
coupling_mode
nbr_procs
platform
component->local_name
component->long_name
component->nbr_procs
Information-localName
Information-longName
Could be in NMM Component Technical Properties
TechnicalProperties-machine-machineName
Plus more-compilers,options,libraries,history etc
NMMComponent_ID localIdentifier
See above
![Page 23: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/23.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
PRISM SCC.dtdPRISM SCC.dtd
SCC NMM
experiment
driver->nbr_proc
start_date
end_date
run with multiple applications/components
NMMModel
Could be in Technical Properties
Numerical Properties
another NMMModel with multiple components
![Page 24: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/24.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
PRISM PMIOD.dtdPRISM PMIOD.dtdPRISM/OASIS PMIOD NMMCodebase
code->contact
code->documentation
Fortran_Units
Gridfamily
physical space/sampled space
compute_space
Transient local/standard
units,datatype,computation
intent
contacts
references
TechnicalProperties-codingLanguage
HorizonalRepresentation-schemeType-grid
ScienceProperties standard and local scheme and parameters
![Page 25: Exeter Metadata Workshop May 9-12, 2006 L. Steenman-Clark, K. A. Bouton, R. Hatcher](https://reader035.vdocuments.site/reader035/viewer/2022062423/56814871550346895db57cc1/html5/thumbnails/25.jpg)
NCAS CGAM, Exeter Metadata Workshop, May 9-12 2006
PRISM SMIOC.dtdPRISM SMIOC.dtdPRISM/OASIS SMIOC NMMModel
code->contact
code->documentation
Fortran_Units
Gridfamily
physical space/sampled space
compute_space
transient local/standard
units,datatype etc
intent
contacts
references
TechnicalProperties-codingLanguage
NumericalProperties-HorizonalRepresentation-schemeType-grid
ScienceProperties standard and local scheme and parameter and value