ensuring credibility of nasa’s earth science data session title: in11d. data curation,...
TRANSCRIPT
Ensuring Credibility of NASA’s Earth Science Data
Session Title: IN11D. Data Curation, Credibility, Preservation Implementation, and Data Rescue to Enable Multi-source Science I
AGU Fall Meeting 2013
M. Maiden, H. K. Ramapriyan, A. Mitchell, S. Berrick, J. Walter, K. MurphyNASA Headquarters, and
NASA Goddard Space Flight CenterDecember 9, 2013
Presentation Overview
NASA’s Earth Science Data Systems Program has been working on transparency, completeness, permanence, and ease of access and use over many years.
The presentation covers:• NASA Earth Science Division Missions Current and Planned• Earth Observing System (EOS) Data and Information System
(EOSDIS)• NASA Data Policy• NASA Earth Science Data Processing and Management • NASA Earth Science Standards Process, Metadata
Architecture Enhancements & ISO 19115 Metadata Support• Data Quality, Fitness-for-Purpose and Provenance of NASA
Data• NASA readiness to work with others on interoperability and
useful Earth science data availability
Landsat-8 (USGS)
NASA Earth Science Operating Missions (2013)
3
•
••
•
•
•
•
•
•
•
•
•
••
Calendar Year 2005 2006 2007 2008 2009 2010 2011 2012+
•
•
•
•
•
Sal Island, Cape Verde
San Jose, Costa Rica
Vera Cruz, Mexico
Honolulu, HI
Anchorage, AK
Fairbanks, AK
Thule, Greenland
Portsmouth, NH
Warner Robins, GA
Houston, TX
Edwards, CA
Ponca City, OK
Mildenhall, England
Punta Arenas, Chile
Antarctica
Thailand
Svalbard, Norway
Iceland
Cedar Key, FL
Barrow, AK
•
Saskatoon Goose Bay
Tobago
Wallops, VA
Papeete, French Polynesia
YellowknifeCold Lake
•
••
Ft. Lauderdale, FL
•
•
•
•
•
••
•
•
•
•u
•
•Kiruna, Sweden
Surprise Valley, NV
Palmdale, CA
Darwin, Australia
•
Kona, HI•
Guam
2005-2012 Airborne Campaigns
SMAP2014
ICESat-22016
SWOT2020
PACE2020
(NOTIONAL)
L-Band SARNET 2021
(NOTIONAL)
CLARREONET 2022
OCO-22015
SAGE-III(on ISS) 2015
GRACE-FO2017
OCO-3(on ISS) 2017
GPM2014
CYGNSSEVM-1, 2017
TEMPOEVI-1, 2019
EVI-22020
EVM-22021
EVI-32022
NASA Earth Science Planned Missions (2014-2023)
5
EOSDIS Mission
The Earth Science Data and Information System (ESDIS) project provides access to data through the development and operation of the science systems of the Earth Observing System (EOS) Data and Information System (EOSDIS).
Data Access Policy – enabling transparency
NASA Earth Science Data Policy• No period of exclusive access• Except where agreed upon with international partners, data and derived
scientific products are available at no cost to all users • Any variation in access will result solely from user capability, equipment, and
connectivity• All NASA-generated standard products are made available (upon request)
along with the source code for algorithm software, coefficients, and ancillary data used to generate these products.
• See (http://science.nasa.gov/earth-science/earth-science-data/) for full text of the policy
Data are made available to all users promptly• After an initial checkout period• Appropriate caveats about data quality are provided in product documentation
NASA ESD fully supports the GEOSS Data Collection of Open Resources for Everyone (GEOSS Data-CORE) which is a distributed pool of documented datasets with full and open, unrestricted access at no more than the cost of reproduction and distribution.
Earth Science Mission Standard Products: Data Processing and Management
NASA’s Earth Science Division (ESD) requires each of the Earth science mission to develop and maintain a Data Management Plan (DMP) to address the management of data from Earth science missions, from the time of their data collection/observation, to their entry into permanent archives.
Instrument Principal Investigators (PIs) are responsible for science algorithms, science data processing software, product quality assessment and validationMost EOS standard products are generated at Science Investigator-led Processing Systems (SIPSs) under supervision of PIsProcessed data are delivered to EOSDIS Data Centers for archival and distributionIn a few cases standard products are generated at EOSDIS Data Centers• Science software is delivered by PIs to Data Centers• Priorities for processing and reprocessing are set by PIs
NASA Earth Science Division Approved Standards – enabling completeness
NASA has developed a process for both identification of appropriate standards and subsequent adoption for use in NASA earth science data systems.
NASA’s ESDIS Standards Office (ESO) provides standards guidance and vision:
• Facilitate the identification of standards being used across ESD missions and projects
• Provide coordination for and facilitate the use of ESDIS endorsed standards and ESDIS developed standards.
• Identify future candidate standards and facilitate the production for review.• Solicit and/or encourage submission of candidate standards, technical notes
and standards-related documents from the ESD community
Metadata standards - http://earthdata.nasa.gov/data/standards-and-references/metadata-standards
Data format standards - http://earthdata.nasa.gov/data/standards-and-references/data-format-standards
Metadata Architecture Enhancements
ESDIS initiated a study of EOSDIS metadata capabilities/processes in the spring of 2012 to evaluate opportunities for improved data access and use, reduce efforts by metadata providers, and improve metadata integrity.
The Common Metadata Repository (CMR) will manage the evolution of NASA Earth Science metadata in a unified and consistent way by providing a central storage and access capability that streamlines current workflows while increasing overall data quality and anticipating future capabilities.
Developing a Unified Metadata Model (UMM) that unifies the EOSDIS metadata assets into one cohesive model. The UMM represents the unification of the GCMD DIF, EMS and ECHO 10 Metadata formats and provides a framework to map existing metadata into broadly accepted formats such as ISO19115.
NASA ISO 19115 Metadata Support
NASA established the Metadata Evolution of NASA Data Systems (MENDS) working group to achieve consensus on implementation of ISO 19115.
Requirement for future missions: NASA’s Satellite Mission Data Systems Requirements for Science Data Management states the <<mission/instrument name(s)>> science data products metadata shall conform to ISO 19115 Geographic Information - Metadata standards
• NASA ISO 19115 Best Practices – With inputs from the ECHO and DIF metadata model mapping, we have developed the NASA ISO 19115 Best Practices convention with supporting schema and style sheets that focus on including ISO-compliant content in NASA science formats.
NASA’s Soil Moisture Active Passive (SMAP) mission is the first in a new series of missions, to employ metadata based on the ISO 19115 standard.
• Developed a toolkit of ISO reading, writing and parsing tools for initial use by SMAP mission but reusable for other missions and data systems.
NASA’s SAGE (Stratospheric Aerosol and Gas Experiment) III is the next mission slated to support ISO 19115 (also 2014 on ISS).
Preservation of NASA Data and Information Contents – enabling permanence
NASA started in 2010 a dedicated programme “Earth System Data Records Uncertainty Analysis”: estimating, validating, and conveying quantitative measurement differences between sensors, validation measurements and/or models, systematic errors in long-term Earth system data records. • Number of investigations supported - 21
Definition of Provenance/Context Content specification:• Developed and published NASA Earth Science Data
Preservation Content Specification (PSC). PSC is used by NASA Earth science missions for closeout planning and ensuring required contents are preserved
Data Quality & Fitness for Purpose – enabling ease of access and use today
NASA fully embraces the QA4EO key principle Data quality is the key solution to support the increasing number of sensors and end users, and the generation of data used for sensible applications.
ISO 19115 provides mechanisms for including comprehensive data quality in metadata which is critical for ensuring understanding and appropriate use of data.
ISO 19157 extends the Data Quality section of 19115. This technical standard defines a set of measures for the data quality sub elements identified in the ISO Geographic Information series.
The measures will be applicable when evaluating the quality of geophysical datasets and assessing their fitness for purpose. Multiple measures can be defined for each data quality sub element and the choice of which to use will depend on the type of data and its intended purpose.
Data Quality & Fitness for Purpose – enabling ease of access and use tomorrow
• Committee on Earth Observation Satellites (CEOS) Working Group on Information Systems and Services (WGISS) – NASA is providing system engineering support to the development of the CEOS WGISS Integrated Catalog (CWIC), an international community catalog of satellite data that currently provides search and access to over 1800 data products (~50 million granules) from NASA, NOAA, USGS, INPE-Brazil and CCMEO-Canada with more data providers on the way.
• US Group on Earth Observations (USGEO) Data Management Working Group – NASA is supporting the work of USGEO to facilitate earth science data set and data system interoperability across the US federal agencies and implement the data management practices described in the National Strategy for Civil Earth Observations.
• The President’s Climate Action Plan – NASA plans to support the implementation of Climate Data Initiative and the Toolkit for Climate Resilience described in the plan.
The NASA Earth Science Data Program is enabling more and more tools and connections to Earth Observation data from other sources, working as part of:
CWIC uses “Connectors” to provide translation from the CWIC (CSW) API to the agency system.
Data Quality & Fitness for Purpose – enabling ease of access and use tomorrow
NASA’s Worldview (https://earthdata.nasa.gov/labs/worldview/) is but one entry into images of NASA datasets in a GIS-viewable way. Apps have been built by third parties to the Global Imagery Browse Services (GIBS) API.
.
Opportunities exist to partner with researchers, applied scientists, information specialists, and the public to create user-friendly tools and portals.
• As an example, in the 2014-2015 timeframe, NASA will be opening up a Sea Level Rise portal for sea level change research to be accessible and useful to scientists, the general public, policy-makers, businesses, and other stakeholders.
Conclusions
NASA has been developing new tools and ways to improve ease of access and use of the data for the Earth System Science research community. NASA was in the forefront of free and open sharing of data and associated information for Earth observations. NASA utilizes standards for data format, metadata content, and required documentation for any data that are ingested into our distributed Earth Observing System Data and Information System, or EOSDIS. NASA has been improving its data management practices for over twenty years to assure permanence of data utility through reliable preservation of bits, readability, understandability, usability and reproducibility of results.NASA is increasingly focused on using the new technologies available to work with partners across the government, with the education and commercial sectors, and internationally, to crack open the power of Earth science data and information for all.
Related AGU Sessions
Poster• IN13A-1545. Ramapriyan, H. - Data Preservation – Progress in NASA’s Earth
Observing System Data and Information System (EOSDIS)
• IN31C-1509. Pilone, D. et al. - The Common Metadata Repository: A High Performance, High quality Metadata Engine for Next Generation EOSDIS Applications
• IN31C-1512. Boller, R. et al. - See it first: Interactively and Visually Discovering Interesting Satellite Data with NASA Worldview
• PA31A-1818. Behnke, J. et al. - Public Access to NASA's Earth Science Data
Oral• IN31D-02. Murphy, K. - Learning from LANCE: Developing a Web Portal
Infrastructure for NASA Earth Science Data
• IN34A-04. Mitchell, A. et al. - NASA’s Earth Observing Data and Information System – Supporting Interoperability through a Scalable Architecture
• IN43B-08. Walter, J. et al. - Evolving NASA’s Earth Science Data Systems