daac status - national snow and ice data center · managing 729 data sets as of may 2018 data...

NSIDC UWG – 12-13 September 2018

DAAC Status

Thank you for coming and providing your feedback!

New members: Lawrence Mudryk, Environment and Climate Change

Canada

Eric Sproles, Montana State University

Mark Raleigh, NSIDC/CU

New DAAC contract was awarded as of 1 August 2018 5 years (Base year plus 4 option years)

Ends 31 May 2023

Value of ~$50M

No significant changes in the Statement of Work

Managing 729 data sets as of May 2018

Data distribution increased 140%from 180 TB/year in Contract Year 1 (CY1) to 432 TB/year in CY5

User community increased 76%from 18,963 users/year in CY1 to 33,672 users/year in CY5

Total1 June 2013 to 31 May 2018

Volume Distributed

(PB)Files Distributed Total Users User Requests

1.4 564,900,650 95,626 14,192

Annual: 1 June to 31 May

2013/2014

2014/2015

2015/2016

2016/2017

2017/2018

Volume Distributed (TB) 179.94 205.54 201.99 392.92 432.49

Files Distributed (millions) 76 100 110 117 162

Total Users 18,963 22,296 22,497 38,361 33,672

ACSI Score 75 78 77 76 79

User Requests 2385 3011 3557 3163 2076

Archive volume increased by 400% from 123 TB in 2013 to 627 TB in 2018

Ingest, archive and distribute Earth Science data designated for the EOSDIS Snow and Ice Distributed Active Archive Center using the EOSDIS infrastructure systems

Provide operations support for acquiring, processing, archiving and distributing data for new and future missions as well as supporting/validating data from in-situ sources. Provide preservation content to ensure long term stewardship of NASA data assets.

Meet needs of diverse community of users. Ensure data is archived and served in a way that facilitates its use. Recommend to NASA data formats or additional data products that might better serve community needs.

Provided leadership in the cryospheric community on data formats, strategic services, data interoperability, data usability, and data capabilities.

NSIDC Director

Management Team

Service Groups

Science Communications(15)

User ServicesTech Writing

WebOutreach

IT Services(13)

Sys AdminOpsDBA

Security

Software Development

(13)

Software DevScience ProgArchitecture

Programs and Projects

(12)

NASA DAAC

NSF, NOAA, Others

Science Team(23)

Administrative Staff

DAAC ManagerAmanda Leon

DAAC Scientist Walt Meier

Systems EngineerChris Bond

Data StewardshipShannon Leslie

Project ManagersSteve TannerDonna Scott

Renea EricsonKara Gergely

DeputyKate Heightley

RaytheonMark SchwabFrank Schaffer

Rich Lutes

Mission Support Engineering

OperationsUser Support

Administration

Infrastructure

DAAC Manager

DAAC Project Managers

Service Group Managers

Deputy DAAC Manager

Data management of satellite, airborne and in-situ data

Data acquisition and processing

Coordination with enteral interfaces: Missions, Science Teams, Science Investigator-led Processing Systems

Data system configuration and operation for missions

Science support for mission data users



Administration

Infrastructure

Planned

ICESat-2 JPSS-2 MEaSURES 2017

Active

AMSR-2 MODIS/Aqua MODIS/Terra SMAP SSM/I-SSMIS SNPP VIIRS JPSS-1 VIIRS Airborne Snow

Observatory HMA IceBridge SnowEx MEaSUREs 2012 PARCA Program for Arctic

Regional Climate Assessment

Completed

AMSR AMSR-E* Aquarius* GLAS/ICESat Nimbus* SMMR TOVS MEaSUREs 2007 CLP-X Cold Land Processes

Experiment

SCP Scatterometer Climate

Record Pathfinder

* Mission completed but on-going data processing and ingest

Mission Support Summary

AIRBORNE

SATELLITE

FIELD

•Operations

•Developer

•User Services

•Scientist

•Technical Writers

•Scientist

•Operations

•Developer

•Scientist

Acquire or

ProduceDescribe

DistributeSupport

Product Team• Cross-functional team of staff

from service groups

• DAAC SMEs advise on DAAC

and EOSDIS best practices

and considerations

• Product teams established

around data expertise: VIS/IR,

airborne, passive microwave,

LIDAR

Mission

Support

Managed By

DAAC Project Manager

Work Category

Program management

Financial management

Staff supervision



Administration

Infrastructure

Develop and maintain applications for science data users and data management operations

Guide overall system architecture to align with internal and external (EOSDIS) standards

Integrate applications with EOSDIS core components (e.g., Earthdata Login, OPeNDAP)



Administration

Infrastructure

Data set Landing Page

IceBridgePortal and Subsetting

Cumulus Core

Satellite Observations

of Arctic Change

•Operations

•Developers

•Sys Admins

•User Services

•Scientist

•UI/UX

•Developers

•Web designer

•UI/UX

•Product Owner

•User Services

•Scientist

Define Build

ReleaseEvaluate

Project Team

Engineering

Managed By

DAAC Project Manager

Work Category

• Cross-functional team of staff

from service groups

• DAAC SMEs advise on cross-

DAAC and EOSDIS

architecture and considerations

• Use LEAN and Agile

methodologies

Provide support for DAAC users (non-mission specific) through a range of services (e.g., users requests, “How Tos”, FAQs)

Understand and integrate user needs into DAAC activities Maintain Web site as a channel for directing users to data

access, information, tools, and services User outreach via Sensing our Planet, social media, and

Earthdata site and webinars



Administration

Infrastructure

Provide overall systems administration of computing and data storage infrastructure used to

Ingest, archive, and distribute DAAC data

Build and maintain Web site

Develop and deploying data service applications

General day-to-day business activities.

Maintain facilities including hardware maintenances and upgrades

Protect the infrastructure through IT security



External

Coordination

Infrastructure

Operate all DAAC data systems, including daily data ingest and distribution

Monitor and resolve or report system problems

Install and test data system components and upgrades

Implement and maintain EOSDIS core components (e.g., Earthdata Login, OPeNDAP)

Perform metadata management and export

Manage data archives to ensure integrity and completeness



Administration

Infrastructure

Work Categories

Science Communications

Service Group

User Support

WebOutreach

Writers

User Support Infrastructure Operations

Managed By

Service Group Managers

IT Services

Service Group

System Admins

OperationsSecurity

DBA

UWG Prioritization of DAAC Initiatives

Highest: Data access on data

set landing pages

Use UWG input to improve user

workflows

Feedback from 2017 UWG,

Blink study, & testing today

Increase UWG membership

Added 3 new members

Joint UWG meeting with other DAACs

Need to identify theme and

engage DAACs early

Adjustments

Earthdata Cloud 2021

NSIDC DAAC migration to the cloud-based data system (Cumulus) put on hold

ESDIS reprioritized in March to focus on preparing for SWOT and NiSAR

NSIDC DAAC now part of team developing Cumulus Core

Data access projects

SSM/I-SSMIS subsetting and reformatting services project put on hold to focus on data access workflows on landing pages

Needed to resolve key workflow issues before creating more transformation services

Reflects UWG prioritization

Accomplishments

Demonstrated ICESat-2 operational readiness through participation in ground systems tests and mission reviews (FOR,ORR)

Established ICESat-2 data transformation services

Spatial & Parameter Subsetting

Reformatting

Expanded IceBridge prototype point-cloud and waveform subsetting (ATM, LVIS)

Created file-level metadata for high value SSM/I-SSMIS data sets to enable richer data discovery and access

Accomplishments

First product releases for Airborne Snow Observatory and High Mountain Asia

Significant data releases for SnowEx, SMAP, MEaSURES and IceBridge

Added SMAP Soil Moisture, Freeze/Thaw and Carbon data sets to AppEEARS Application for Extracting and Exploring Analysis Ready Samples – LP DAAC

Creates spatial subsets of a time series for points and polygons

Released the Data Map Services API Provides polar geo-registered map images using OGC protocols

Updated data set Level of Service model

Accomplishments

• Total: 147 data sets

• Average/month: 13.4 data sets

• Increase of 41 data sets from FY17 (& the FY isn’t over…)

AMSR-E, 1 AMSRU, 4Aquarius, 1

ASO, 3

HM…

IceBridge, 46

MEaSUREs, 35

MODIS, 0

SMAP, 15

SMAPVex, 13

SnowEx, 8

SSM/I-SSMIS, 13

V0/Other, 3 VIIRS, 1

New/updated data sets published during Oct 2017 – Aug 2018

Themes

User workflows: improve data and information

workflows to remove barriers to science

User engagement: evolve user engagement and

support to meet the current needs of data users

Leverage cloud: proactively identify how we can

enable and support current and potential science applications/use cases

The goal of moving NASA Earth science data to the cloud (AWS) is to enable more scalable, open science

Will be evaluating which data have value in being in the cloud

Will identify science use cases which could benefit from the cloud

Break existing

workflows

CheaperEverything

in the cloud

Major Activities

Supporting ICESat-2 data usage and continuing to understand and respond to community needs

Major data releases for VIIRS, SnowEx, ASO, High Mountain Asia, and SMAP

Web site data access workflow improvements

Participate in ESDIS Cumulus Core development

ICESat-2

• Supporting SDT and users post launch

Passive Microwave

• Evaluating alternative input sources

IceBridge

• OIB flights will continue through mid-2020

SnowEx

• 2nd campaign in 2019

Mission Support

ASO

• Additional data sets and campaign years

High Mountain Asia

• Potential for up to ~60 data sets

SMAP

• New version in spring 2019

AMSR-Unified

• Additional AMSR-U data sets

Mission Support

Data set Landing Page

• Improved workflows for downloading data

IceBridge

• Continue OIB Portal and subsetting efforts

Cumulus Core

• Involved in development of Cumulus Core (NASA cloud data system)

OpenAltimetry

• Visualization of laser altimeter data developed by SDSC (NASA ACCESS)

Tools and Services for Users

daac status - national snow and ice data center · managing 729 data sets as of may 2018 data...

Documents