icesat science investigator-led processing system (i-sips) · icesat specific software • science...

1
ICESat Specific Software Science Processing Run as independent jobs Creates science data product and performs QA functions using algorithms developed by science teams Science Team maintains science applications for life of mission Science Planning and Interfaces Define processing rules that automatically schedule jobs Provides database that tracks product history Includes ingest and distribution functions to provide data to external users Data and Archive Management Scheduling Utilities & Logging Support Science Planning & External Interfaces Science Planning & External Interfaces Science Processing Product Generation Data Ingest Data Distribution SMDS Science S/W ICESat Scheduling and Data Management System (SDMS) SDMS Provides Adaptable Infrastructure For Science Information Processing Generalized Architecture – Science and Mission Specific Code Isolated Maintains Clear Lines Between Mission Specific and Reusable Code Adapt and Tailor the Processing System to the Mission Allows Independent Development & Testing of the Science Applications Minimizes the Interface between the System and the Science Processing Code Runs Applications Written in Any Language Isolates the Production System Developers From the Science Details Data and Archive Management Data and Archive Management Scheduling Scheduling Utilities & Logging Support Utilities & Logging Support SDMS Science S/W Science Planning, Status & Ordering Science Processing Product Generation Data Ingest Data Distribution Legend Planning Execution Data Management Ingest Process Distribute Scheduling File Mgmt. Archive Mgmt. Planning Services API Fetch Store Data From External Systems Data To External Systems External Request & Status Interface External Request & Status Interface Total Re-use Part Re-use Science SDMS Architecture and Re-use Update Product Database Ingest Planning Production Planning Distribution Planning File Database Archive Database Planning Archives Scheduling Display Affiliations 1 NASA/GSFC Wallops Flight Facility Wallops Island, VA 23337 2 NASA/Goddard Space Flight Center Greenbelt, MD 20771 3 Raytheon ITSS Greenbelt, MD 20771 4 Raytheon ITSS Wallops Island, VA 23337 5 University of Texas Austin, TX 78712 Mission Context for SIPS Any mission which requires one or more of the following: generation of science data or data products through ordered processing steps archive and inventory of data/products distribution of data/products to users and external systems through subscriptions and random orders capture of data from external sources implementation of complete operational concept ICESat SIPS The ICESat Science Investigator-led Processing System (I-SIPS) is located at GSFC in Building 22. It houses the I-SIPS and GLAS Science Algorithm software teams and the I-SIPS computing facility which will run the GLAS data processing in a nearly autonomous 24x7 operation. The I-SIPS Wing of the 1 st Floor of Building 22 GSFC Building 22 Exterior The Computer Facility I-SIPS Interfaces Archive and Distribution NSIDC DAAC EDOS GLAS ISF Precision Orbit and Precision Attitude Determination UTGLAS Team Analysis/Research ICESat SCF Requests Data Products Science Data Processing/ Data Quality Assessment I-SIPS EDS Expedited Data Sets (X-band) ISF Instrument Support Facility PDS Production Data Sets (X-band) Level 0 Data Products Level 1, 2, 3 Data Prodcuts Metadata Ancillary Data Meteorological Data Level 0 Data Products (PDS) Level 0 Data Products (PDS,EDS) Level 0, 1 Data Products Calibration, Event Logs Data Products Requests Status Precision Orbit and Attitude ECS DAAC ICESat SIPS/GLAS Science Algorithm Software (GSAS) GLA00_xx (APIDs) GLAS_L0proc GLAS_L1A L1A ANC29 ANC32 L1A ALT L1A ATM L1A ENG L1A ATT GLA01 GLA02 GLA03 GLA04 GLAS_Alt GLA05 L1B Waveform/ Range Dist Assessment, Std Range Corr, POD/PAD, Inst Corr, Det Geoloc, Calc WF Characteristics L1B and 2 Elevation POD Interp, Geoid, Trop, Tides, Rough & Slope, Std Spot Loc & Elev, Reflectivity, Surface Elevation & Characteristics Ice Sheet Spot & Elev Sea Ice Spot & Elev Land Spot & Elev Ocean Spot & Elev MET_Util ANC01 ANC40 GLAS_Atm L1B and 2 Atmosphere Backscatter GLA08 GLA09 GLA10 GLA11 Boundary Layers Cross Sections Optical Depth GLA07 ATM_Anc ANC36 Create_GLA16 GLAxx GLA16 GLAS_Reader GLAxx TEXT ANCxx QAP01 QAP03 QAP04 QAP02 QAP05 QAP06 QAP12 QAP13 QAP14 QAP15 GLA15 GLA14 GLA13 GLA12 GLA06 QAP08 QAP09 QAP10 QAP11 QAP07 GSAS: GLAS Science Algorithm Software : Science Data Processing and Utilities GLAS_GPS GLA00_GPS ANC39 BROWSE QAP_xx BRWxx ICESat Spacecraft and GLAS Instrument Scheduling Data Management System Process Description Outputs GLAS_L1A (L1A) This process applies instrument related calibrations and equations to the raw GLA00 data to produce data in engineering units. It also computes precise time-tagging information for each laser shot, and quality assurance measurements. The Level 1A altimeter and atmosphere basic measurements calibrated for instrument effects (GLA01, GLA02), and the pointing and the engineering data (GLA03, GLA04). GLAS_Alt (Altimetry) This process evaluates the GLAS waveforms and computes elevation parameters. GLAS_Alt functionality is divided into waveform and elevation processes. The waveform processes compute required range and surface characteristics needed to calculate elevations. The elevation processes compute the required geophysical corrections and corrected elevations using precise pointing and location data. The Level 1B global altimeter products with precise time and location (GLA05 and GLA06), and the Level 2 regional specific altimetry products (GLA12-15). GLAS_Atm (Atmosphere) This process computes the LIDAR 532 and 1064 calibrated attenuated backscatter cross sections and determines cloud and aerosol boundary layers. Optical depths are estimated. The level 1B backscatter product (GLA07), and the atmospheric level 2 products (GLA08-11) Utilities (Utility) The utilities provide the miscellaneous information needed to process the data and to compute corrections. Various intermediate geophysical files needed for corrections and data calibration files. These are not distributed. Create_GLA16 (HDF-EOS) This process selects parameters from a number of elevation and atmosphere products, compresses some of it, and formats it into HDF- EOS. An interdisciplinary altimeter and atmosphere compressed data product of selected and filtered parameters in the HDF-EOS format. Note: See poster session G31A-06 for detailed product descriptions GLAS Science Measurement Overview Science Measurements Source Reference for: Range 1064 nm Laser Clouds 532/1064 nm Laser Aerosols 532 nm Laser Stars Laser pointing angles Rayleigh 532 nm sensitivity Scatter GPS Spacecraft x,y,z GPS Epoch Time GPS Altimeter Clock freq. GLAS Pre-Vibration Testing The Ice, Cloud and Land Elevation Satellite (ICESat) mission will measure changes in elevation of the Greenland and Antarctic ice sheets as part of NASA’s Earth Observing System (EOS) of satellites. The Geoscience Laser Altimeter System (GLAS) on ICESat has a 1064 nm laser channel for surface altimetry and dense cloud heights and a 532 nm lidar channel for the vertical distribution of clouds and aerosols. The GLAS standard data products (SDP) will be produced by the I- SIPS located at the Goddard Space Flight Facility under the direction of the ICESat project scientist following the EOS Science Investigator-led Processing System (SIPS) concept. The National Snow and Ice Data Center (NSIDC) will distribute the SDPs to the general user community. There are sixteen defined SDPs related to measurement of elevations, cloud heights, vertical structure of clouds and aerosols, vegetation canopy heights, sea ice roughness, sea ice thickness, ocean surface elevations, and surface reflectivity. The level 1 and 2 SDPs contain time ordered data stored in 15 separate products in binary integer format in direct access records. There is one defined level 3 HDF_EOS formatted product. The SDP granules vary from a quarter orbit to 14 orbits each. David W. Hancock III 1 ([email protected]); Dr. Jay Zwally 2 ([email protected]); Anita Brenner 3 ([email protected]); Suneel Bhardwaj 3 ([email protected]); John Bay 3 ([email protected]); John P. DiMarzio 3 ([email protected]); Jeff Lee 4 ([email protected]); Bob E. Schutz 5 ([email protected]) ICESat Science Investigator-led Processing System (I-SIPS) G31A-07 http://glas.wff.nasa.gov The I-SIPS software consists of two parts, the SDMS (Scheduling and Data Management System) and the GSAS (GLAS Science Algorithm System). The SDMS automatically ingests data, schedules the GSAS processing, tracks processing, archives processing logs, archives data, and distributes data. The GSAS implements the GLAS Science Team-defined algorithms to transform the GLAS raw instrument data into geolocated geophysical data products that are ready to use for scientific applications. The SDMS is applicable to any SIPS or production processing. The GSAS is specifically designed to process the GLAS data but has a core that can be useful to other systems. We will present the I-SIPS SDMS design and capabilities, the GLAS data algorithm processing design, and the resulting GLAS standard data products. SIPS Concept

Upload: others

Post on 30-May-2020

7 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: ICESat Science Investigator-led Processing System (I-SIPS) · ICESat Specific Software • Science Processing – Run as independent jobs – Creates science data product and performs

ICESat Specific Software• Science Processing

– Run as independent jobs– Creates science data product and

performs QA functions usingalgorithms developed by scienceteams

– Science Team maintains scienceapplications for life of mission

• Science Planning and Interfaces– Define processing rules that

automatically schedule jobs– Provides database that tracks product

history– Includes ingest and distribution

functions to provide data to externalusers

Data and ArchiveManagement

Scheduling

Uti

litie

s &

Lo

gg

ing

Su

pp

ort

Sci

ence

Pla

nn

ing

& E

xter

nal

Inte

rfac

esS

cien

ce P

lan

nin

g &

Ext

ern

al In

terf

aces

Science Processing• Product Generation• Data Ingest• Data Distribution

SMDS

Science S/W

ICESat Scheduling and DataManagement System (SDMS)

• SDMS Provides AdaptableInfrastructure For Science InformationProcessing– Generalized Architecture – Science

and Mission Specific Code Isolated– Maintains Clear Lines Between

Mission Specific and ReusableCode

– Adapt and Tailor the ProcessingSystem to the Mission

• Allows Independent Development &Testing of the Science Applications– Minimizes the Interface between the

System and the Science ProcessingCode

– Runs Applications Written in AnyLanguage

– Isolates the Production SystemDevelopers From the ScienceDetails

Data and ArchiveManagement

Data and ArchiveManagement

SchedulingScheduling

Uti

litie

s &

Lo

gg

ing

Su

pp

ort

Uti

litie

s &

Lo

gg

ing

Su

pp

ort

SDMS

Science S/W

Sci

ence

Pla

nn

ing

, Sta

tus

&O

rder

ing Science Processing

• Product Generation• Data Ingest• Data Distribution

Legend

Planning

Execution

Data Management

Ingest Process Distribute

Scheduling

FileMgmt.

ArchiveMgmt.

Planning Services API

Fetch Store

DataFrom

ExternalSystems

DataTo

ExternalSystems

ExternalRequest& StatusInterface

ExternalRequest& StatusInterface

Total Re-use

Part Re-use

Science

SDMS Architecture and Re-use

Update

ProductDatabase

IngestPlanning

ProductionPlanning

DistributionPlanning

FileDatabase

ArchiveDatabase

Planning

Archives

Scheduling

Display

Affiliations

1NASA/GSFC Wallops Flight Facility

Wallops Island, VA 23337

2NASA/Goddard Space Flight Center

Greenbelt, MD 20771

3Raytheon ITSSGreenbelt, MD 20771

4Raytheon ITSSWallops Island, VA 23337

5University of TexasAustin, TX 78712

Mission Context for SIPS

• Any mission which requires one or more of thefollowing:– generation of science data or data products

through ordered processing steps– archive and inventory of data/products– distribution of data/products to users and

external systems through subscriptions andrandom orders

– capture of data from external sources– implementation of complete operational concept

ICESat SIPS

The ICESat Science Investigator-ledProcessing System (I-SIPS) islocated at GSFC in Building 22. Ithouses the I-SIPS and GLASScience Algorithm software teamsand the I-SIPS computing facilitywhich will run the GLAS dataprocessing in a nearly autonomous24x7 operation.

The I-SIPS Wing of the 1st Floor of Building 22

GSFC Building 22 Exterior The Computer Facility

I-SIPS Interfaces

Archive andDistribution

NSIDC DAAC

EDOS

GLAS ISF

Precision Orbit and Precision Attitude Determination

UTGLAS

Team Analysis/Research

ICESat SCF

Requests

Data Products

Science Data Processing/ Data Quality Assessment

I-SIPS

EDS Expedited Data Sets (X-band) ISF Instrument Support Facility PDS Production Data Sets (X-band)

Level 0 Data Products

Level 1, 2, 3 Data Prodcuts

Metadata

Ancillary Data

MeteorologicalData

Level 0 Data Products (PDS)

Level 0Data Products (PDS,EDS)

Level 0, 1Data Products

Calibration, Event Logs

Data Products

Requests

Status

Precision Orbit and Attitude

ECSDAAC

ICESat SIPS/GLAS Science Algorithm Software (GSAS)

GLA00_xx(APIDs)

GLAS_L0proc

GLAS_L1AL1A

ANC29ANC32

L1A ALT

L1A ATM

L1A ENG

L1A ATT

GLA01

GLA02

GLA03

GLA04

GLAS_Alt

GLA05

L1B Waveform/ Range DistAssessment,

Std Range Corr,POD/PAD, Inst Corr,

Det Geoloc,Calc WF

Characteristics

L1B and 2 Elevation

POD Interp, Geoid, Trop, Tides,

Rough & Slope,Std Spot Loc & Elev,Reflectivity, Surface

Elevation & Characteristics

Ice SheetSpot &Elev

Sea IceSpot &Elev

LandSpot &Elev

OceanSpot &Elev

MET_Util ANC01

ANC40

GLAS_AtmL1B and 2 Atmosphere

Backscatter

GLA08 GLA09 GLA10 GLA11

BoundaryLayers Cross Sections Optical Depth

GLA07

ATM_Anc

ANC36

Create_GLA16

GLAxx

GLA16

GLAS_ReaderGLAxx

TEXT

ANCxx

QAP01

QAP03

QAP04

QAP02

QAP05 QAP06 QAP12 QAP13 QAP14 QAP15

GLA15GLA14GLA13GLA12GLA06

QAP08 QAP09 QAP10 QAP11QAP07

GSAS: GLAS Science Algorithm Software : Science Data Processing and Utilities

GLAS_GPS

GLA00_GPS

ANC39

BROWSE

QAP_xx

BRWxx

ICESat Spacecraft and GLAS Instrument

Scheduling Data Management System

Process Description Outputs

GLAS_L1A(L1A)

This process applies instrument related calibrations and equations tothe raw GLA00 data to produce data in engineering units. It alsocomputes precise time-tagging information for each laser shot, andquality assurance measurements.

The Level 1A altimeter and atmosphere basicmeasurements calibrated for instrument effects (GLA01,GLA02), and the pointing and the engineering data(GLA03, GLA04).

GLAS_Alt(Altimetry)

This process evaluates the GLAS waveforms and computes elevationparameters. GLAS_Alt functionality is divided into waveform andelevation processes. The waveform processes compute requiredrange and surface characteristics needed to calculate elevations. Theelevation processes compute the required geophysical corrections andcorrected elevations using precise pointing and location data.

The Level 1B global altimeter products with precise timeand location (GLA05 and GLA06), and the Level 2regional specific altimetry products (GLA12-15).

GLAS_Atm(Atmosphere)

This process computes the LIDAR 532 and 1064 calibrated attenuatedbackscatter cross sections and determines cloud and aerosolboundary layers. Optical depths are estimated.

The level 1B backscatter product (GLA07), and theatmospheric level 2 products (GLA08-11)

Utilities(Utility)

The utilities provide the miscellaneous information needed to processthe data and to compute corrections.

Various intermediate geophysical files needed forcorrections and data calibration files. These are notdistributed.

Create_GLA16(HDF-EOS)

This process selects parameters from a number of elevation andatmosphere products, compresses some of it, and formats it into HDF-EOS.

An interdisciplinary altimeter and atmospherecompressed data product of selected and filteredparameters in the HDF-EOS format.

Note: See poster session G31A-06 for detailed product descriptions

GLAS Science MeasurementOverview

Science Measurements

Source Reference for:Range 1064 nm LaserClouds 532/1064 nm LaserAerosols 532 nm LaserStars Laser pointing anglesRayleigh 532 nm sensitivityScatter GPS Spacecraft x,y,zGPS Epoch TimeGPS Altimeter Clock freq.

GLAS Pre-Vibration Testing

The Ice, Cloud and Land Elevation Satellite (ICESat) mission will measure changes in elevation of the Greenland and Antarctic ice sheets as part of NASA’s Earth Observing System (EOS) of satellites. The Geoscience Laser Altimeter System (GLAS) on ICESat has a 1064 nm laser channel for surface altimetry and dense cloud heights and a 532 nm lidar channel for the vertical distribution of clouds and aerosols. The GLAS standard data products (SDP) will be produced by the I-SIPS located at the Goddard Space Flight Facility under the direction of the ICESat project scientist following the EOS Science Investigator-led Processing System (SIPS) concept. The National Snow and Ice Data Center (NSIDC) will distribute the SDPs to the general user community. There are sixteen defined SDPs related to measurement of elevations, cloud heights, vertical structure of clouds and aerosols, vegetation canopy heights, sea ice roughness, sea ice thickness, ocean surface elevations, and surface reflectivity. The level 1 and 2 SDPs contain time ordered data stored in 15 separate products in binary integer format in direct access records. There is one defined level 3 HDF_EOS formatted product. The SDP granules vary from a quarter orbit to 14 orbits each.

David W. Hancock III1 ([email protected]); Dr. Jay Zwally2([email protected]); Anita Brenner3 ([email protected]); Suneel Bhardwaj3 ([email protected]); �John Bay3 ([email protected]); John P. DiMarzio3 ([email protected]); Jeff Lee4 ([email protected]); Bob E. Schutz5 ([email protected])

ICESat Science Investigator-led Processing System (I-SIPS)G31A-07 http://glas.wff.nasa.gov

The I-SIPS software consists of two parts, the SDMS (Scheduling and Data Management System) and the GSAS (GLAS Science Algorithm System). The SDMS automatically ingests data, schedules the GSAS processing, tracks processing, archives processing logs, archives data, and distributes data. The GSAS implements the GLAS Science Team-defined algorithms to transform the GLAS raw instrument data into geolocated geophysical data products that are ready to use for scientific applications. The SDMS is applicable to any SIPS or production processing. The GSAS is specifically designed to process the GLAS data but has a core that can be useful to other systems. We will present the I-SIPS SDMS design and capabilities, the GLAS data algorithm processing design, and the resulting GLAS standard data products.

SIPS Concept