ipopp 2.3 users guide

International Polar Orbiter Processing Package (IPOPP)

User's Guide

Version 2.3

April 4, 2014

GODDARD SPACE FLIGHT CENTER GREENBELT, MARYLAND

Version 2.3 Page i April 4, 2014

Table of Contents

1. General ...................................................................................................................... 1

2. Purpose ...................................................................................................................... 1

3. Software Version ........................................................................................................ 1

4. IPOPP Installation ...................................................................................................... 2

5. IPOPP Operation........................................................................................................ 3

6. IPOPP Tools .............................................................................................................. 9 6.1 IPOPP Dashboard ................................................................................................ 9 6.2. Status/Event Logging System Console .............................................................. 11 6.3. Pass Manager ................................................................................................... 16

7. Installing/Updating/Configuring SPAs ....................................................................... 18

8. Diagnosing and Recovering from IPOPP Operational Errors .................................... 19

Appendix A. System Requirements ............................................................................ A-1

Appendix B. Science Processing Algorithms and Products ........................................ B-1

Appendix C. Location of Ancillaries and Data Products .............................................. C-1

Version 2.3 Page 1 April 4, 2014

1. General

The International Polar Orbiter Processing Package (IPOPP) processes science data and derivative products from the Suomi National Polar-orbiting Partnership (SNPP), Aqua, and Terra missions and, in the future, the Joint Polar Satellite System (JPSS) mission. IPOPP was initially designed for real-time science data processing but has evolved into a more general data processing package equally suited for both real-time and non-real-time science data processing. IPOPP's real-time data processing capability maximizes the utility of Earth science data for real-time applications and decision-making. Please direct any comments or questions regarding this software to the DRL via the "Contact DRL" mechanism at the DRL Web Portal: http://directreadout.sci.gsfc.nasa.gov/?id=dspContent&cid=66

2. Purpose

This document provides instructions for installing and operating the IPOPP software. IPOPP can ingest SNPP Raw Data Record (RDR) files and Terra/Aqua Production Data Set (PDS) files. It automates multi-level processing of the ingested RDR/PDS files and produces SNPP Sensor Data Records (SDRs), SNPP Environmental Data Records (EDRs), and Terra/Aqua Level-1 and Level-2 products. For a list of science algorithms and associated products included in this package, refer to Appendix B, "Science Processing Algorithms and Products." IPOPP can process both real time sensor data (e.g., direct broadcast downlink), and non-real time sensor data (e.g., downloaded from global processing archives). A Front End System (FES) set up to receive raw sensor data from a direct broadcast downlink can serve as a real-time source of SNPP RDR files and/or Terra/Aqua PDS files. The FES is not part of this distribution; however, FES software technologies (e.g., RT-STPS and Simulcast), along with IPOPP and SPAs, are available at the DRL Web Portal for use by the commercial sector. A description of IPOPP architecture can be found at: http://directreadout.sci.gsfc.nasa.gov/?id=dspContent&cid=68

3. Software Version

IPOPP Version 2.3 was developed by the Direct Readout Laboratory (DRL), NASA Goddard Space Flight Center. This version includes the following enhancements:

A new beta version Dashboard GUI improves the user experience significantly for chain process configuration and process monitoring. Graphical depiction of data handling agents, Science Processing Algorithms (SPAs) and their input/output relationships makes configuration easy and intuitive.

C-SDR_SPA has been updated to v1.6 (Mx8.0), and VIIRS-AF_SPA has been updated to v1.2.9.

JPSS GTMImagery SPA (w/Near Constant Contrast) and MODIS Burnscar SPA are included in IPOPP for the first time.

VIIRS SPAs have been updated to provide improved support for compressed and/or chunked HDF5 input files. This will result in significant reduction of SPA execution times when processing these input files.

The User’s Guide has been updated to describe operation of the beta version Dashboard GUI and other new features in Version 2.3, which is in direct response to user request.

Copyright 1999-2007, United States Government as represented by the Administrator for the National Aeronautics and Space Administration. All Rights Reserved.

http://directreadout.sci.gsfc.nasa.gov/?id=dspContent&cid=66



4. IPOPP Installation

Step 1: Download the IPOPP software package

a) Users must be registered with DRL to download IPOPP. New users must complete the "MyDRL Registration Form" at http://directreadout.sci.gsfc.nasa.gov/?id=registration to request an account.

b) Go to: http://directreadout.sci.gsfc.nasa.gov/?id=software and select "IPOPP" from the list of available packages. Enter your registered email address and password and follow the specific download and installation instructions, provided on the Web Portal, to download IPOPP.

Step 2: Setup the computer (new users only; skip to Step 3 if you are updating IPOPP)

A system administrator logged in as 'root' must complete the following instructions.

a) Ensure the target computer meets the system requirements stated in Appendix A, "System Requirements"

b) Install and configure MySQL as required.

MySQL must be installed and configured to execute when the operating system boots. MySQL must be executing with the root password set to "b28c935". To initially set the root password, enter:

mysqladmin -u root password b28c935

To change an existing root password, enter:

mysqladmin -u root password -p b28c935

A password prompt will appear for the user to enter the existing password. Type in the existing password and press Enter.

c) Create a user account with standard privileges for IPOPP.

This is the account where IPOPP will be installed and executed. This user will operate IPOPP.

d) Create the /raid directory with correct permissions. The IPOPP user must own and have read/write access to /raid.

Step 3: Install IPOPP

The following steps must be completed by the IPOPP user. The instructions contained in this User's Guide assume that $HOME is the home directory of the IPOPP user account (e.g., /home/ipopp).

a) Log in as the IPOPP user.

b) rm -r $HOME/IPOPP [delete the $HOME/IPOPP directory, if it exists]

c) tar -C $HOME -xzf DRL-IPOPP_X.Y.tar.gz [Untar tar file obtained in Step 1 into $HOME]

Execution takes several minutes and creates $HOME/IPOPP.

d) rm DRL-IPOPP_X.Y.tar.gz [Optionally delete the source tar file]

e) chmod -R 755 $HOME/IPOPP [Change $HOME/IPOPP permission]

f) cd $HOME/IPOPP [Change directory]

g) Close open IPOPP GUI tools [Prepare to uninstall existing IPOPP]

h) ./uninstall_ipopp.sh [Uninstall existing IPOPP]

http://directreadout.sci.gsfc.nasa.gov/?id=registration

http://directreadout.sci.gsfc.nasa.gov/?id=software


Confirm your intent to uninstall by typing 'YES' at the prompt.

NOTE: IPOPP uninstallation will delete all IPOPP-related directories including all data products. Copy data products that you wish to retain to another location before you uninstall IPOPP.

Wait for the message "IPOPP X.Y uninstallation complete" to ensure uninstallation is complete.

i) ./install_ipopp.sh [Install IPOPP from $HOME/IPOPP]

Installation takes several minutes. Wait for the message "IPOPP installation complete" to ensure that installation is complete. You are now ready to start IPOPP processing. Refer to Section 5, "IPOPP Operation."

5. IPOPP Operation

The following steps describe the steps required to operate IPOPP. IPOPP tools are also described in more detail in Section 6, "IPOPP Tools." Step 1: Configure IPOPP using the IPOPP Dashboard

a) $HOME/drl/tools/dashboard.sh [Open IPOPP Dashboard]

The IPOPP Dashboard has two modes: Configuration Editor Mode (see Figure 1a) and Process Monitor Mode (see Figure 1b). The Dashboard opens in the Process Monitor Mode when first invoked. IPOPP configuration is allowed only in the Configuration Editor Mode.

b) Click "Mode > IPOPP Configuration Editor" [Switch to Configuration Editor Mode]

Confirm the stopping of all SPA services on the dialog box that appears. SPA services will be stopped to enter Configuration Editor Mode. Each square box on the Dashboard (see Figure 1a) represents an SPA station. The flow lines represent the hierarchical relationships among the SPA stations. An SPA station is an IPOPP processing agent responsible for generating a subset of the output products available from the SPA as a whole. An SPA may thus have multiple stations. Each SPA Station on the Dashboard is labeled; the label includes the SPA name to associate the station to its source SPA. Information on data products available from each SPA station is available by clicking on the Product Information button labeled "i" on the SPA Station. SPA stations are distributed in three tabs: (i) the "EOS" tab has stations that process data from Terra/Aqua MODIS; (ii) the "SNPP-VIIRS" tab shows stations that process SNPP VIIRS data; and (iii) the "SNPP-ATMS/CrIS/OMPS" tab shows SPA stations that process SNPP ATMS, CrIS and OMPS data.

c) Enable/Disable SPA stations [Edit IPOPP Configuration]

Each SPA Station on the Dashboard acts as a toggle button to allow users to enable or disable it. As you mouse-over each SPA Station, the immediate upstream and downstream SPA stations are highlighted. Note that the immediate upstream stations may depend on their own upstream stations, which in turn can have their own prerequisites. In order to correctly enable a station you will need to follow the dependencies all the way up the processing chain and enable all predecessor stations.

NOTE: VIIRS_C-SDR and VIIRS-SDR are mutually exclusive and therefore should not be enabled simultaneously.

NOTE: In order to balance resource requirements, it is recommended to perform Aqua/Terra MODIS and SNPP instrument processing on separate IPOPP installations.


Figure 1a. IPOPP Dashboard: Configuration Editor Mode (refer to Table 3 in Section 6.1 for detailed descriptions)


Figure 1b. IPOPP Dashboard: Process Monitor Mode (refer to Table 3 in Section 6.1 for detailed descriptions)


Step 2: Start IPOPP Services

a) Click "Mode> IPOPP Process Monitor" [Switch to Process Monitor Mode]

Click 'Yes' to save the Configuration just edited, and press 'OK' on the "IPOPP configuration saved" prompt. The display will be synchronized before entering the Process Monitor Mode. IPOPP utilities including the one that starts IPOPP services are only available in the Process Monitor Mode. At this point all SPA stations (including the ones that were enabled in Step 1) should report their status as "OFF".

b) Click "Actions> Start Services" [Start IPOPP Services]

Confirm starting of IPOPP services. A progress bar dialog box shows progress until all services are started. All SPA stations enabled in Step 1 should now appear as "ON" on the Dashboard Window. IPOPP Services include SPA services, Ancillary Retrieval and Registration Services, Ingest Services, File Management Services, Logging Services, and Database Maintenance Services. "Network Unreachable" or other transient error messages may be written to the SLS Console until the IPOPP Services completely initialize.

c) Wait about 2 minutes [Allow time for retrieval of ancillary files]

This is required only if you are starting IPOPP services for the first time (i.e., after IPOPP installation or IPOPP reset).

NOTE: The following IPOPP tools are available to monitor IPOPP activity:

i. "Actions>Check IPOPP Services"

Check IPOPP Services is available when the Dashboard is in Process Monitor mode. When clicked, a dialog box will report IPOPP status. If some services are reported as not running, use the Actions>Start Services option to start all services. If the utility reports an error, refer to Section 6.1, "IPOPP Dashboard," for more details.

ii. SLS Console

SLS Console opens up along with the Dashboard (SLS Console may be hidden behind the Dashboard). Logs will appear on the SLS Console showing all IPOPP activity. Refer to Section 6.2, "Status/Event Logging System" for more details.

iii. Dashboard (in Process Monitor Mode)

Switching between Dashboard Modes is available through the Mode menu item (refer to Section 6.1, "IPOPP Dashboard"). SLS Console must be in LIVE Mode with all filtering disabled to view real-time activity on the Process Monitor (refer to Section 6.2, "Status/Event Logging System").

Step 3: Ingest Data

a) Click "Actions>Check IPOPP Services" [Ensure all IPOPP services are running]

Check IPOPP Services is available when Dashboard is in Process Monitor mode. A dialog box will report IPOPP status. If some services are reported as not running, use the Actions>Start Services option to start all services. If the utility reports an error, refer to Section 6.1, "IPOPP Dashboard," for more details.

b) Skip to Step c) if you are processing real-time instrument data.

For non-real-time processing of historical instrument data, users will need to ingest the required ancillaries. The DRL maintains a Compressed Ancillary Archive at: ftp://is.sci.gsfc.nasa.gov:/CompressedArchivedAncillary. Each Ancillary Archive tar file contains ancillaries spanning an 11-day temporal period. The start time of the temporal period is encoded into the tar file name as follows: DRLAncillary_YYYY-MM-DD.tgz. For example,

ftp://is.sci.gsfc.nasa.gov/CompressedArchivedAncillary


DRLAncillary_2008-01-17.tgz spans 2008-01-17 thru 2008-01-27

i. Identify the ancillary tar file(s) that are required for processing.

Identify the ancillary tar file that encompasses the instrument data start time as well as the ancillary tar files that immediately precede and follow it. (Naming conventions for RDR/PDS files are shown in Tables 1 and 2). For example, to process data from 2008-01-20, you would need: DRLAncillary_2008-01-17.tgz, DRLAncillary_2008-01-07.tgz and DRLAncillary_2008-01-27.tgz.

ii. Download and place ancillary tar file(s) in /raid/pub/CompressedArchivedAncillary

You may use the wget utility to download the tar file(s):

wget ftp://is.sci.gsfc.nasa.gov:/CompressedArchivedAncillary/DRLAncillary_YYYY-MM-DD.tgz

iii. $HOME/drl/reprocessing/ingest-ancillaries.sh [Ingest ancillary tar file]

Wait for the message "Archived Ancillary files registration complete" to ensure that ancillary registration is complete. Ancillaries ingested from archive tar files using the ingest-ancillaries.sh script are always retained (unless IPOPP is reset). Real time ancillaries are not retained beyond 14 days. Refer to Appendix C, "Location of Ancillaries and Data Products," to locate the registered ancillaries.

iv. rm /raid/pub/CompressedArchivedAncillary/*.tgz [Delete ancillary tar file(s)]

This step will prevent re-registration of the same ancillaries in the future.

c) Place the SNPP RDR file or Terra/Aqua PDS (packet file and CSR) file pair in /raid/dsm/nisfes_data

Tables 1 and 2 show the RDR/PDS files and naming conventions required for IPOPP ingest. Field Terminal users can set up their Front End System (FES) to automatically push SNPP RDR files or Terra/Aqua PDS file pairs into this directory as they become available. Other users can initiate processing by manually placing the required files into this directory. Any automatic or manual method must conform to the recommendations noted in "Important Instructions" below. SNPP RDR files and Terra/Aqua PDS file pairs placed into this directory are automatically discovered by IPOPP, which then proceeds to register and subsequently process them. If the registration step fails, IPOPP will move the file(s) to the /raid/dsm/nisfes_data/FAILED subdirectory. All processing activity will be logged on the SLS Console if it is running in LIVE Mode. Processed data products can be found under /raid/pub/gsfcdata. Refer to Appendix C, "Location of Ancillaries and Data Products."

Important Instructions: o The recommended method to transfer SNPP RDR files and Terra/Aqua PDS (packet file

and CSR) file pairs to the /raid/dsm/nisfes_data subdirectory is to first download the RDR files and PDS file pairs to a different staging directory on the same disk as the /raid/dsm/nisfes_data/ directory. Once the files are fully downloaded, use the 'mv' command to move them to the /raid/dsm/nisfes_data/ directory. While transferring PDS file pairs, make sure to transfer the larger packet file to /raid/dsm/nisfes_data/ before the smaller CSR file. Following these instructions will protect against IPOPP prematurely registering an incompletely transferred file.

o SNPP RDR files and Terra/Aqua PDS (packet file and CSR) file pairs must have mission-

compliant file names (as shown in Tables 1 and 2) in order to be correctly registered and processed by IPOPP. The CRECBuilder package can be used to rename nonconforming Aqua/Terra packet files and generate CSRs from packet files where CSRs are unavailable. The CRECBuilder software package is available at: http://directreadout.sci.gsfc.nasa.gov/?id=software




Table 1: SNPP RDR Files (and Naming Conventions) Required for IPOPP Ingest

Sensor RDR files required for IPOPP ingest

Required Naming convention

VIIRS VIIRS RDR RNSCA-RVIRS_npp_dYYYYMMdd_thhmmssS_ehhmmssS_*.h5

ATMS ATMS RDR RATMS-RNSCA_npp_ dYYYYMMdd_thhmmssS_ehhmmssS _*.h5

CrIS CRIS RDR RCRIS-RNSCA_npp_ dYYYYMMdd_thhmmssS_ehhmmssS _*.h5

OMPS OMPS ROTCS and RONPS RDRs

RNSCA-ROTCS_npp_ dYYYYMMdd_thhmmssS_ehhmmssS _*.h5 RNSCA-RONPS_npp_ dYYYYMMdd_thhmmssS_ehhmmssS _*.h5

NOTE: 'YYYYMMdd' represents the date of the start of the swath (YYYY: 4 digit year; MM: month; dd: day of month). The first and the second 'hhmmssS' represent the start and end of swath, respectively (hh: hour; mm: minutes; ss: seconds; S: 10

th of a second).

Table 2: EOS PDS/CSR File Pairs (and naming conventions) required for IPOPP Ingest

Sensor PDS/CSR file pairs required for IPOPP

ingest

Required Naming convention

Terra MODIS

Terra MODIS packet file Terra MODIS CSR file

P0420064AAAAAAAAAAAAAAyyDDDhhmmss001.PDS P0420064AAAAAAAAAAAAAA yyDDDhhmmss000.PDS

Aqua MODIS

GBAD Packet file GBAD CSR file Aqua MODIS Packet file Aqua MODIS CSR file

P1540957AAAAAAAAAAAAAAyyDDDhhmmss001.PDS P1540957AAAAAAAAAAAAAAyyDDDhhmmss000.PDS P1540064AAAAAAAAAAAAAAyyDDDhhmmss001.PDS P1540064AAAAAAAAAAAAAAyyDDDhhmmss000.PDS

NOTE: yyDDDhhmmss' represents the date/time of the start of the swath (yy: 2-digit year; DDD: Julian day; hh: hour; mm: minutes; ss: seconds).

NOTE: SNPP RDR and Terra/Aqua PDS test data are available at the following locations:

VIIRS: ftp://is.sci.gsfc.nasa.gov/gsfcdata/npp/viirs/level0/ ATMS: ftp://is.sci.gsfc.nasa.gov/gsfcdata/npp/atms/level0/ CrIS: ftp://is.sci.gsfc.nasa.gov/gsfcdata/npp/cris/level0/ OMPS ftp://is.sci.gsfc.nasa.gov/gsfcdata/npp/omps/level0/ Terra MODIS: ftp://is.sci.gsfc.nasa.gov/gsfcdata/terra/modis/level0/ Aqua MODIS: ftp://is.sci.gsfc.nasa.gov/gsfcdata/aqua/modis/level0/ Aqua GBAD: ftp://is.sci.gsfc.nasa.gov/gsfcdata/aqua/gbad/

Step 4: Autonomous IPOPP Processing

As long as IPOPP services are running, IPOPP will autonomously discover raw sensor data, retrieve ancillaries from external ancillary repositories, register data/ancillaries in its database, manage resource requests from running SPAs, schedule SPA executions, produce and register outputs, and manage the IPOPP file system. Users can monitor all activity using the SLS Console (in LIVE Mode) and the Dashboard (in Process Monitor Mode). Step 5: Stop IPOPP Services

As long as you are ingesting and processing data, there is no need to stop IPOPP services. However, IPOPP services can be stopped from the IPOPP Dashboard in its Process Monitor Mode:

Click "Actions>Stop Services" [Stop IPOPP Services] Confirm stopping of IPOPP services. A progress bar dialog box shows progress until all services are stopped. Return to Step 1 to restart IPOPP processing.

NOTE: Some or all of IPOPP's services will be stopped when (i) restoring IPOPP health to recover

ftp://is.sci.gsfc.nasa.gov/gsfcdata/npp/viirs/level0/

ftp://is.sci.gsfc.nasa.gov/gsfcdata/npp/atms/level0/

ftp://is.sci.gsfc.nasa.gov/gsfcdata/npp/cris/level0/

ftp://is.sci.gsfc.nasa.gov/gsfcdata/npp/omps/level0/

ftp://is.sci.gsfc.nasa.gov/gsfcdata/terra/modis/level0/

ftp://is.sci.gsfc.nasa.gov/gsfcdata/aqua/modis/level0/

ftp://is.sci.gsfc.nasa.gov/gsfcdata/aqua/gbad/


from an undesired state (refer to Section 6.1, "IPOPP Dashboard"); (ii) installing/updating/configuring an SPA (refer to Section 7, "Installing/Updating/Configuring SPAs"); (iii) resetting IPOPP (refer to Section 6.1); (iv) enabling/disabling an SPA that was previously disabled/enabled (refer to Section 6.1). If you perform any of these operations, return to Step 1 to restart IPOPP processing.

6. IPOPP Tools

6.1 IPOPP Dashboard Purpose: The IPOPP Dashboard has two modes: Configuration Editor Mode and Process Monitor Mode. In Configuration Editor Mode, the Dashboard allows you to enable or disable SPAs that are already installed. In its Process Monitor Mode, the Dashboard allows you to perform IPOPP operations (Start/Stop/Check/Reset), as well as Monitor the ON/OFF/Processing status of SPAs. Invoking the Dashboard and Its Modes: The IPOPP Dashboard can be invoked with the following command:

$HOME/drl/tools/dashboard.sh

This command opens up the IPOPP Dashboard in Process Monitor Mode. The Process Monitor functionality and IPOPP utilities are available only in this mode. Enabling and disabling of SPAs can be done by switching to the Configuration Editor Mode. Switching between modes is available via the "Mode" Menu Item on the Dashboard Window. Detailed Description: Table 3 contains descriptions of the GUI components. Please refer to Figures 1a and 1b depicting the IPOPP Dashboard in its two modes. Use the diagram indices to map the corresponding GUI component to its location in the figures.

Table 3: IPOPP Dashboard Description

Diagram Index

GUI Component Description

1 Mode> IPOPP Configuration Editor

Switch to the Configuration Editor Mode.

2 Mode> IPOPP Process Monitor

Switch to the Process Monitor Mode.

3 SPA Station

In the Configuration Editor Mode this unit acts as a toggle button to allow users to enable or disable the SPA station. In this mode the state can be one of the following:

The SPA station is enabled.

The SPA station is disabled.

The SPA station is not installed. You may download SPAs from the DRL Web Portal and install them as described in Section 7, "Installing/Updating/ Configuring SPAs."

Please note that a station must be enabled to be started by 'Action>Start Services' or 'Action>Start SPA Services'. Starting of IPOPP Services (including SPA Services) has to be done as a separate step after returning to the Process Monitor Mode. In the Process Monitor Mode this unit reports the ON/OFF/Processing status of the SPA station. In this mode the state can be one of the following:


The SPA station is running.

The SPA station is not running either because the station is disabled or because IPOPP SPA services are not running.

The SPA station failed to start.

The SPA station is not installed. You may download SPAs from the DRL Web Portal and install them as described in Section 7, "Installing/Updating/ Configuring SPAs."

The SPA station is ingesting data. Processing is about to start.

.h:mm:ss

The SPA station is processing. The timer below shows the time elapsed since processing started.

.h:mm:ss

The SPA station successfully processed. The timer below shows the time it took to complete processing.

.h:mm:ss

The SPA station generated a warning during processing but was otherwise successful. Some warnings are harmless. For example some algorithms are Daytime algorithms and generate warnings when enough daytime scans are not available. Additional information on the warning can be found from the SLS Console (refer to Section 6.2). The timer below shows the time it took to complete processing.

.h:mm:ss

The SPA station encountered an error during processing (refer to Section 8). The timer below shows how long it ran before it failed.

NOTE: Users must have SLS Console in LIVE Mode in order to view real-time activity on the Process Monitor (its default mode; refer to Section 6.2 for details).

4 Product Information Button

Clicking on this button will open up a dialog box identifying the data products generated by the SPA station.

5 SPA Relationship Highlight

Only available in Configuration Editor Mode. Moving your mouse over an SPA Station highlights the station's immediate upstream and downstream SPA stations. Note that the immediate upstream stations may depend on their own upstream stations, which in turn can have their own prerequisites. In order to correctly enable a station you must follow the dependencies all the way up the processing chain and enable all predecessor stations.

6 Mission/Sensor Tabs

SPA stations are distributed in three tabs: (i) the "EOS" tab shows stations that process data from Terra/Aqua MODIS; (ii) the "SNPP-VIIRS" tab shows stations that process SNPP VIIRS data; and (iii) the "SNPP-ATMS/CrIS/OMPS" tab shows SPA stations that process SNPP ATMS, CrIS and OMPS data.

7 Action>Save Configuration

Only available in Configuration Editor Mode. Allows you to save the current Configuration.


8 Ingest Display Unit

Only available in Process Monitor Mode. This unit reports the ON/OFF status of the IPOPP Ingest Services. On the EOS tab it reports the status of the PDS Ingest Services, while on the SNPP tabs it reports the status of the RDR Ingest Services.

9 Action>Start Services

1

Only available in Process Monitor Mode. Clicking on this will start all IPOPP Services.

10 Action>Stop Services

2

Only available in Process Monitor Mode. Clicking on this will stop all IPOPP Services.

11 Action>Start SPA Services

1

Only available in Process Monitor Mode. Clicking on this will start IPOPP's SPA Services. Caution: Starting SPA Services will not start other IPOPP services that are required for IPOPP processing. Use this option when you are sure that other IPOPP Services are already running. If in doubt, use the "Action>Start Services" option.

12 Action>Stop SPA Services

2

Only available in Process Monitor Mode. Clicking on this will stop IPOPP's SPA Services.

13 Action>Check IPOPP Services

1

Only available in Process Monitor Mode. This will check the status of all IPOPP services and determine System Health. It will list the services that are not running. If System Errors are detected, it will provide an option for users to restore IPOPP health. In that case, click 'OK' to restore System Health. This should resolve the detected system errors and stop services. Users will be provided an option to start services once the process is complete.

14 Action>Reset IPOPP

Only available in Process Monitor Mode. Clicking on this will reset IPOPP. An IPOPP reset can be used to recover from unexpected IPOPP errors. A reset will stop the IPOPP Services, clear product registration from the IPOPP database and remove ancillary and data product files. NOTE: IPOPP reset deletes all data products. Copy data products that you want to retain to another location before you reset IPOPP.

1NOTE: Log files created by any failing SPA in its corresponding run directory are cleared by "Actions>Start

Services", "Actions>Start SPA Services", or when restoring system health from "Actions> Check IPOPP Services". Refer to Section 8, "Diagnosing and Recovering from IPOPP Operational Errors." 2NOTE: Choose a time of data processing inactivity to stop IPOPP services or stop SPA services. The SLS

Console can be used to determine current data processing activity (refer to Section 6.2,. "Status/Event Logging System Console").

6.2. Status/Event Logging System Console Purpose: The Status/Event Logging System (SLS) Console logs status and event messages sent by the IPOPP components.

The SLS allows system monitoring and provides indications of system problems or anomalies.

The SLS can serve as a diagnostic tool. Users who wish to find more information beyond the top-level error/warning log can do so by using the Back Trace utility.

Log messages can be viewed in real time (LIVE Mode) or replayed from a selected time span. Available message filters allow searching through logs.

Invoking the SLS Console: To run the SLS Console, enter:


$HOME/drl/tools/dashboard.sh

This opens the IPOPP Dashboard and the SLS Console. The SLS Console may be hidden behind the Dashboard. SLS Console is in LIVE Mode when it is first launched (refer to Table 4).

Detailed Description: The SLS Console is depicted in Figures 2a and 2b. Table 4 describes its GUI components and how to use them to perform various operations. Use the diagram indices to map the corresponding GUI component to its location in the figures.


Figure 2a: Status/Event Logging Console (Log Messages and Back Trace Controls)


Figure 2b: Status/Event Logging Console (Menu, Mode Panel, Filter and Playback Controls)


Table 4: SLS Console

Diagram Index

GUI Component

Description

Log Messages

1 Log Message Window

Informational log messages are displayed over a white background. Warning log messages are displayed over a yellow background, and error log messages are displayed over a red background.

Back Trace Controls

2 Back Trace Log Messages correspond to various events and include diagnostic back trace text. Clicking on this button opens up the Event Details Window containing the back trace text.

3 Examine Previous Event/Examine

Next Event

[Available on the Event Details Window] Clicking on these buttons displays the back trace text for the previous/next log or event respectively.

4 Save Event Details to File

[Available on the Event Details Window] Clicking on this button allows the user to save the back trace text displayed on the Event Details Window to a file.

5 Close Event Details

[Available on the Event Details Window] Clicking on this button closes the Event Details Window.

Menu Controls

6 File>Connect…

This is an advanced feature for connecting to a remote IPOPP.

7 File>Disconnect Clicking on this menu item disconnects SLS console from IPOPP.

8 File>IPOPP Dashboard

Clicking on this menu item opens up the IPOPP Dashboard (if it was closed).

9 File>Exit Clicking on this menu item closes SLS Console.

10 Help>About NSLS…

Clicking on this menu item displays the SLS Version.

Mode Panel Controls

11 Start LIVE Mode Clicking on this button starts LIVE Mode. In LIVE Mode messages are displayed in the log message window as they are reported. The 'LIVE MODE' status is displayed in the lower left corner of the Console.

12 Stop LIVE Mode3

Clicking on this button stops LIVE Mode and starts Playback Mode. Playback Mode allows messages to be displayed for a selected time span. Playback controls become available (refer to items 16 and 17).

13 Display Filter Panel

Clicking on this button forces message Filters to be displayed if they are hidden.

14 Erase Events Clicking on this button clears displayed messages.

15 Toggle Auto Scrolling of

Events

Clicking on this button toggles the start and stop of automatic message scrolling.

Playback Controls (available in Playback Mode)

16 Play Events3

[Available in Playback Mode (after you click on Stop LIVE Mode)] Clicking on this button plays back the events that satisfy the criteria specified in the Filter Panels.

17 Stop Playing Events

[Available in Playback Mode] Clicking on this button stops playing back the events.


Filter Controls

18 Display Text Filter Clicking on this button opens up a Text Filter Panel. Users can open up more than one Text Filter Panel. This is unlike other filters, which can have only one panel.

19 Display Date/Time Filter

Clicking on this button opens up the Date/Time Filter Panel.

20 Display Source Filter

Clicking on this button opens up the Source Filter Panel.

21 Display Level Filter

Clicking on this button opens up the Level Filter Panel.

22 Count Panel The "Count" Panel allows selection of the number of log messages to be displayed on the Log Message Window at a time. '50' and '500' are available as fixed options, but the empty third option may be used to set it to any value. A large value captures more messages but requires more processing time, and makes the scroll bars more sensitive.

23 Text Filter Panel This panel is opened using the Display Text Filter button and allows selection of messages containing a text string.

24 Date/Time Filter Panel

This panel is opened using the Display Date/Time Filter button and allows message selection by Date and Time.

25 Source Filter Panel

This panel is opened using the Display Source Filter button and allows message selection by IPOPP component origin. The DSM (Database Services), NCS (SPA services), IS (ancillary retrieval services) and other components may be specified as the source.

26 Level Filter Panel

This panel is opened using the Display Level Filter button and allows message selection by Information, Warning, and Error message levels.

27 Delete Filter [Available on each Filter Panel] Clicking on this button will remove the corresponding Filter Panel.

28 Close Filter Panel Clicking on this button closes the Filter Panel(s). Note that closing the Filter Panel(s) does not remove any filtering already selected. Use the Delete Filter button on the corresponding panels to remove the filters.

3NOTE: SLS Console must be in LIVE Mode, without any filtering enabled to view real-time activity on the

Process Monitor. Events/logs played back In Playback Mode will also appear on the Process Monitor.

6.3. Pass Manager Purpose: The Pass Manager facilitates:

1. The reprocessing of SNPP RDR files and Terra/Aqua PDS (packet file and CSR) file pairs corresponding to a satellite overpass, resulting in the regeneration of all downstream science data products associated with the overpass. The Reprocess utility may be useful after the installation of a newer version of an SPA into IPOPP. When a newer version of an SPA is installed, it will start producing science products for all subsequent overpasses. You may use the Reprocess utility to reprocess older overpasses. Reprocessing may also be useful after a system error. The Pass Manager Table may be used to select and reprocess overpasses that were being processed at the time of the error. In this case reprocessing is recommended to ensure that all downstream data products are generated.

2. The deletion of all science data products (including the SNPP RDR file or Terra/Aqua PDS [packet file and CSR] file pairs) associated with a satellite overpass. The Delete utility may be useful to manage limited file system space. By default IPOPP will automatically remove overpasses that are more than 14 days old. In situations where there is a need to remove more recent passes due to disk space limitations or similar concerns, the Delete utility provides a safe way to reclaim space by deleting overpasses that may no longer be of interest. Manually removing data products is not recommended.

Invoking the Pass Manager: The Pass Manager Table can be launched using the command line:


$HOME/drl/dsm/gui-scripts/passmanager.sh

Detailed Description: The Pass Manager is depicted in Figure 3. Table 5 describes its GUI components and how to use them to perform reprocessing and deletion. Use the diagram indices to map the corresponding GUI component to its location in the figure.

Figure 3. Pass Manager Table

Table 5: Pass Manager

Diagram Index

GUI Components

Description

1 Pass Manager Display Window

Each row in the Pass Manager Table represents a satellite/instrument overpass that has been registered with the IPOPP database, and includes the following information associated with the overpass:

a) Pass ID;

b) Spacecraft;

c) Acquisition of Signal (AOS) (start time of satellite overpass);

d) Loss of Signal (LOS) (end time of satellite overpass).

2 View The View button can be used to view details such as (start, stop and creation time) about products associated with an overpass.


3 Refresh By clicking 'Refresh', the table will be updated to the most current state. The table does not update automatically.

4 Reprocess Click on a single row or multiple rows (by holding down the CTRL key) to select a single overpass or multiple overpasses. Click on 'Reprocess', and a dialog box will ask for confirmation. Clicking 'Yes' on the confirmation dialog box will result in deletion of all science products associated with the selected overpass(es) (except the SNPP RDR file or the Terra/Aqua PDS file pair) and complete reprocessing of the overpass(es) (i.e., re-runs of relevant SPAs and regeneration of all science data products associated with the overpass).

5 Delete Click on a single row or multiple rows (by holding down the CTRL key) to select a single overpass or multiple overpasses. Click on 'Delete', and a dialog box will ask for confirmation. Clicking 'Yes' on the confirmation dialog box will result in the unregistering and deletion of all science data products associated with the overpass(es), including the SNPP RDR files and Terra/Aqua PDS (packet file and CSR) file pairs. The delete operation may take some time. Wait for the message box to confirm that the delete was successful, then click on the 'Refresh' button to view the changes. NOTE: Please use caution when using the Delete utility, as it is irreversible.

7. Installing/Updating/Configuring SPAs

Science Processing Algorithms (SPAs) are remote sensing science algorithms packaged with the DRL's algorithm wrapping technique. This algorithm wrapper provides a common command and execution interface to encapsulate multi-discipline, multi-mission science processing algorithms. Once packaged as an SPA, a science algorithm can be plugged into IPOPP for automatic processing. This version of IPOPP already contains the most current SNPP SPAs and Terra/Aqua MODIS SPAs available at the time of release. For the list of SPAs included in this package refer to Appendix B, "Science Processing Algorithms and Products." New or updated SPAs will be made available via the DRL Web Portal. Go to http://directreadout.sci.gsfc.nasa.gov/?id=software for the most current list of SPAs and products supported by IPOPP. Follow the instructions below for installing new or updated SPAs into an existing IPOPP.

NOTE: To preconfigure an existing SPA rather than install/update it (e.g., SPAs like BURNSCAR_SPA and H2G_SPA that are already included with IPOPP), perform only steps 2, 5 and 6 as described below.

1. Download algorithm_versionno_SPA_wrapperversionno.tar.gz into $HOME/drl Go to http://directreadout.sci.gsfc.nasa.gov/?id=software and select the SPA to be installed by clicking on the hyperlink. Follow the prompts and download the compressed archive file to the $HOME/drl/ subdirectory. The SPA compressed archive file name will be of the form: algorithm_versionno_SPA_wrapperversionno.tar.gz (e.g, C-SDR_1.6_SPA_1.5.tar.gz).

2. Close the Dashboard and the SLS Console, if they are open.

3. cd $HOME/drl [Change Directory]

4. $HOME/drl/tools/install_spa.sh algorithm_version_SPA_wrapperversion.tar.gz [Install SPA]

This will delete the old SPA directory (if it exists), create a new <spaname> subdirectory in the $HOME/drl/SPA directory, and install the new SPA into IPOPP.

5. Pre-configure if needed [Skip this step if SPA(s) do not require pre-configuration]




Some SPAs (e.g., BURNSCAR_SPA and H2G_SPA) need pre-configuration to activate certain functionalities. Refer to the corresponding SPA User's Guide for details on pre-configuration. Also note that you may have to pre-configure an upstream or downstream SPA to realize the full utility of the pre-configuration. For example BURNSCAR_SPA pre-configuration should be done in combination with H2G_SPA pre-configuration to allow them to be in sync with respect to the spatial tiles identified for processing (refer to the BURNSCAR_SPA and H2G_SPA User's Guides). Once pre-configuration is complete, execute the following from the $HOME/drl subdirectory for each SPA that was pre-configured:

$HOME/drl/SPA/<spaname>/NISGSinstall.sh

6. Configure IPOPP and start IPOPP services [Refer to Section 5, "IPOPP Operation"] The SPA is now ready to be enabled and run within IPOPP. The SPA installation process will have stopped all SPA services and the IPOPP Ingest Services. Use the IPOPP Dashboard to configure IPOPP and restart all services.

8. Diagnosing and Recovering from IPOPP Operational Errors

The most common problems IPOPP users may encounter are described below, along with possible causes and resolution strategies. If you are experiencing a problem that is not described here, you may search the MyDRL Forum at: http://directreadout.sci.gsfc.nasa.gov/?id=forum for similar issues/resolutions reported by other users. If you are still unable to resolve your problem, please post a question to the MyDRL Forum or use "Contact DRL" to submit your question directly: http://directreadout.sci.gsfc.nasa.gov/?id=dspContent&cid=66 PROBLEM 1: An SPA station is failing (red error logs on SLS Console, errors in Dashboard:Process Monitor): Are you running out of hard disk space? Remove unwanted files (tar files or other non-IPOPP files) from the drl/ folder and use the Pass Manager Table 'Delete' utility to reclaim space under /raid. Is your system running low on resources (e.g., low memory)? Try disabling some SPA stations that are not needed (refer to Step 1 in Section 5, "IPOPP Operation"). Ensure that your system meets the recommended system requirements (refer to Appendix A, "System Requirements"). Are there clues in the FAIL directories? When a station fails, standard output (stdfile*), standard error (errfile*), and other log and intermediate files from the failed run are retained under drl/ncs/<stationname>/FAIL* directories. If the error is obvious and resolvable, correct the problem and use the Pass Manager Table 'Reprocess' utility to reprocess the overpass (refer to Section 6.3, "Pass Manager"). Was the SNPP RDR file or Terra/Aqua PDS (packet file and CSR) file pair ingested correctly (i.e., is the RDR file or PDS file pair incomplete)? Delete the pass using the Pass Manager Table 'Delete' utility (refer to Section 5.6, "Pass Manager") and follow the process documented in Step 3 in Section 5, "IPOPP Operation," to re-transfer the affected files into /raid/dsm/nisfes_data. Does the error message indicate that the station is unable to obtain one or more ancillary files?

Are you trying to process a real time swath but the ancillary retrieval service is not running? Check the status of IPOPP services using the "Check IPOPP Services" menu option on the IPOPP Dashboard and if 'Ancillary Retriever and Registration Services' is reported as 'OFF', start IPOPP services using the 'Actions>Start Services' menu item on the IPOPP Dashboard (refer to Section 6.1, "IPOPP Dashboard").

http://directreadout.sci.gsfc.nasa.gov/?id=forum



Are you trying to process a non-real time swath, but have not ingested the required ancillaries? Ingest required ancillaries (follow the process documented in Step 3 in Section 5, "IPOPP Operation") and then use the Pass Manager Table 'Reprocess' utility to reprocess the overpass (refer to Section 6.3, "Pass Manager"). None of the above seems to be the problem. It is possible that the IPOPP database has been corrupted and the existing ancillaries are not registered with the IPOPP database. Try resetting IPOPP using the "Actions>Reset IPOPP" menu option on the IPOPP Dashboard (refer to Section 6.1, "IPOPP Dashboard").

Is your system able to run the core algorithm executables? An easy way to discover whether your system is able to run the core algorithm executables is to install the SPA testdata tar file and try to run the testscript (refer to the corresponding SPA User's Guide). If the testscript fails, your system may not meet the recommended system requirements (refer to Appendix A, "System Requirements" and the SPA User's Guide). PROBLEM 2: A product is not being produced. Is the corresponding SPA installed? Install the SPA (refer to Section 7, "Installing/Updating/Configuring SPAs"). Is the SPA station enabled? Use IPOPP Dashboard in Configuration Editor Mode to verify that the station is enabled (refer to Section 6.1, "IPOPP Dashboard"). Are prerequisite stations enabled? Use Dashboard in Configuration Editor Mode to verify that prerequisite stations are enabled (refer to Section 6.1, "IPOPP Dashboard"). Is the SPA station ON? Use Dashboard in Process Monitor Mode to verify status (refer to Section 6.1, "IPOPP Dashboard"). If the station is OFF, try starting the station using the "Start Services" menu option on the IPOPP Dashboard (refer to Section 6.1, "IPOPP Dashboard").

PROBLEM 3: Actions performed from the IPOPP Dashboard take an unexpectedly long time or seem to wait indefinitely. Are you running out of resources? This rare event may occur if you are running all SPA stations corresponding to both EOS and SNPP missions simultaneously, or your computer does not meet the recommended system requirements. A terminate_dashboard.sh utility is available at the IPOPP download site on the DRL Web Portal to address this situation. Download the TerminateDashboard_utility.tar.gz tar file and place it under $HOME/drl. Then do the following:

Abort (Ctrl-C) any IPOPP scripts that you may have invoked on the command line. cd $HOME/drl [Change directory] tar –xzf TerminateDashboard_utility.tar.gz [Unpack the utility] $HOME/drl/tools/terminate_dashboard.sh [Execute the script]

This will terminate all IPOPP services along with the Dashboard and any actions issued by it. At this point we recommend that you reduce the number of enabled SPAs, or increase system memory.

Version 2.3 Page A-1 April 4, 2014

Appendix A System Requirements

Hardware IPOPP has been tested on computers with the following configuration: Processors: Dual Quad Core Opteron Shanghai processors

RAM: 32GB DDR3 buffered ECC

Operating System Disk: SATA 2 RAID-1 (500 GB)

Data Disk: SATA 2 RAID-5 (1.5 Tb), mounted at /raid

Motherboard: TYAN Thunder n3600M (S2932) dual socket F It is recommended that separate physical disks and raid controllers be used for the operating system and the data storage at /raid. Performance on less capable systems may be improved by disabling unneeded SPAs. SNPP processing is CPU-intensive; processors with higher clock speeds will produce output proportionally faster. Operating Systems IPOPP has been tested under these operating systems:

a) Fedora 18 X86_64;

b) CentOS Linux 6.4 X86_64;

c) Open SUSE Linux 12.1 X86_64; and

d) Kubuntu 13.04 X86_64.

System Time Some IPOPP ancillary file retrievals and product generations are time-dependent; the system must use 24-hour UTC time and be synchronized through a Network Time Protocol (NTP) Server. Required Software The following software must be installed:

a) Java Runtime Environment (JRE) 1.6.0_25 or later;

b) MySQL Client and Server 5.1 or later;

c) libXp 1.0 or later;

d) bash 4.1.2 or later;

e) tcsh 6.17 or later;

f) bc 1.06 or later;

g) ed 1.1 or later;

h) Native 32-bit support or ia32-libs (ia32-libs should not be installed on systems providing native support such as Redhat Enterprise, Redhat Fedora, OpenSUSE, and CentOS Linux). Additionally, Fedora Linux requires the 32-bit version of ld-linux.so.2, which can be obtained by installing glibc for i686.

Version 2.3 Page A-2 April 4, 2014

Network Firewall Configuration and Port Access These TCP ports should be open:

a) Ports 20 and 21 for FTP;

b) Port 3306 for MySQL;

c) Ports 3500 through 3550 for SLS;

d) Ports 4900 through 4950 for IPOPP Data Storage Manager (DSM); and

e) Ports 49152 through 65535 for passive mode FTP.

Known Issues and Resolutions

a) GUI tools on OpenSUSE: On systems using OpenSUSE, GUI tools such as the Dashboard, SLS Console or Pass Manager (refer to Section 6, "IPOPP Tools") may not work when you ssh into the system hosting IPOPP. This can be fixed by setting:

X11UseLocalhost no in the configuration file /etc/ssh/sshd_config on the system hosting IPOPP. Setting X11UseLocalhost to "no" is a workaround for OpenSUSE platforms where the DISPLAY environment variable may not be correctly set. On other platforms the DISPLAY environment variable is expected to be set regardless of the value of X11UseLocalhost.

b) System Process Settings: In rare cases, processes may fail to start if the restriction on the number of processes allowed per user is set very low. This problem has been reported on Fedora and CentOS platforms. To avoid this issue on Fedora and CentOS platforms, this number can be increased by editing the /etc/security/limits.d/90-nproc.conf file and adding the following tab-separated text: # Default limit for number of user's processes to prevent # accidental fork bombs. * soft nproc 65535

NOTE: This procedure must be performed as the root user.

Version 2.3 Page B-1 April 4, 2014

Appendix B Science Processing Algorithms and Products

The table below lists Science Processing Algorithms (SPAs) included in this package along with the data products they produce. More detailed descriptions of SPAs are contained in the respective SPA User's Guides available with the SPA packages.

SNPP SPAs Version Included

Output Products

C-SDR 1.6 VIIRS Imagery resolution SDRs VIIRS Moderate resolution SDRs VIIRS Day/Night Band (DNB) SDR VIIRS On Board Calibrator IP Intermediate Product (IP) VIIRS Calibrated Dual Gain IP VIIRS Geolocation products ATMS SDR ATMS Temperature Data Record (TDR) ATMS Geolocation products CrIS SDR and Geolocation products

CVIIRS 1.0 VIIRS Imagery and Moderate Resolution Corrected Reflectance Level2 Products

ActiveFires 1.5.06.07 VIIRS Active Fires Application Related Product (ARP) (with Fire Mask)

GTMImagery 1.5.07.02 VIIRS I-Band Ground Track Mercator (GTM) Imagery EDR VIIRS M-Band GTM Imagery EDR VIIRS Near Constant Contrast (NCC) Albedo GTM Imagery EDR

CloudMask 1.5.06.07 VIIRS Cloud Mask IP

LST 1.5.06.07 VIIRS Land Surface Temperature EDR

Aerosol 1.5.06.07 VIIRS Aerosol Optical Thickness (AOT) IP VIIRS Aerosol Model Index (AMI) IP VIIRS Aerosol Environmental Data Record (EDR) VIIRS Suspended Matter (SM) EDR VIIRS Aerosol Geolocation

SurfReflect 1.5.06.07 VIIRS Land Surface Reflectance IP

VegIndex 1.5.06.07 VIIRS Vegetation Index EDR

COP 1.5.06.07 VIIRS Cloud Optical Properties IP VIIRS Ice & Night Water Cloud Top Temperature IP

SnowCov 1.5.06.07 VIIRS Snow Binary Map EDR VIIRS Snow Fraction EDR

VIIRS-AF 1.2.9 VIIRS Active Fire Level 2 (NASA algorithm) VIIRS Fire Location Text File

OMPSnadir 0.0.5 OMPS Total Column Level 1A EV (TCL1AEV) OMPS Total Column SDR EV(TCSDREV) OMPS Total Column Total Ozone (TCTO3) OMPS Total Column Total SO2 NRT (TCTSO2NRT) OMPS Nadir Profile Level 1A EV (NPL1AEV) OMPS Nadir Profile SDR EV (NPSDREV) OMPS Nadir Profile SDR Deluxe (NPSDRDELUXE) OMPS Nadir Profile Ozone (NPO3) PNG image for Ozone PNG image for Reflectivity at 331nm PNG image for Ultraviolet Aerosol

MODIS SPAs

Version Included

Output Products

Aqua GBAD 2.7 Aqua Ephemeris and Attitude files

MODISL1DB 1.8 MODIS Level 1A (MOD01/MYD01) and Geolocation (MOD03/MYD03) MODIS Level 1B 1 km (MOD021KM/MYD021KM), half km (MOD02HKM/MYD02QKM), and quarter km (MOD02QKM/MYD02QKM)

Version 2.3 Page B-2 April 4, 2014

products

IMAPP 2.2 MODIS Cloudmask Level 2(MOD35) MODIS Cloudtop Properties, Cloud Phase and Cloud Optical properties Level 2 (MOD06) MODIS Atmospheric Profiles Level 2(MOD07) MODIS Aerosol Level 2(MOD04) NOTE: The IPOPP package does not contain the optional IMAPP*_MOD06OD_COEFF.tar.gz coefficient files tar file. This file is available on the DRL Web Portal and may be optionally installed to enable the MODIS Cloud Optical Properties software module (MOD06OD). Additional instructions are available in the IMAPP_SPA User's Guide.

L2GEN 6.4.2 MODIS Ocean Color Level 2 (daytime product, includes Chlorophyll-a [CHLOR_A] concentration) MODIS Sea Surface Temperature (SST) Level 2

MOD14 6.2.1 MODIS Active Fire (MOD14) Level 2 MODIS Fire Location Text File

CREFL 1.7.1 MODIS Level 2 Corrected Reflectance Level 2

NDVIEVI 2.2 MODIS Vegetation Indices (MOD13) Level 2

MODLST 4.14 MODIS Land Surface Temperature Level 2

BURNSCAR 1.1 MODIS Half km Corrected Reflectance Level 2 MODIS Corrected Reflectance Daily Composite Level 3 MODIS Active Fires Daily Composite Level 3 MODIS Prescar Level 3 (Standalone Mode only) MODIS Burnscar Level 3 (Standalone Mode only)

Other SPAs

Version Included

Output Products

H2G 2.3 Geolocated GeoTIFF images, for various parameter datasets in SNPP SPA products and MODIS Level 2 SPA products. H2G also creates standard true color images for supported VIIRS and MODIS science products.

Version 2.3 Page C-1 April 4, 2014

Appendix C Location of Ancillaries and Data Products

The table below shows the location of the ancillaries and processed instrument data products.

Directory Location Ancillaries/Products

/raid/pub/ancillary Subdirectories under this folder hold registered ancillaries required for real-time processing.

/raid/pub/ArchivedAncillary Subdirectories under this folder hold registered ancillaries required for non-real-time processing.

/raid/pub/gsfcdata/terra/modis/level{0,1,2} Terra MODIS Level 0, Level 1 and Level 2 products

/raid/pub/gsfcdata/aqua/modis/level{0,1,2} Aqua MODIS Level 0, Level 1 and Level 2 products

/raid/pub/gsfcdata/combined/modis/level3 Aqua/Terra Composite Daily products (produced only when MODIS BurnScar SPA is installed)

/raid/pub/gsfcdata/aqua/gbad/ Aqua GBAD Level 0, ephemeris and attitude files

/raid/pub/gsfcdata/npp/viirs/level{0,1,2} SNPP VIIRS RDRs, SDRs and EDRs

/raid/pub/gsfcdata/npp/atms/level{0,1} SNPP ATMS RDRs, SDRs

/raid/pub/gsfcdata/npp/cris/level{0,1} SNPP CrIS RDRs, SDRs

/raid/pub/gsfcdata/npp/omps/level{0,1,2} SNPP OMPS RDRs, Level I and Level 2 products

ipopp 2.3 users guide

Documents

data products

ipopp software

science processing algorithms

ipopp installation

ipopp tools

ipopp dashboard

utility of earth science

multilevel processing