interface control document : idp & olqcemits.sso.esa.int/emits-doc/alcatel/sentinel2/ipf/... ·...

Interface ControlDocument : IDP

& OLQC

REF : S2-PDGS-TAS-DI-BPDP-ICD-IPF

ISSUE : 03

DATE : 06/04/12

PAGE : 1 / 63

This document may not be disclosed to a third party or reproduced without the prior written consent of Thales Alenia Space FranceAll rights reserved, 2012, Thales Alenia Space

Interface Control Document :IDP & OLQC

Written by Company Responsibility Date Signature

C.Samson ThalesServices

IPF Technical Managerfor TS

06/04/12

P.Branet ThalesServices

TS Design Authority 06/04/12

Verified by

R.Joyeux ThalesServices

Project Manager for TS 06/04/12

S. Bonnot TAS Technical Manager 06/04/12

Approved by

A. Le Ber TAS Project Manager 06/04/12

Approval evidence is kept within the documentation management system.

Documentation Information

Submitted to ESA for Filename:

Information Review ApprovalS2_PDGS_TAS_DI_BPDP_ICD_IPF_03_Interface_C

ontrol_Document_IDP_OLQC.doc

lebera

Tampon


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 2 / 63


CHANGE RECORDS

ISSUE DATE § CHANGE RECORDS AUTHOR

01 Draft 08/12/2012 First draft version for ESA review C. SAMSON

01 13/01/2012 First official version for PDR C. SAMSON

02 15/03/2012 Implementation of PDR-RID-286/EC-17: firstversion of OLQC-GIPP XSD scheme defined(including new test configuration management).

Implementation of PDR-RID-287/EC-18: detailson OLQC interface, first version of OLQC-GIPPand OLQC Report XSD files.

Implementation of PDR-RID-308/EC-23: I/F perIDP-SC or OLQC-SC are detailed. I/F IPU toOLQC has been set to "mixed", a link to ENVIformat description is given.

Implementation of PDR-RID-316/EC-24: PVIformat is set to GML jpeg2000.

C. SAMSON

A. FIROBEN

R. AURIEMMA

N. BENECH

03 06/04/2012 Modifications after pre-TEB meeting

Jpeg 2000 for TCI

List of Reference Documents modified

References clean-up

C. SAMSON

A. FIROBEN

R. AURIEMMA

N. BENECH

TyposFile Naming Convention:

Implementation of RIDS PDR-RID-40/FM-12Implementation of RIDS PDR-RID-37/FM-09


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 3 / 63


TABLE OF CONTENTS

1. INTRODUCTION .................................................................................................................................. 6

1.1 PURPOSE OF THE DOCUMENT ....................................................................................................... 61.2 DOCUMENT STRUCTURE .............................................................................................................. 61.3 DOCUMENTATIONS AND GLOSSARY ............................................................................................ 6

2. INSTRUMENT PROCESSING FACILITY CONTEXT ................................................................... 8

3. EXTERNAL INTERFACES ............................................................................................................... 10

3.1 DPC – IDP & OLQC INTERFACE .............................................................................................. 103.1.1 DPC / IDP-SC or OLQC-SC common interfaces ...................................................................... 123.1.2 DPC / IDP-SC interfaces for L0c processing............................................................................ 183.1.3 DPC / IDP-SC interfaces for L1A/B radiometric processing and L1A formatting.................... 223.1.4 DPC / IDP-SC interfaces for L1B geometric processing and formatting.................................. 303.1.5 DPC / IDP-SC interfaces for L1C processing ........................................................................... 373.1.6 DPC / OLQC-SC interface ........................................................................................................ 41

4. INTERNAL INTERFACES................................................................................................................. 43

4.1 INTERNAL INTERFACES LIST ...................................................................................................... 434.2 IDP-ORCHESTRATOR / IDP-SC & OLQC-SC INTERFACES ....................................................... 434.3 IDP-ORCHESTRATOR / IPF TASK TABLE INTERFACE................................................................ 454.4 DECOMPRESSION CFI INTERFACES............................................................................................ 454.5 OLQC-SC/AMALFI INTERFACES ............................................................................................... 464.6 IDP-SC/JP2K INTERFACE .......................................................................................................... 47

5. INTERFACES DEFINITION ............................................................................................................. 48

5.1 EXTERNAL INTERFACES DEFINITION.......................................................................................... 485.1.1 Interface Name: DEM to IDP.................................................................................................... 485.1.2 Interface Name: DPC - IPF control .......................................................................................... 495.1.3 Interface Name: DPC - IPF monitoring .................................................................................... 505.1.4 Interface Name: ECMWF AuxData to IDP ............................................................................... 515.1.5 Interface Name: GIPP to IDP ................................................................................................... 515.1.6 Interface Name: GIPP to OLQC ............................................................................................... 525.1.7 Interface Name: GRI to IDP...................................................................................................... 535.1.8 Interface Name: IERS AuxData to IDP ..................................................................................... 535.1.9 Interface Name: IPU to IDP...................................................................................................... 545.1.10 Interface Name: IPU to OLQC.............................................................................................. 545.1.11 Interface Name: OLQC Report to DPC ................................................................................ 555.1.12 Interface Name: POD Product to IDP (Contingency) .......................................................... 555.1.13 Interface Name: Processed IPU from IDP............................................................................ 565.1.14 Interface Name: Quality Annotated IPU from OLQC........................................................... 56

5.2 FILE NAMING CONVENTIONS ..................................................................................................... 575.3 STANDARDS ............................................................................................................................... 61


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 4 / 63


Table of Figures

FIGURE 1: IPF IN THE FRAMEWORK OF PRODUCTION SERVICES....................................................................... 9FIGURE 2: IDP-SC AND OLQC-SC INTERFACE WITH DPC............................................................................. 11FIGURE 3: IPF INTERFACES FOR THE STAND-ALONE CONFIGURATION ........................................................... 44

Table of Tables

ERREUR ! AUCUNE ENTREE DE TABLE D'ILLUSTRATION N'A ETE TROUVEE.


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 5 / 63



& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 6 / 63


1. INTRODUCTION

1.1 Purpose of the document

This document provides the interfaces description of the Instrument Processing Facility (IPF).

1.2 Document Structure

This document is structured as follows:

Chapter 1: Provides the current introduction, describing the scope and the structure of thedocument, and the list of applicable and reference documents;

Chapter 2: Describes the environment of Instrument Processing Facility.Chapter 3: Provides the Instrument Processing Facility external interfaces.Chapter 4: Provides the Instrument Processing Facility internal interfaces.Chapter 5: Details the definition of interfaces.

1.3 Documentations and Glossary

Acronyms used in the documentation are defined in S2 PDGS Glossary & definitions Document[S2-PDGS-TAS-DI-CMS-GLODEF]

The list of Applicable and Reference documents is defined in the following tables:

Applicable Documents

Reflabel Reference Version Date Title

CORE-BPDP-SOW-IPF

S2-PDGS-TAS-DI-BP-SOW 5.0 05/04/2012

Statement of Work for S2 PDGS IPFProject

PSDS2-PDGS-TAS-DI-PSD

3.0 06/04/2012 Product Specification Document

Reference Documents


CCTS-IPFS2-PDGS-TAS-DI-BPDP-CCTS-IPF

3.0 06/04/2012Instrument Processing Facility RequirementSpecifications


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 7 / 63



SMICDS2-PDGS-TAS-DI-ICD-MICD

2.0 15/03/2012 Master Interface Control Document (MICD)

NRTPOD-ICD

GMES-GSEG-EOPG-IC-11-0022

0.1 draft 25/03/2011 Sentinel-2 Core PDGS NRT POD ICD

GRI-TNGMES-GSEG-EOPG-TN-10-0071

0.1 draft 03/12/2010 Sentinel-2 Global Reference Images Generation

POD-FFSGMES-GSEG-EOPG-FS-10-0075

1.1 28/02/2011GMES Sentinel Precise Orbit Determination (POD)Service: OFL POD Service and NRT POD Facility FileFormat Specifications

GPP-DEM GS2-IF-GSGP-70-CNES 1.1 18/03/2010System DEM Description (GLOBE, filtered SRTM)content and format

EOFFS-PDGS

GMES-GSEG-EOPG-TN-2010-0099

1.0 28/02/2011Earth Observation GS File Format Standard – Tailoringfor the Sentinel Missions PDGS

HMA-GML OGC 06-080r2 0.9.0 18/06/2007GML 3.1.1 Application schema for Earth Observationproducts

SAFE-TNGMESPH-ACS-TEC-TNO24-E

1.1 25/12/2010GMES Products Harmonisation SAFE Implementation :Sentinel2 Support

SAFE-SPECGAEL-P264-DOC-0001-01-01

1.0 25/07/2011 Sentinel Standard Archive Format for Europe

GEN-PDGSIPF

GMES-GSEG-EOPG-TN-09-0016

1.0 24/09/2009 GMES Generic PDGS-IPF Interface Specifications

AMALFI-SUM

GAEL-P264-SUM-001 3.3 10/2011 Amalfi User Manual

DRB-SUM GAEL-P243-DOC-001 1.1 30/09/2008 Data Request Broker DRB API Handbook

TDS-TN GPP-TN-DLR-S2-0140 3.0 16/09/11 Test Data Specifications

MDS-SUM-MRCPBG

MAG-MRCPBG-MIMU-01-LAN

6.0 10/11/11MRCPBG v9.0 installation instructions and usermanual


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 8 / 63


2. INSTRUMENT PROCESSING FACILITY CONTEXT

The Level-0 and Level-1 Instrument Processing Facility (IPF) corresponds to a set of dataprocessing software elements fulfilling the requirements of the Instrument Data Processing (IDP)function of the PDGS associated to MSI level-0 and Level-1 data processing. The IPF alsocontains a set of essential quality checks performed on each product generated by the IDP thanksto the implementation of the On Line Quality Control (OLQC) function.

In the operational configuration (hereafter referred to as DPC-driven mode), the PDGS DataProcessing Control (DPC) component is in charge of performing the IDP function byorchestrating a set of Instrument Data Processing Software Components (IDP-SC) in workflowin order to generate the Level-0c up to Level-1C products. The DPC is also responsible of productquality checks by calling the OLQC Software Component OLQC-SC in the production workflows.The main goal of the IPF developments is to implement the IDP and OLQC functions. Thesedevelopments will be integrated by the PDGS Prime Contractor within the DPC environment.

In order to test the complete workflow, the IPF also includes the development of a specificsoftware, the "IDP-Orchestrator", that allows to test the end-to-end processing in isolation fromthe DPC. The IDP-Orchestrator software component will implement an end-to-end processing in asimplified (e.g. not fully optimized in term of processing timeline compared to the DPC environmentconfiguration) workflow triggering all IDP-SC and OLQC-SC in a logical sequence.

The detailed description of IPF software components can be found in [CCTS-IPF], the followingsections are focusing on detailing the set of IPF external and internal interfaces.


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 9 / 63


<IPF>

<IPF BestPractice>PDGS-IAS

<CFI>S2_WICOM

DECOMPRESSION

<OLQC>OLQC-SC

<IDP>Hosted Processing

<IDP>Image Processing

<DPC>DPC orchestrator

launch

use

managemanage

<IDP>IDP-SC

launch

<IPF>IDP Orchestrator

Instrument ProcessingFunctionality

execute

Production Services

<IDP>Telemetry

Processing

<CFI>AMALFI

use

manage

use

<IPF>IPF Hosted Processing

<IAS>HP-IAS

<IAS>HP-IAS

<IAS>HP-SC

execute

<COTS>JP2K

COMPRESSOR

<DPC>Data Exchange

Manager

manage

<CFI>DFEP

use

<CFI>S2_WICOM

DECOMPRESSION

use

Figure 1: IPF in the framework of Production Services


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 10 / 63


3. EXTERNAL INTERFACES

As shown on Figure 1, IPF external interfaces are limited to IDP-SC and OLQC-SC interfaces withthe DPC for use in the DPC-driven mode.These interfaces are the one that shall be used by the DPC to directly run the IDP-SC or OLQC-SC in the nominal configuration (DPC-driven automatic configuration).

Note that before their integration into the Production Service (DPC interfacing), the IDP-SC andOLQC-SC shall be run by the IDP-Orchestrator on a similar way, but in this configuration suchinterfaces are considered as internal.All other interfaces are therefore considered as IPF internal interfaces.

3.1 DPC – IDP & OLQC Interface

The table hereafter lists these interfaces:

I/F Name From ToJob Order DPC IDP-SC / OLQC-SCProcessing Configuration DPC IDP-SC / OLQC-SCStart/Stop DPC IDP-SC / OLQC-SCAuxiliary data DPC IDP-SC / OLQC-SCAncillary data DPC IDP-SCInput Processing Unit DPC IDP-SC / OLQC-SCOutput Processing Unit IDP-SC / OLQC-SC DPCLogging IDP-SC / OLQC-SC DPCProduct Report IDP-SC / OLQC-SC DPCExit Code IDP-SC / OLQC-SC DPCData exchange repository all all

Table 1: List of interfaces between DPC and IDP-SC / OLQC-SC

These interfaces with DPC may be represented in the following way:


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 11 / 63


IDP-SCIDP-SC

DPC

Job Order

Start/Stop

Processing Configuration

Logging

Product Report

Exit Code

L0 dataL0c (reprocessing)

L0c/L1A/L1B/L1C Products(inspected by OLQC)

IDP-SCor

OLQC-SC

Input PU

Ouptut PU

Auxiliary data

Ancillary data

Aux/Anc data

IPF Workflow PDIs

Figure 2: IDP-SC and OLQC-SC interface with DPC


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 12 / 63


3.1.1 DPC / IDP-SC or OLQC-SC common interfaces

3.1.1.1 Job Order Interface

This interface is used by the DPC to launch an IDP-SC or the OLQC-SC, referred to as SC.

The Job Order file is generated by the DPC.

A job order XML file is associated to each SC. The naming convention is JobOrder_<order_id>.xml where order_id is a unique integer ID associated to this production order.

The Job Order contains:

The SC name and version

The list of input files and pathname which is a sub-set of (depending on the SC):

Auxiliary data

Ancillary data

Input Processing Unit files (Image data, Metadata, Masks)

Processing Configuration (TBD)

The list of output files and pathname which is a sub-set of (depending on the SC):

Output Processing Unit files (Image data, Metadata, Masks)

Product Report file

The settings for a trace mode through (Breakpoints):

Breakpoint enabling

List of breakpoint files

A complete XSD schema of the Job Order file is provided as a separate file.

Note: in DPC driven mode, Task Tables are never used. Only the Job Order interface is used.

A naming convention will be provided by the PDGS Prime Contractor to identify the various inputand output data in Job order interface.


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 13 / 63


3.1.1.2 Processing Configuration Interface

This interface is an XML file used by the DPC to configure an IDP-SC or the OQLC-SC. The set ofconfigurations is made of any processing options and/or algorithmic performance for each SC thatis not already in the Auxiliary GIPP file.Typical examples of these switches are:

performance options (e.g. maximum RAM size to be used by a given IDP-SC)

3.1.1.3 Start/Stop Interface

This interface is used by the DPC to start/stop the execution of an IDP-SC or OQLC-SC (hereafterreferred to as Job).The Start interface consists in executing a Job through a command line with the related Job Orderas an argument. The Job Order is the only argument of the command line used to launch eachIPD-SC or OLQC-SC.

The Stop interface allows to stop the execution of a Job with a killing signal (kill -2). In this case,the IDP-SC shall:

send a log message notifying the end on kill, terminate the execution of the report in a clean way and exits with a particular return code.

3.1.1.4 Auxiliary Data Input

All auxiliary input data that shall be ingested by an IDP-SC or OLQC-SC are listed in its Job Order.

The set of Auxiliary data is:

The Ground Image Processing Parameters (GIPP) corresponding to a set of XML filesassociated to a processing component (i.e. IDP/OLQC-SC) to define a set of parametersand their values. A GIPP file is associated to a validity period since the parameters can betuned during the PDGS lifetime.

The Digital Elevation Model (DEM) is made of binary files, “tiles”, covering the Earth Globe(up to certain latitudes). For Shuttle Radar Topography Mission (SRTM) DEM, the files arein dted format.

The Global Reference Image (GRI) [GRI-TN] is a set of Sentinel-2 mono-spectral Level-1Bproducts (granules, metadata) covering all land regions within the acquisition plan. TheGRI is used during geometric processing to improve the geo-location accuracy and to reachthe absolute geo-location accuracy and multi-temporal co registration requirements.

Ozone, surface pressure and water vapour data are required for Level-1C processing.These data are published by the European Centre for Medium-Range Weather Forecasts(ECMWF). The data are GRIdded Binary format GRIB described athttp://www.wmo.int/pages/prog/www/WDM/Guides/Guide-binary-2.html


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 14 / 63


The International Earth Rotation & Reference Systems service (IERS) is publishing dailyASCII TXT data bulletins about Earth orientation and Terrestrial reference system (EarthPole position, UT1-UTC,). These data are required for the computation of the geometricalmodel.

3.1.1.5 Ancillary Data Input

The OLQC-SC does not need any Ancillary Data as an input.

All Ancillary input data that shall be ingested by an IDP-SC are listed in the relevant Job Order.

The Ancillary data are made of a set of files for

System Ancillary Data (observation mode...)

Instrument Ancillary Data (compression ratio, acquisition datation...)

Satellite Ancillary Data (orbit, position)

Input Precise Orbit Determination POD data (optional: if the attitude information is notmatching the expected accuracy)

The Ancillary data is contained in a XML file: directly issued from the Front-End-Processing orcontained in the metadata XML file.

3.1.1.6 Input Processing Units

All the input PU files that shall be ingested by an IDP-SC or OLQC-SC are listed in the relevantJob Order.The input PU is made of the following files (or sub-set of files, depending on IDP-SC):

A set of image files which can be:

o On-board compressed binary files (up to L0c)

o ENVI format images (a "flat binary file" and a small associated ASCII (text) headerfile, see http://geol.hu/data/online_help/ENVI_File_Formatsh.html)

o JP2K compressed images (QuickLook images in case of reprocessing)

A set of XML Metadata files

A set of GML Mask files

A naming convention will be provided by the PDGS Prime Contractor to identify the various inputand output data files.


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 15 / 63


3.1.1.7 Output Processing Units

All the output PU files that shall be issued by an IDP-SC or OLQC-SC are listed in the relevant JobOrder.The output PU is made of the following files (or sub-set of files, depending on IDP-SC):

A set of image files which can be:

o ENVI format images (a "flat binary file" and a small associated ASCII (text) headerfile, see http://geol.hu/data/online_help/ENVI_File_Formatsh.html)

o JP2K compressed images

A set of XML Metadata files

A set of GML Mask files

A naming convention will be provided by the PDGS Prime Contractor to identify the various inputand output data files.

3.1.1.8 Logging Interface

The Logging interface allows to monitor the status of the running IDP-SC/OLQC-SC through a setof messages sent to the DPC.The DPC is able to analyze the messages and possibly route them to the S2-PDGS M&C,depending on severity.Logging messages are syslog style ASCII strings, variable length, NL terminated. The messagesshall allow monitoring the events (DEBUG, INFO), warnings (WARNING) and health status(STATUS) and are sent on stdout, except errors (ALARM) and progress messages (PROGRESS)that are sent to stderr.

The message format consists of sub-strings fields separated by a blank character. The fulldescription is given in the next table:

Field Format DescriptionDate and Time 26 characters, in the format

yyyy-MM-ddThh:mm:ss.nnnnnnwhere the dash,colon, period and Tcharacters are sub-field separators,and the sub-field meaning is as follows

yyyy year : 4 digitsMM month : 2 digits, leftpadded with zerosdd day : 2 digits, left paddedwith zeroshh hours : 2 digits, leftpadded with zerosmm minutes : 2 digits, leftpadded with zerosss seconds : 2 digits, leftpadded with zerosnnnnnn microseconds : 6digits, right padded withzeros

Node Name variable length alphanumeric string(no whitespace characters allowed)

The node name to be put inthe message shall be the


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 16 / 63


Field Format Descriptionstring retrieved through thegethostname system call

IDP-SC/OLQC-SC Name variable length alphanumeric string(no whitespace characters allowed)

This field identifies the IDP-SC or OLQC-SC

IDP-SC/OLQC-SC Version 5 characters, in the format nn.nn The embedded period isincluded in the count, andmust be present)

PID 12 characters (including squarebrackets) in the format[nnnnnnnnnn]

This field specifies thesystem level process id ofthe processor, obtainedthrough the getpid systemcall

Header Separator 1 colon character : No space shall be insertedbetween the PID and theseparator;

Message Type [<type>](square brackets are mandatory)

This field identifies the typeof message being issuedwhich can be one of thefollowing:ALARM: critical failureWARNING: some failure butnot criticalINFO: simple informationDEBUG: simple informationfor developers, availableonly for debug when thislogging mode is activatedPROGRESS: particularinformation messagesindicating processingcompletion progress, with %indication (0% at IDP-SC orOLQC-SC start, 100% atIDP-SC or OLQC-SCsuccessful completion)STATUS: working statusnotification (periodicallyevery n minutes with nconfigurable)Other TBD

Message Text (for ALARM,WARNING, INFO,DEBUG, PROGRESS)

variable length string, no restriction,NL-terminated (unix-style)

This field contains thedescriptive free text of themessage itself

Status Text (for STATUS) <status_string> This field contains thecomponent status, which canbe one of the following:ON: working properlyOFF: shut down (if relevant)FAILURE: some criticalfailure (reasons shall benotified to operator throughproper ALARM messages)


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 17 / 63


Field Format DescriptionDEGRADED: some failure,and is working in degradedwayOther TBD

Table 2: Logging format

A typical example of log entry is shown below (ASCII String)

2011-11-04T14:02:07.458000 IPFS2 RESAMPLE 01.04 [0000013875]:[INFO] Processor starting. Config fileis /home/ResampleS2/config.xml

3.1.1.9 Product Report Interface

This interface is used to generate an XML report file at the end of each IDP-SC or OLQC-SCexecution. The path and filename of the Product Report are defined in the Job Order file.

The full content of the Product Report file has to be tailored for each IDP-SC or OLQC-SC but itshall at least contain:

The name and version of the IDP/OLQC-SC

The messages of the Logging Interface (optional, configurable according to log severity)

Self-explanatory information that can help identifying the status of the main processingsteps, to be defined later for every IDP-SC according to image quality mission performanceassessment needs

The Job Order file

An XSD schema of a Report file is provided as a separate file.

3.1.1.10 Exit Code Interface

All the IDP-SC or OLQC-SC shall return an exit code through a return statement at the end of themain function for instance, or through a call to the exit system call.The exit code values are listed in the table below:

Code Definition Description0 OK The IDP-SC/ or OLQC-SC

has been succefullycompleted

1-10 WARNING Reserved exit codes to bedefined later in coordinationwith PDGS Prime Contractor

11-127 INCOMPLETE Recoverable errors havebeen encountered


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 18 / 63


Code Definition Description128 FAILURE Any other unrecoverable

errors

Table 3: Exit codes

If a child process is launched by the main process started by DPC, the exit code of the childprocess shall be propagated to the main process.

3.1.1.11 Data exchange repository

All the files exchanged between DPC and every IDP-SC or OLQC-SC (job order files, configurationfiles, input processing files, output processing files, reports, etc), are available in a direct accessfile tree shared between DPC and IPF components.This tree is created by DPC before IDP-SC and OLQC-SC launch, for files and subdirectories read,write and delete.IPF components temporary files will be stored in a dedicated subdirectory created by DPC beforelaunch.After IDP-SC and OLQC-SC completion, the deletion or further use of files and working directoriesis managed by DPC.

3.1.2 DPC / IDP-SC interfaces for L0c processing

The following paragraphs present the interfaces of each IDP-SC in charge of L0c processing.

3.1.2.1 INIT_LOC_L0 interfaces

Interface DescriptionJob Order Interface Cf. 3.1.1.1Processing Configuration Interface NoneStart/Stop Interface Cf. 3.1.1.3Auxiliary Data Input The XML GIPP files :

o GS2_DATATION_PARAMETERS.xmlo GS2_INIT_LOC_INV_PARAMETERS.xmlo GS2_VIEWING_DIRECTIONS.xmlo GS2_SPACECRAFT_MODEL_PARAMETERS.xml

GLOBE DEM files The IERS data bulletin at the date of ISP acquisition.

Ancillary Data Input Either SAD data or read from Metadata at datastrip level (ephemeris,attitudes, datation model, …)

Input Processing Units Metadata : datastrip xml file for each granule of the processed datastrip, a granule xml file

Output Processing Units Metadata : datastrip xml files (input updating) granule xml files (input updating)

Logging Interface Cf. 3.1.1.8


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 19 / 63


Interface DescriptionProduct Report Interface Cf. 3.1.1.9Exit Code Interface Cf. 3.1.1.10Data exchange repository Cf. 3.1.1.11

3.1.2.2 QL_GEO interfaces


o GS2_LR_EXTRACTION_PARAMETERS.xmlo GS2_INIT_LOC_INV_PARAMETERS.xmlo GS2_VIEWING_DIRECTIONS.xmlo GS2_SPACECRAFT_MODEL_PARAMETERS.xml

GLOBE DEM. The IERS data bulletin at the date of ISP acquisition.

Ancillary Data Input Read from Metadata at datastrip level (ephemeris, attitudes, datationmodel, …)

Input Processing Units Metadata : datastrip xml file for each granule of the processed datastrip, a granule xml file

Data : ATF (decompressed LR images; one by band and detector for

each datablock)Output Processing Units Metadata :

granule xml files (input update)

Data : quicklook images (one by band and detector for each

datablock)Logging Interface Cf. 3.1.1.8Product Report Interface Cf. 3.1.1.9Exit Code Interface Cf. 3.1.1.10Data exchange repository Cf. 3.1.1.11

3.1.2.3 QL_CLOUD_MASK interfaces

Interface DescriptionJob Order Interface Cf. 3.1.1.1Processing Configuration Interface NoneStart/Stop Interface Cf. 3.1.1.3Auxiliary Data Input In the NOMINAL scenario, the set of required Auxiliary data is:

The XML GIPP files :o GS2_INIT_LOC_INV_PARAMETERS.xmlo GS2_ABSOLUTE_CALIBRATION.xmlo GS2_CLOUD_INV_PARAMETERS.xml


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 20 / 63


Interface Descriptiono GS2_INTER_DETECTOR_PARAMETERS.XML

In the REPROCESSING scenario, the set of required Auxiliary data is: The XML GIPP files :

o GS2_INIT_LOC_INV_PARAMETERS.xmlo GS2_ABSOLUTE_CALIBRATION.xmlo GS2_CLOUD_INV_PARAMETERS.xml


Input Processing Units Metadata : datastrip xml file

In the NOMINAL scenario, the set of required Data input PU is: quicklook images (one by band and detector for each

datablock)In the REPROCESSING scenario, the set of required Data input PU is:

quicklook images (one by band for each datablock)Output Processing Units Metadata :

datastrip xml file (input update)In the NOMINAL scenario, the set of required Data input PU is:

concatenated QuickLook images (one file by band and ATF) cloud mask vector (one file by ATF)

In the REPROCESSING scenario, the set of required Data input PU is: cloud mask vector

Logging Interface Cf. 3.1.1.8Product Report Interface Cf. 3.1.1.9Exit Code Interface Cf. 3.1.1.10Data exchange repository Cf. 3.1.1.11

3.1.2.4 FORMAT_IMG(QL JP2000) interfaces

Interface DescriptionJob Order Interface Cf. 3.1.1.1Processing Configuration Interface NoneStart/Stop Interface Cf. 3.1.1.3Auxiliary Data Input The set of required Auxiliary data is:

GIPP:o GS2_JP2K_PARAM.xml: specific processing

parameters (e.g. compression parameters)Ancillary Data Input NoneInput Processing Units Metadata :

datastrip xml fileData :

concatenated QuickLook images (one file by band and inputdatablock)

Output Processing Units Data : concatenated QuickLook images (one JPEG2000 file by band

and by input datablock)Logging Interface Cf. 3.1.1.8Product Report Interface Cf. 3.1.1.9Exit Code Interface Cf. 3.1.1.10


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 21 / 63


Interface DescriptionData exchange repository Cf. 3.1.1.11

3.1.2.5 FORMAT_ISP interfaces

Interface DescriptionJob Order Interface Cf. 3.1.1.1Processing Configuration Interface NoneStart/Stop Interface Cf. 3.1.1.3Auxiliary Data Input None (TBC)Ancillary Data Input NoneInput Processing Units Metadata :

datastrip xml file granule (for each granule) xml files

Data : Image Source Packets (gathered by band and detector for

each datablock)Output Processing Units Metadata :

granule xml file (input update)Data :

SAFE packaged ISP (generated for each granule)Logging Interface Cf. 3.1.1.8Product Report Interface Cf. 3.1.1.9Exit Code Interface Cf. 3.1.1.10Data exchange repository Cf. 3.1.1.11

3.1.2.6 FORMAT_METADATA(GR-L0c) interfaces

Interface DescriptionJob Order Interface Cf. 3.1.1.1Processing Configuration Interface NoneStart/Stop Interface Cf. 3.1.1.3Auxiliary Data Input None (TBC)Ancillary Data Input Taken from Input DS metadata (TBC)Input Processing Units Metadata :

datastrip xml file granule (for each granule) xml files

Data : cloud mask vector (for the whole datastrip, by ATF) SAFE packaged ISP for each granule

Output Processing Units Metadata : L0c Metadata at granule level in SAFE format (input update)

Data : SAFE formatted granule



& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 22 / 63


3.1.2.7 FORMAT_METADATA(DS-L0c) interfaces


GS2_ABSOLUTE_CALIBRATION.xml GS2_ CLOUD_INV _PARAMETERS.xml GS2_DATATION_PARAMETERS.xml GS2_INIT_LOC_INV_PARAMETERS.xml GS2_LR_EXTRACTION_PARAMETERS.xml GS2_SPACECRAFT_MODEL_PARAMETERS.xml GS2_VIEWING_DIRECTIONS.xml

IERS bulletin DEM: reference to coarse DEM (Globe) used in L0 processing

Ancillary Data Input ASP : Ancillary Source PacketInput Processing Units Metadata :


quicklook imagesOutput Processing Units Metadata :

datastrip xml file (input update)Data :

SAFE formatted datastripLogging Interface Cf. 3.1.1.8Product Report Interface Cf. 3.1.1.9Exit Code Interface Cf. 3.1.1.10Data exchange repository Cf. 3.1.1.11

3.1.3 DPC / IDP-SC interfaces for L1A/B radiometric processing and L1A formatting

The following paragraphs present the interfaces of each IDP-SC in charge of L1A/B radiometricprocessing and L1A formatting.

3.1.3.1 UNFORMAT_SAFE(GR) interfaces

Interface DescriptionJob Order Interface Cf. 3.1.1.1Processing Configuration Interface NoneStart/Stop Interface Cf. 3.1.1.3Auxiliary Data Input None (TBC)Ancillary Data Input NoneInput Processing Units Data :

SAFE formatted granuleOutput Processing Units Data :

SAFE unformatted granule (IPF internal format)


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 23 / 63


Interface Description Set of ISP (for all available bands) for each processed granule

Metadata : granule xml file (IPF internal format)


3.1.3.2 UNFORMAT_SAFE(DS) interfaces

Interface DescriptionJob Order Interface Cf. 3.1.1.1Processing Configuration Interface NoneStart/Stop Interface Cf. 3.1.1.3Auxiliary Data Input None (TBC)Ancillary Data Input Ancillary Source Packet (ASP)Input Processing Units Data :

SAFE formatted datastripOutput Processing Units Metadata :

datastrip xml filesData :

Quicklook JPEG2000 imagesLogging Interface Cf. 3.1.1.8Product Report Interface Cf. 3.1.1.9Exit Code Interface Cf. 3.1.1.10Data exchange repository Cf. 3.1.1.11

3.1.3.3 UPDATE_LOC interfaces


IERS bulletin Coarse DEM (GLOBE) GIPP :

o GS2_DATATION_PARAMETERS.xmlo GS2_INIT_LOC_PROD_PARAMETERS.xmlo GS2_SPACECRAFT_MODEL.xmlo GS2_VIEWING_DIRECTIONS.xml

Ancillary Data Input POD data (degraded case only) Exogenous Datation model (degraded case only) Datation information at ISP level (degraded case only) SAD data (reprocessing only) Otherwise read from Metadata at datastrip level (ephemeris,

attitudes, datation model, …)Input Processing Units Metadata :


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 24 / 63


datastrip xml file for each granule of the processed datastrip, a granule xml file

Output Processing Units Metadata : datastrip xml file (update) for each granule of the processed datastrip, a granule xml file

(update)Logging Interface Cf. 3.1.1.8Product Report Interface Cf. 3.1.1.9Exit Code Interface Cf. 3.1.1.10Data exchange repository Cf. 3.1.1.11

3.1.3.4 QL_DECOMP interfaces


GIPP:o GS2_DECOMP_PARAM.xml : DECOMP specific

processing parametersAncillary Data Input NoneInput Processing Units Data :

Full swath JPEG2000 compressed QuickLook images (one fileby band and by datablock)

Output Processing Units Metadata : datastrip xml file

Data : Full swath concatenated QuickLook images (one file by band

and datablock)Logging Interface Cf. 3.1.1.8Product Report Interface Cf. 3.1.1.9Exit Code Interface Cf. 3.1.1.10Data exchange repository Cf. 3.1.1.11

3.1.3.5 QL_CLOUD_MASK interfaces


The XML GIPP files :o GS2_INIT_LOC_PROD_PARAMETERS.xmlo GS2_ABSOLUTE_CALIBRATION.xmlo GS2_CLOUD_INV_PARAMETERS.xml


Input Processing Units Metadata :


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 25 / 63


Interface Description datastrip xml files

Data : quicklook images (one by band and detector for each

datablock)Output Processing Units Metadata :

datastrip xml files (input update)Data :

concatenated QuickLook images (one file by band and ATF) cloud mask vector (one file by ATF)


3.1.3.6 INIT_LOC_L1 interfaces


GIPP :o GS2_INIT_LOC_PROD_PARAMETERS.xmlo GS2_SPACECRAFT_MODEL_PARAMETERS.xmlo GS2_VIEWING_DIRECTIONS.xmlo GS2_ABSOLUTE_CALIBRATION.xmlo GS2-CLOUD_INV_PARAMETERS.xml

Coarse DEM (GLOBE)Ancillary Data Input Read from Metadata at datastrip levelInput Processing Units Metadata :

datastrip xml files granule xml files for each granule of the processed detectors

Data : Coarse Cloud Mask in QuickLook geometry

Output Processing Units Metadata : granule xml files (update)

Data : ATF, for each band of each processed detector :

o Detector footprinto Cloud mask in Focal plan geometryo Technical masks (ancillary data)



& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 26 / 63


3.1.3.7 DECOMP interfaces


GIPP:o GS2_DECOMP_PARAM.xml : DECOMP specific

processing parametersAncillary Data Input NoneInput Processing Units Metadata :

datastrip xml files granule files for each granule of the processed ATF

Data : Image Source Packets (gathered by band for each granule)

Output Processing Units Metadata : datastrip xml files (update) granule files for each granule of the processed ATF (update)

Data : Decompressed images (one file by band) Mission Data Quality Masks (degraded and lost)


3.1.3.8 RADIO_AB interfaces


GIPP :o GS2_RADIO2_PARAMETERS.xmlo GS2_RADIO2_REARRANGEMENT.xmlo GS2_RADIO2_EQUALIZATION_ONBOARD.xmlo GS2_RADIO2_EQUALIZATION_ONGROUND.xmlo GS2_BLIND_PIXELS.xmlo GS2_RADIO2_SWIR_ARRANGEMENT.xmlo GS2_RADIO2_CROSSTALK_CORRECTION.xmlo GS2_RADIO2_DEFECTIVE_PIXELS.xmlo GS2_RADIO2_DECONVOLUTION_FILTER.xmlo GS2_RADIO2_INSTRUMENT_NOISE_MODEL.xmlo GS2_RADIO2_WAVELET_FILTERS.xmlo GS2_RADIO2_L2_NORM_COEFFICIENTS.xmlo GS2_RADIO2_DENOISING_THRESHOLDs.xmlo GS2_RADIO2_BINNING.xml

Ancillary Data Input Read from Metadata at datastrip level


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 27 / 63


Interface DescriptionInput Processing Units Metadata :


On-board decompressed images (all available bands)Output Processing Units Metadata :

datastrip xml files (update)Data :

Images without deconvolution and denoising scenario : ATF images files (one file by band) Or a set of granule images files (one file by band and by

granule)Logging Interface Cf. 3.1.1.8Product Report Interface Cf. 3.1.1.9Exit Code Interface Cf. 3.1.1.10Data exchange repository Cf. 3.1.1.11

3.1.3.9 RADIO_FINALIZE interfaces


GIPP :o GS2_INIT_LOC_PROD_PARAMETERS.xmlo GS2_VIEWING_DIRECTIONS.xmlo GS2_RADIO2_INSTRUMENT_NOISE_MODEL.xmlo GS2_ABSOLUTE_CALIBRATION.xmlo GS2_MASK2_PARAMETERS.xml

Ancillary Data Input A list of ancillary data by ATFInput Processing Units Metadata :


for each detector : for each ATF for each band :o A list of Detector footprint by ATFo A list of Cloud mask in Focal plan geometry by ATFo A list of Technical masks (ancillary data) by ATFo A list of Technical masks (mission data) by ATFo A list of Radiometric masks by ATF

Output Processing Units Metadata : datastrip xml files (update)

Data : for each band of each processed detector)

o Detector footprinto Cloud mask in Focal plan geometryo Technical maskso Radiometric masks

Logging Interface Cf. 3.1.1.8Product Report Interface Cf. 3.1.1.9


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 28 / 63


Exit Code Interface Cf. 3.1.1.10Data exchange repository Cf. 3.1.1.11

3.1.3.10 FORMAT_METADATA (GR-L1A) interfaces

Interface DescriptionJob Order Interface Cf. 3.1.1.1Processing Configuration Interface NoneStart/Stop Interface Cf. 3.1.1.3Auxiliary Data Input NoneAncillary Data Input Read from Metadata at datastrip levelInput Processing Units Metadata :

datastrip xml files granule xml file for each granule

Data : ATF :

o List of L1A radiometric masks (defective pixels,saturated pixels, No_Data pixels)

o List of Technical masks (mission data quality masks,ancillary data quality masks)

o List of Cloud masksOutput Processing Units Metadata :

granule xml file for each granule (update)Data :

Pixel level Quality indicators (masks at granule level)o L1A radiometric masks,o Coarse cloud masks,o Technical masks


3.1.3.11 FORMAT_METADATA (DS-L1A) interfaces

Interface DescriptionJob Order Interface Cf. 3.1.1.1Processing Configuration Interface NoneStart/Stop Interface Cf. 3.1.1.3Auxiliary Data Input GIPP :

o GS2_ABSOLUTE_CALIBRATION.xmlo GS2_BLIND_PIXELS.xmlo GS2_CLOUD_INV _PARAMETERS.xmlo GS2_DATATION_PARAMETERS.xmlo GS2_INIT_LOC_PROD_PARAMETERS.xmlo GS2_MASK2_PARAMETERS.xmlo GS2_RADIO2_BINNING.xmlo GS2_RADIO2_CROSSTALK_CORRECTION.xmlo GS2_RADIO2_DECONVOLUTION_FILTER.xml


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 29 / 63


Interface Descriptiono GS2_RADIO2_DEFECTIVE_PIXELS.xmlo GS2_RADIO2_DENOISING_THRESHOLDs.xmlo GS2_RADIO2_EQUALIZATION_ONBOARD.xmlo GS2_RADIO2_EQUALIZATION_ONGROUND.xmlo GS2_RADIO2_INSTRUMENT_NOISE_MODEL.xmlo GS2_RADIO2_L2_NORM_COEFFICIENTS.xmlo GS2_RADIO2_PARAMETERS.xmlo GS2_RADIO2_REARRANGEMENT.xmlo GS2_RADIO2_SWIR_ARRANGEMENT.xmlo GS2_RADIO2_WAVELET_FILTERS.xmlo GS2_SPACECRAFT_MODEL_PARAMETERS.xmlo GS2_VIEWING_DIRECTIONS.xml

IERS bulletin DEM: reference to coarse DEM (Globe) used in L1A

processingAncillary Data Input NoneInput Processing Units Metadata :

datastrip xml filesOutput Processing Units Metadata :

datastrip xml files (update)Data :

Formatted L1A datastripLogging Interface Cf. 3.1.1.8Product Report Interface Cf. 3.1.1.9Exit Code Interface Cf. 3.1.1.10Data exchange repository Cf. 3.1.1.11

3.1.3.12 FORMAT_IMG (L1A_JP2000) interfaces

Interface DescriptionJob Order Interface Cf. 3.1.1.1Processing Configuration Interface NoneStart/Stop Interface Cf. 3.1.1.3Auxiliary Data Input GIPP:

o GS2_JP2K_PARAM.xml : specific processingparameters (e.g. compression parameters)

Ancillary Data Input NoneInput Processing Units Metadata :

datastrip xml file

Data : L1A images for each band of processed ATF L1A images for each band of each granule of processed ATF

Output Processing Units Data : for each granule, L1A JPEG2000 compressed image (one file

by available band)Logging Interface Cf. 3.1.1.8Product Report Interface Cf. 3.1.1.9Exit Code Interface Cf. 3.1.1.10


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 30 / 63


Data exchange repository Cf. 3.1.1.11

3.1.4 DPC / IDP-SC interfaces for L1B geometric processing and formatting

The following paragraphs present the interfaces of each IDP-SC in charge of L1A/B radiometricprocessing and L1B geometric processing and formatting

3.1.4.1 GET_GRI interfaces

Interface DescriptionJob Order Interface Cf. 3.1.1.1Processing Configuration Interface NoneStart/Stop Interface Cf. 3.1.1.3Auxiliary Data Input NoneAncillary Data Input NoneInput Processing Units Metadata :

GRI Metadata at datastrip level GRI Metadata at datastrip level Metadata at datastrip level

Data : Detector footprint for band used by IMAGE-GRI registration

Output Processing Units Metadata : GRI metadata at datastrip level, for each GRI that intercept the

datastrip (update)Data :

list of GRI that intercept the datastripLogging Interface Cf. 3.1.1.8Product Report Interface Cf. 3.1.1.9Exit Code Interface Cf. 3.1.1.10Data exchange repository Cf. 3.1.1.11

3.1.4.2 UNFORMAT_GRI interfaces

Interface DescriptionJob Order Interface Cf. 3.1.1.1Processing Configuration Interface NoneStart/Stop Interface Cf. 3.1.1.3Auxiliary Data Input NoneAncillary Data Input NoneInput Processing Units In “decompression only” scenario :

Metadata :=o GRI Metadata at datastrip level

Data :o JPEG2000 compressed GRI images (one by

granule of the processed ATF)

In “mask aggregate only” scenario :


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 31 / 63


Interface DescriptionMetadata :

o GRI Metadata at datastrip level Data :

o masks (by granule of the processed detector)Output Processing Units In “decompression only” scenario :

Data : decompressed GRI images (one by granule of the

processed ATF)In “mask aggregate only” scenario :

Data : masks Aggregated by detectors


3.1.4.3 INIT_VS_GEO interfaces


DEM (SRTM class) GIPP :

o GS2_EARTH_MODEL_PARAMETERS.xmlo GS2_GEOMETRICAL_PARAMETERS.xmlo GS2_GEOS2_PARAMETERS.xmlo GS2_INTER_DETECTOR_PARAMETERS.xmlo GS2_PARAMETERS_TO_BE_REDEFINED.xmlo GS2_SPACECRAFT_MODEL_PARAMETERS.xmlo GS2_VIEWING_DIRECTIONS.xml

Ancillary Data Input Read from metadata at datastrip levelInput Processing Units Metadata :

Metadata at datastrip level GRI metadata at datastrip level (for Image-GRI registration

only)Output Processing Units Data :

Virtual Sensor Viewing modelLogging Interface Cf. 3.1.1.8Product Report Interface Cf. 3.1.1.9Exit Code Interface Cf. 3.1.1.10Data exchange repository Cf. 3.1.1.11

3.1.4.4 RESAMPLE_TO_VS interfaces

Interface DescriptionJob Order Interface Cf. 3.1.1.1


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 32 / 63


Interface DescriptionProcessing Configuration Interface NoneStart/Stop Interface Cf. 3.1.1.3Auxiliary Data Input DEM (SRTM class)

GIPP :o GS2_EARTH_MODEL_PARAMETERS.xmlo GS2_GEOMETRICAL_PARAMETERS.xmlo GS2_GEOS2_PARAMETERS.xmlo GS2_INTER_DETECTOR_PARAMETERS.xmlo GS2_PARAMETERS_TO_BE_REDEFINED.xmlo GS2_SPACECRAFT_MODEL_PARAMETERS.xmlo GS2_VIEWING_DIRECTIONS.xml

Ancillary Data Input Read from metadata at datastrip level (TBC)Input Processing Units Metadata :

Metadata at datastrip level GRI metadata at datastrip level (for Image-GRI registration

only)Data :

Radiocorrected Images

Output Processing Units Data : Images resampled in Virtual Sensor geometry (either Image

and GRI or VNIR and SWIR images) Masked area (not to be used area mask)


3.1.4.5 TP_COLLECT interfaces


o GS2_GEOS2_PARAMETERS.xmlAncillary Data Input None (TBC)Input Processing Units In “Densification of input collection” scenario :

Data : Images resampled in Virtual Sensor geometry (either

Image and GRI or VNIR and SWIR images) List of filtered homologous point (could be empty) Masked area (not to be used area mask)

In “Automatic” scenario :Data :

Images resampled in Virtual Sensor geometry (eitherImage and GRI or VNIR and SWIR images)


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 33 / 63


Interface Description Masked area (not to be used area mask)

Output Processing Units Data : List of homologous tie-points


3.1.4.6 TP_FILTER interfaces


o GS2_GEOS2_PARAMETERS.xmlAncillary Data Input None (TBC)Input Processing Units Metadata :

Metadata at datastrip levelData :

Lists of collected Tie-Points/GCP (one list by input ATF, inputATF combination represent all the detector)

Output Processing Units Data : Lists of filtered Tie-Points/GCP (one list by input ATF)


3.1.4.7 SPATIO interfaces

Interface DescriptionJob Order Interface Cf. 3.1.1.1Processing Configuration Interface NoneStart/Stop Interface Cf. 3.1.1.3Auxiliary Data Input Precise DEM (SRTM class)

GIPPo GS2_GEOS2_PARAMETERS.xmlo GS2_PARAMETERS_TO_BE_REFINED.xml

Ancillary Data Input None (TBC)Input Processing Units Metadata :

at datastrip level, xml file

Data : Filtered set of Tie-Points Filtered set of Ground Control Points

Output Processing Units Metadata : at datastrip level, xml file (update)


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 34 / 63


Interface DescriptionLogging Interface Cf. 3.1.1.8Product Report Interface Cf. 3.1.1.9Exit Code Interface Cf. 3.1.1.10Data exchange repository Cf. 3.1.1.11

3.1.4.8 GEO1B_FINALIZE interfaces


GIPP :o GS2_EARTH_MODEL_PARAMETERS.xmlo GS2_GEOMETRICAL_PARAMETERS.xmlo GS2_INTER_DETECTOR_PARAMETERS.xmlo GS2_PARAMETERS_TO_BE_REDEFINED.xmlo GS2_SPACECRAFT_MODEL_PARAMETERS.xmlo GS2_VIEWING_DIRECTIONS.xml

Ancillary Data Input Read from Metadata at datastrip levelInput Processing Units Metadata :

at datastrip level, xml fileOutput Processing Units Data :

densificated Detector footprint, for each band of eachprocessed detector


3.1.4.9 FORMAT_METADATA (GR-L1B) interfaces


GIPP :o GS2_EARTH_MODEL_PARAMETERS.xmlo GS2_GEOMETRICAL_PARAMETERS.xmlo GS2_INTER_DETECTOR_PARAMETERS.xmlo GS2_PARAMETERS_TO_BE_REDEFINED.xmlo GS2_SPACECRAFT_MODEL_PARAMETERS.xmlo GS2_VIEWING_DIRECTIONS.xml

Ancillary Data Input Read from Metadata at datastrip levelInput Processing Units Metadata :

Metadata at datastrip level L1B Metadata at granule level


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 35 / 63


Interface Description L1A checked metadata at granule level for each granule

Data : Technical masks for each granule Cloud masks for each granule

Output Processing Units Metadata : Updated L1B Metadata at granule level

Data : Pixel level Quality indicators (masks at granule level)

o L1B radiometric masks,o Coarse cloud masks,o Technical masks


3.1.4.10 FORMAT_METADATA (DS-L1B) interfaces


o GS2_ABSOLUTE_CALIBRATION.xmlo GS2_BLIND_PIXELS.xmlo GS2_CLOUD_INV _PARAMETERS.xmlo GS2_DATATION_PARAMETERS.xmlo GS2_EARTH_MODEL_PARAMETERS.xmlo GS2_GEOMETRICAL_PARAMETERS.xmlo GS2_GEOS2_PARAMETERS.xmlo GS2_INIT_LOC_PROD_PARAMETERS.xmlo GS2_INTER_DETECTOR_PARAMETERS.xmlo GS2_MASK2_PARAMETERS.xmlo GS2_PARAMETERS_TO_BE_REFINED.xmlo GS2_RADIO_DATA.xmlo GS2_RADIO2_BINNING.xmlo GS2_RADIO2_CROSSTALK_CORRECTION.xmlo GS2_RADIO2_DECONVOLUTION_FILTER.xmlo GS2_RADIO2_DEFECTIVE_PIXELS.xmlo GS2_RADIO2_DENOISING_THRESHOLDs.xmlo GS2_RADIO2_EQUALIZATION_ONBOARD.xmlo GS2_RADIO2_EQUALIZATION_ONGROUND.xmlo GS2_RADIO2_INSTRUMENT_NOISE_MODEL.xmlo GS2_RADIO2_L2_NORM_COEFFICIENTS.xmlo GS2_RADIO2_PARAMETERS.xmlo GS2_RADIO2_REARRANGEMENT.xmlo GS2_RADIO2_SWIR_ARRANGEMENT.xmlo GS2_RADIO2_WAVELET_FILTERS.xml


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 36 / 63


Interface Descriptiono GS2_SPACECRAFT_MODEL_PARAMETERS.xmlo GS2_VIEWING_DIRECTIONS.xml


processing DEM: reference to precise DEM (SRTM class) used in L1B

processing list of GRI that intercept the datastrip (to be referenced)


L1B Metadata at datastrip level L1A checked Metadata at datastrip level

Data Densificated Detector footprint

Output Processing Units Metadata : Updated L1B Metadata at datastrip level

Data : Datastrip footprint Formatted L1B datastrip


3.1.4.11 FORMAT_IMG (L1B_JP2000) interfaces




at datastrip level, xml file

Data : L1B images (all available bands)

Output Processing Units L1B JPEG2000 compressed image (one file by available band)Logging Interface Cf. 3.1.1.8Product Report Interface Cf. 3.1.1.9Exit Code Interface Cf. 3.1.1.10Data exchange repository Cf. 3.1.1.11


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 37 / 63


3.1.5 DPC / IDP-SC interfaces for L1C processing

The following paragraphs present the interfaces of each IDP-SC in charge of L1C processing

3.1.5.1 GET_TILE_LIST interfaces


o GS2_TILINGS2_PARAMETERS.xmlAncillary Data Input NoneInput Processing Units Metadata :


datastrip footprintOutput Processing Units Metadata :

datastrip xml file (update)Logging Interface Cf. 3.1.1.8Product Report Interface Cf. 3.1.1.9Exit Code Interface Cf. 3.1.1.10Data exchange repository Cf. 3.1.1.11

3.1.5.2 TILE_INIT interfaces

Interface DescriptionJob Order Interface Cf. 3.1.1.1Processing Configuration Interface NoneStart/Stop Interface Cf. 3.1.1.3Auxiliary Data Input DEM (DEM quality mask)

ECMWF raw data GIPP :

o GS2_TILINGS2_PARAMETERS.xmlo GS2_RESAMPLES2_PARAMETERS.xmlo GS2_VIEWING_DIRECTIONS.xmlo GS2_EARTH_MODEL_PARAMETERS.xmlo GS2_GEOMETRICAL_PARAMETERS.xmlo GS2_SPACECRAFT_MODEL_PARAMETERS.xmlo GS2_ABSOLUTE_CALIBRATION.xmlo GS2_ECMWF_IMPORT_PARAM.xml : ECMWF

processing parameters (ECMWF resampled griddefinition)


datastrip xml fileOutput Processing Units Metadata :

Creation and updating of metadata at tile level , xml file


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 38 / 63


Interface DescriptionData :

DEM quality mask ECMWF L1C tile


3.1.5.3 GEN_ORTHO_TOA interfaces


GIPP :o GS2_RESAMPLES2_PARAMETERS.xmlo GS2_VIEWING_DIRECTIONS.xmlo GS2_EARTH_MODEL_PARAMETERS.xmlo GS2_GEOMETRICAL_PARAMETERS.xmlo GS2_INTER_DETECTOR_PARAMETERS.xmlo GS2_SPACECRAFT_MODEL_PARAMETERS.xmlo GS2_ABSOLUTE_CALIBRATION.xml

Ancillary Data Input Read from metadata at datastrip levelInput Processing Units Metadata :

datastrip xml file metadata at tile level , xml file

Data : Detector footprint (for processed bands) L1B masks (for processed bands) L1B Radiocorrected images (for processed bands)

Output Processing Units Data : Detector Masks (for processed bands) L1B projected masks (for processed bands) Satellite incidence angles grid (for processed bands) LookUp Table for reflectance conversion (for processed

bands) TOA reflectance images (for processed bands)


3.1.5.4 TILE_FINALIZE interfaces

Interface DescriptionJob Order Interface Cf. 3.1.1.1Processing Configuration Interface None


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 39 / 63


Interface DescriptionStart/Stop Interface Cf. 3.1.1.3Auxiliary Data Input NoneAncillary Data Input NoneInput Processing Units Metadata :

datastrip xml file metadata at tile level , xml files

Data : Satellite incidence angles grid (for all available bands) LookUp Table used for reflectance conversion (for all available

bands) MSI and ancillary data quality masks in L1C geometry (for all

available bands)Output Processing Units Metadata :

metadata at tile level , xml files (update)Logging Interface Cf. 3.1.1.8Product Report Interface Cf. 3.1.1.9Exit Code Interface Cf. 3.1.1.10Data exchange repository Cf. 3.1.1.11

3.1.5.5 MASK_S2 interfaces


o GS2_MASK2_PARAMETERS.xmlAncillary Data Input Read from Metadata at datastrip level (ephemeris, attitudes, datation

model, …) (TBC)Input Processing Units Metadata :

datastrip xml file at tile level, xml files

Data : Images in TOA reflectance for 5 bands, for processed tile

Output Processing Units Metadata : metadata at tile level , xml files (update)

Data : Cloud masks Land/Water masks



& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 40 / 63


3.1.5.6 FORMAT_IMG (L1C_JP2000) interfaces



Ancillary Data Input Read from Metadata at tile level (footprint, …)Input Processing Units Metadata :

at datastrip level , xml file at tile level (for geographic information to be embedded in

GML-JP2000) , xml filesData :

L1C images (all available bands)Output Processing Units Data :

For each tile, L1C GML-JPEG2000 compressed image (onefile by available band)


3.1.5.7 FORMAT_IMG (PVI & TCI) interfaces




at datastrip level , xml file at tile level (for geographic information to be embedded in

GML-JP2000) , xml filesData :

Images in TOA reflectance for 3 bandsOutput Processing Units Metadata :

at tile level, xml files (update)Data :

TCI GML-Jpeg2000 file PVI GML-Jpeg2000 file



& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 41 / 63


3.1.5.8 FORMAT_METADATA (TILE - L1C) interfaces

Interface DescriptionJob Order Interface Cf. 3.1.1.1Processing Configuration Interface NoneStart/Stop Interface Cf. 3.1.1.3Auxiliary Data Input None (TBC)Ancillary Data Input NoneInput Processing Units Metadata :

at datastrip level , xml file at tile level, xml files

Data : all files relevant to processed tile (metadata, masks, …)

Output Processing Units Metadata : at tile level, xml files (update)


3.1.5.9 FORMAT_METADATA (DS - L1C) interfaces

Interface DescriptionJob Order Interface Cf. 3.1.1.1Processing Configuration Interface NoneStart/Stop Interface Cf. 3.1.1.3Auxiliary Data Input GIPP : all GIPP used in L1A, L1B and L1C processing


processing DEM: reference to precise DEM (SRTM class) used in L1B

and L1C processing Reference to the list of GRI used for L1B refining


at datastrip level , xml fileOutput Processing Units Metadata :

at datastrip level , xml file (update)Logging Interface Cf. 3.1.1.8Product Report Interface Cf. 3.1.1.9Exit Code Interface Cf. 3.1.1.10Data exchange repository Cf. 3.1.1.11

3.1.6 DPC / OLQC-SC interface

Interface DescriptionJob Order Interface Cf. 3.1.1.1


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 42 / 63


Interface DescriptionProcessing Configuration Interface NoneStart/Stop Interface Cf. 3.1.1.3Auxiliary Data Input An OLQC-GIPP is defined through the XSD file referred

to as "olqcGipp.xsd" (provided as a separate file)Ancillary Data Input NoneInput Processing Units The input PU can be one of the following:

A Data Granule (or set of) L0, L1A, L1B ,L1C A Data Strip L0, L1A, L1B, L1C A L1B image data An Auxiliary data file (ECMWF data) A TCI or PVI image file

Output Processing Units Depending on the number of checklist to be performed per Inputprocessing Unit, several XML reports can be generated(maximum being one for each checklist)

Logging Interface Cf. 3.1.1.8 and in case of failed status, a log message shallsummarize the reason of the failed status and the log messagetext shall start with the OLQC_RESULT keyword in order toallow the DPC identifying those messages.

Product Report Interface Cf. 3.1.1.9Exit Code Interface Cf. 3.1.1.10Data exchange repository Cf. 3.1.1.11


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 43 / 63


4. INTERNAL INTERFACES

4.1 Internal Interfaces list

IPF internal interfaces the interfaces between IPF components or interfaces with items provided asCFIs for IPF. The internal interfaces are:

IDP-Orchestrator interface with IDP-SC and OLQC-SC

IDP-SC interface with Decompression CFI software

IDP-SC interface with JP2K COTS software

OLQC-SC interface with Amalfi CFI software

4.2 IDP-orchestrator / IDP-SC & OLQC-SC Interfaces

These interfaces are the one used by IDP-Orchestrator to directly run the every IDP-SC or OLQC-SC individually or a set of chained IDP-SC and OLQC-SC.

The Figure 3 presents these interfaces for the stand-alone configuration mode (when IDP-SC andOLQC-SC are not interfaced with the DPC but with the IDP-Orchestrator).

In both the stand-alone or DPC-driven mode, these interfaces are partially compliant with GMESGeneric PDGS-IPF Interfaces defined in [GEN-PDGSIPF].


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 44 / 63


IDP-SC

IDP-SC

Job Order

Start /Stop

Processing Configuration

Logging

Product Report

Exit CodeIDP-SC

orOLQC-SC

Input PU

Ouptut PU

Auxiliary data

Ancillary data

Aux/Anc data

IPF Workflow PDIs

Task Table

Task Table

Task Table

PDIs input/outputsAuxiliary/Ancillary dataIntermediate PDI-PU

Exe

cu

tio

nsta

tus

Operator

IDP

-SC

orc

he

str

atio

n

IDP

Orc

hes

trato

r

Exe

cu

tion

ord

ers

Figure 3: IPF Interfaces for the stand-alone configuration


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 45 / 63


Note that in the DPC-driven configuration (see Figure 1),the IDP-SC and OLQC-SC can be run bythe DPC using the same interfaces but in that case, interfaces are considered as external to IPF.

As a stub of the DPC in a stand-alone mode, the interfaces between the IDP-Orchestrator and thethe IDP-SC or OLQC-SC are the same as the ones shared between DPC and IDP-SC or OLQC-SC (same description except replacing “DPC” by “IDP-Orchestrator”). The only difference is thatthe IDP-Orchestrator has an additional IPF Task Table interface hereafter described.

4.3 IDP-Orchestrator / IPF Task Table Interface

To run IDP-SC and OLQC-SC individually or chained, the IDP-Orchestrator uses the IPF TaskTable interface (hereafter referred to as Task Table) , which contains:

The number of composing executables (tasks),

The list of input, output and intermediate file types for each executable.

The Task Table is loaded by the IDP-Orchestrator Management Layer to know the exact sequenceof Tasks to be executed and used to instantiate Job Order files.

A Task Table is provided by the IPF Contractor to run every IDP-SC and OLQC-SC individually.

Task Tables are provided as well by the IPF Contractor to run chained sets of IDP-SC and OLQC-SC to demonstrate functional and performance requirements during testing phases.

From the IDP-Orchestrator graphical user interface, a human operator may select a Task Tableamong the set of available Task Tables, activate the execution of IDP-SC and OLQC-SC withactivation of the relevant Task Table, and check the progress and completion of the correspondingprocessing, while IDP-SC and OLQC-SC are launched and controlled by IDP-Orchestrator.

4.4 Decompression CFI Interfaces

The decompression of on-board compressed data is performed by an IDP-SC at the beginning ofL1A product processing chain.

The decompression step is provided through the WICOM CFI and is based on MRCPBG softwarewhich stands for Multi-resolution binary-plane encoding [MDS-SUM-MRCPBG].

IDP-SC interface to MRCPBG shall comply to interface rules defined in the documentationprovided with the CFI.

The MRCPBG v9.0 requires 4 ASCII TXT as configuration files (.conf, .apid, .ih and .fpft files).The decompressed images can be in ENVI (raw+header) or LUM (CNES specifc) format.

Inputs & Start:The MRCPB is launched by the IDP-SC through a command line that takes as argument:

the input binary encoded bitstream;


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 46 / 63


and optional parameters.

Logs:

Progress indications are displayed on stdout during processing (in percentage form).Logs are also sent to stdout.

Stop:Processing can be interrupted at any time with Ctrl-C signal.

Outputs:

Unix shell return code corresponding to the following cases:

0: no error detected during entire execution.

1: one error occurred which shut down the software.

2: one or more warnings appeared during execution.

3 : one error tied to the software parameterization occurred. Error file might be empty.Run the same configuration in silent mode 0 in order to get more details.

4 : software silent mode is incompatible interactive answers during error detection whileprocessing (check the value of the following parameters confirm_premier_crc,confirm_crc, confirm_depassement_pS, confirm_ebbln_generique,confirm_integrite in the used « .conf » file).

generates a decompressed image in the wdir/DECOMP directory,(ENVI image format)

generates a set of information files in the wdir/RESULT directory,

generates report files in the wdir/TRACE directory,

(wdir is the work directory defined in the software’s configuration).

4.5 OLQC-SC/Amalfi interfaces

The OLQC-SC is built upon the Amalfi-2 software [AMALFI-SUM][DRB-SUM],, which is providedby the Agency as a Customer Furnished Item (CFI) and supports generic functionalities forproduct-data decoding and subsequent quality inspections.

The Amalfi-2 software will be wrapped as a stand-alone processor within the OLQC-SC. Finally,the OLQC-SC will generate an XML output from the Amalfi-2 software and stored in the directoryspecified in the checklist (the options shoold be QI-DATA , MPC or both directory).

OLQC-SC interface to Amalfi-2 shall comply to interface rules defined in the documentationprovided with the CFI.


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 47 / 63


4.6 IDP-SC/JP2K interface

The set of IDP-SC in charge of JPEG2000 compression or decompression are making use ofKakadu COTS SW, hence a specific internal interface shall be implemented in order to call Kakadulibrary (the implementation of IDP-SC in charge of JPEG2000 compression or decompression shallisolate the calls to JPEG2000 library in view of a possible replacement of Kakadu COTS byanother JPEG2000 library)


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 48 / 63


5. INTERFACES DEFINITION

The purpose of the “Interface definition” paragraph is to provide a functional description of theflows related to the document with the following fields : Interface Name : gives the interface name Interface number : defines a unique identifier for the interface name Data Type : gives a classification of the interface (note that this data type is an internal identifier

which is also characterized by a Data Type ID) Interface Type : characterisation of the flow type (file, web, email, stream…) which has to be

compliant with the support service definition. Encoding : specifies the type of encoding related to a file (ASCII, binary, mixed, XML) Support description : description of the physical support category of the interface; the support

type can be defined amongst (Ground Link, DCN (Data Circulation Network), DDN (DataDissemination Network), MCN (Media Circulation Network), …).

Support service : specifies the mechanism which will be used to exchange data for thisinterface; the mechanism can be a well known protocol (e.g. TCP/IP) or a bespoke protocoldefined for the CGS (i.e. DC (Data Circulation)).

Satellite : identification of a dedicated spacecraft when relevant. Use/Description : use and description of the interface Mean/Peak Frequency : defines the mean and peak frequency of the interface when it is

relevant (optional field) Size : defines the interface data size of the interface when it is relevant (optional field) Data structure : defines the internal format of the interface which contains :

Type : defines the type of the internal format Description : points to the internal format description (e.g. a zip which included into a main

zip file attached to the DI-MICD document) File Template Name : provides a filename template / convention for the files (only applicable

for file interfaces) Document reference : identifies the document(s) leading to this interface definition when it is

relevant Allocation : clearly identifies all the physical interfaces involved between centres and

components.Note that filenaming conventions and standards are described in the following sections.

5.1 External Interfaces definition

5.1.1 Interface Name: DEM to IDP

Interface Name: DEM to IDP

Interface number: IF0261

Data Type: 110 AuxData OGCD DEM

Interface Type: File

Encoding: mixed

Support description: Ground link

Support service: Dedicated Data Transfer


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 49 / 63


Satellite:

Use_Description: This Digital Elevation Model is composed of two layers :- The first layer is a Global Coarse DEM. It is called GLOBE. This layerdefines an altitude information with an average value for all the point on aglobal grid with a kilometric resolution. The data are provided by 1 x 1cells. Two sources of data have been used to generate this GLOBE (TheNOAA GLOBE DEM, The GTOPO30, result of 8 sources of altimetric datadelivered by the USGS.) All the information from these DEM has beenreformated.The second layer is a more precise DEM respecting the DTED1 format(resolution of 90m). This data comes from the SRTM data (Shuttle RadarTopographic Mission) distributed by NASA(http://edc.usgs.gov/products/elevation.html)

File Template Name:S2X_OPER_AUX_DEMXXX_PDMC_YYYYMMDDTHHMMSS_VYYYYMMDDTHHMMSS_YYYYMMDDTHHMMSS

Document Reference:Ref Document name

GPP-DEM

Allocation:Component Origin Center Origin Component Destination Center Destination

DPC MPC_CC IDP MPC_CCDPC PAC1 IDP PAC1DPC CGS IDP CGS

5.1.2 Interface Name: DPC - IPF control

Interface Name: DPC - IPF control


Data Type: 210 M&C IPF control


Encoding: mixed



Satellite:

Use_Description: Interfaces to start, stop, monitor and control an image processing by anIDP-SC/OLQC/Any HP.This includes:- JobOrder interface- Processing Configuration interface (typically maximum RAM size to beused)- Start interface (executing a job through a commmand line with relatedJobOrder as an argument)- stop interface (kill signal to a Job)Data are exchanged through a data exchange repository in the disk filesystem.


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 50 / 63


Data Structure:Type: XSDDescription: IPF_JobOrder_1.0.zip


DPC CGS IDP CGSDPC CGS OLQC CGSDPC CGS Hosted IDP CGSDPC PAC1 IDP PAC1DPC PAC1 OLQC PAC1DPC PAC1 Hosted IDP PAC1DPC MPC_CC IDP MPC_CCDPC MPC_CC OLQC MPC_CC

5.1.3 Interface Name: DPC - IPF monitoring

Interface Name: DPC - IPF monitoring


Data Type: 220 M&C IPF monitoring

Interface Type: File / Stream

Encoding: XML



Satellite:

Use_Description: IDP/OLQC/HP monitoring and reporting to DPC in the frame of imageprocessing execution.This includes:- Logging interface from IDP to DPC- Production Report interface from IDP to DPC- Exit Code returned from IDP to DPCData are exchanged through a data exchange repository in the disk filesystem.

Data Structure:Type: XSDDescription: IPF_Report_1.0.zip


Hosted IDP PAC1 DPC PAC1IDP MPC_CC DPC MPC_CCHosted IDP CGS DPC CGSIDP PAC1 DPC PAC1IDP CGS DPC CGSOLQC CGS DPC CGSOLQC PAC1 DPC PAC1


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 51 / 63


OLQC MPC_CC DPC MPC_CC

5.1.4 Interface Name: ECMWF AuxData to IDP

Interface Name: ECMWF AuxData to IDP


Data Type: 111 AuxData OGCD ECMWF


Encoding: mixed



Satellite:

Use_Description: Auxiliary Data File which contains weather forecast parameters delivered byEuropean Centre for Medium-Range Weather Forecasts (ECMWF) toPDGS for 3 geophysical variables: total column ozone, total column watervapour and mean sea level pressure. There is a value per tile of a grid of0,141 resolution tiles in latitude and longitude, computed by ECMWF at 00UTC (data available at 04 UTC) and 12 UTC (data available at 16 UTC)every day with a 42 hours coverage in 3 hours steps (from 09 UTC in theearly morning bunch, from 21 UTC in the afternoon bunch). The data areGRIdded Binary format GRIB described athttp://www.wmo.int/pages/prog/www/WDM/Guides/Guide-binary-2.html

File Template Name:S2X_OPER_AUX_ATMCOR_CCCC_YYYYMMDDTHHMMSS_VYYYYMMDDTHHMMSS_YYYYMMDDTHHMMSS


DPC CGS IDP CGSDPC PAC1 IDP PAC1DPC MPC_CC IDP MPC_CC

5.1.5 Interface Name: GIPP to IDP

Interface Name: GIPP to IDP


Data Type: 113 AuxData OGCD GIPP


Encoding: XML


Support service: direct

Satellite:

Use_Description: Input parameters for processings: set of several tens of XML filesassociated to processing components (IDP) to define a set of parametersand their values. The GIPP to be used are indicated to IDP as specified


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 52 / 63


files in the Job order. GIPP are not copied, but reading accessible by theIDP_SC.

File Template Name:S2X_OPER_CNF_GIPPXX_CCCC_YYYYMMDDTHHMMSS_VYYYYMMDDTHHMMSS_YYYYMMDDTHHMMSS


EOFFS-PDGS


DPC PAC1 IDP PAC1DPC MPC_CC IDP MPC_CCDPC CGS IDP CGS

5.1.6 Interface Name: GIPP to OLQC

Interface Name: GIPP to OLQC


Data Type: 113 AuxData OGCD GIPP


Encoding: XML



Satellite:

Use_Description: Input parameters for processings: set of several tens of XML filesassociated to processing components (IDP) to define a set of parametersand their values. The GIPP to be used are indicated to IDP as specifiedfiles in the Job order. GIPP are not copied, but reading accessible by theIDP_SC.

Data Structure:Type: XSDDescription: olqcGipp_v1.0.xsd

File Template Name:S2X_OPER_CNF_GIPPXX_PDMC_YYYYMMDDTHHMMSS_VYYYYMMDDTHHMMSS_YYYYMMDDTHHMMSS


EOFFS-PDGS


DPC MPC_CC OLQC MPC_CCDPC CGS OLQC CGSDPC PAC1 OLQC PAC1


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 53 / 63


5.1.7 Interface Name: GRI to IDP

Interface Name: GRI to IDP


Data Type: 114 AuxData OGCD GRI


Encoding: mixed

Support description: Dedicated Point-to-Point


Satellite:

Use_Description: The Global Reference Image (GRI) is a set of Sentinel-2 mono-spectralLevel-1B products (granules, metadata) covering all land regions within theacquisition plan. The GRI is used during geometric processing to improvethe geo-location accuracy and to reach the absolute geo-location accuracyand multi-temporal co registration requirements.

File Template Name:S2X_OPER_AUX_GRIXXX_PDMC_YYYYMMDDTHHMMSS_VYYYYMMDDTHHMMSS_YYYYMMDDTHHMMSS

Mean frequency: Once a day


DPC CGS IDP CGSDPC PAC1 IDP PAC1DPC MPC_CC IDP MPC_CC

5.1.8 Interface Name: IERS AuxData to IDP

Interface Name: IERS AuxData to IDP


Data Type: 115 AuxData OGCD IERS


Encoding: mixed



Satellite:

Use_Description: The International Earth Rotation Service (IERS) Bulletin A, which containsrapid determinations for earth orientation parameters (Earth orientationparameters x/y pole, UT1-UTC and their errors).

File Template Name:S2X_OPER_AUX_UT1UTC_PDMC_YYYYMMDDTHHMMSS_VYYYYMMDDTHHMMSS_YYYYMMDDTHHMMSS


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 54 / 63



DPC CGS IDP CGSDPC MPC_CC IDP MPC_CCDPC PAC1 IDP PAC1

5.1.9 Interface Name: IPU to IDP

Interface Name: IPU to IDP


Data Type: 520 Product IPU


Encoding: mixed

Support description: Dedicated Point-to-Point


Satellite:

Use_Description: Image Processing Unit from the DPC to the IDP

Mean frequency: 15 every day per satelliteSize: 36 Gb for 10mn downlink


DPC MPC_CC IDP MPC_CCDPC CGS IDP CGSDPC PAC1 IDP PAC1

5.1.10 Interface Name: IPU to OLQC

Interface Name: IPU to OLQC


Data Type: 520 Product IPU


Encoding: mixed



Satellite:

Use_Description: Image Processing Unit generated by DPC in internal format, to be checkedby OLQC.



& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 55 / 63


DPC PAC1 OLQC PAC1DPC MPC_CC OLQC MPC_CCDPC CGS OLQC CGS

5.1.11 Interface Name: OLQC Report to DPC

Interface Name: OLQC Report to DPC


Data Type: 640 Report Quality


Encoding: XML



Satellite:

Use_Description: The result of the OLQC report at granule or tile level and at product level

Data Structure:Type: XSDDescription: olqcReport_v1.0.xsd

File Template Name:S2X_OPER_REP_OPOLQC_CCCC_YYYYMMDDTHHMMSS_VYYYYMMDDTHHMMSS_YYYYMMDDTHHMMSS


OLQC MPC_CC DPC MPC_CCOLQC PAC1 DPC PAC1OLQC CGS DPC CGS

5.1.12 Interface Name: POD Product to IDP (Contingency)

Interface Name: POD Product to IDP (Contingency)


Data Type: 116 AuxData OGCD POD Product Orbit


Encoding: mixed



Satellite:

Use_Description: Predicted or Restituted orbit data for image processing in case ofinsufficient accuracy of the onboard naviation solution.These data are some ancillary data in the general DPC-IDP interface. Theyare described separately from other ancillary data because they come fromMPC-CC/POD through AI and DPC, while other ancillary data for IDP aregenerated by DPC itself.


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 56 / 63


File Template Name:S2X_OPER_MPL_PODPRE_MPC__YYYYMMDDTHHMMSS_VYYYYMMDDTHHMMSS_YYYYMMDDTHHMMSS


DPC CGS IDP CGSDPC MPC_CC IDP MPC_CCDPC PAC1 IDP PAC1

5.1.13 Interface Name: Processed IPU from IDP

Interface Name: Processed IPU from IDP


Data Type: 340 MRF Report


Encoding: mixed



Satellite: S2A/B

Use_Description: Those are the output IPU as processed by the IDPs


IDP MPC_CC DPC MPC_CCIDP CGS DPC CGSIDP PAC1 DPC PAC1

5.1.14 Interface Name: Quality Annotated IPU from OLQC

Interface Name: Quality Annotated IPU from OLQC


Data Type: 0


Encoding: mixed



Satellite: S2A/B

Use_Description: Those are the IPU have annotation by the OLQC after Quality Check.


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 57 / 63



OLQC PAC1 DPC PAC1OLQC MPC_CC DPC MPC_CCOLQC CGS DPC CGS

5.2 File Naming conventions

The applicable file naming convention used for the following files detailed hereafter. Note that thisfile naming convention is compliant to [EOFF-PDGS] :

MMM_CCCC_TTTTTTTTTT_<instance_id>.AAA

Where :MMM is the Mission ID (3 characters), possible values are :

- S2A for Sentinel 2A specific- S2B for Sentinel 2B specific- S2_ for generic files (N/A for files provided to the FOS)

CCCC is the File Class (4 characters), possible values are :- OPER for routine operations

TTTTTTTTTT is the File Type (10 characters) where :- FFFF = File Category where :

o AUX_: auxiliary data files;o CNF_: configuration fileso LNK_ : metalink files ;o LOG_: log files;o MPL_: mission-planning files;o MAN_: manœuvre files;o MTD_ : metadata ;o REP_: reports;o STO_ : Telemetry Data Storage (acquisition)o TLM_ : HK Telemetry provide by the FOSo PRD_: productso REQ_: file based requesto RSP_: file based responseo Others TBD

- DDDDDD = Semantic Descriptor (for each file type)o AUX_:

1. DEMxxx : Digital Elevation Model (xxx : DEM Id)2. GRIxxx (TBC) : Ground Reference Image (xxx GRI Id)3. ENV___ (TBC) : Environemental data4. UT1UTC: IERS5. ATMCOR: ECMWF


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 58 / 63


6. GPS___ (TBC): IGS7. TBD8. SAD___:

o CNF_:1. OPcccc : cccc component name2. GIPPxx (GIPP id to define)

o LOG_:1. xxxx__ : Monitoring log for the component cccc

o MAN1. MCSF__ : Manoeuvre Calibration Scale Factor2. MACP__ : Manoeuvre Acceleration Profile File3. SMPR__ : Manœuvre - Satellite Mass Properties Report4. GNDTRK : Ground Track Evolution5. MPRED_ : Manoeuvre Prediction

o MPL_:1. FScccc (cccc = Destination Site Centre) : Schedule Plan2. TLEPRE : Two Lines Element3. SAPccc : Station Acquisition Plan

ccc : Identification of the destination station4. PPLccc : Station pass list

ccc : Identification of the destination station5. NPPF__ : Nominal Payload Planning File6. NPIF__ : Nominal Plan Increment File7. SSCF__ : S/C Safety Constraints8. EDRPOI : EDRS pointing information9. EDRAVS : EDRS availability segments10. EDRBKP : EDRS booking plan11. ORBREF : Reference Orbit12. ORBPRE : FOS Predicted Orbital Product13. ORBRES : FOS Restituted Orbital Product14. PODPRE : POD Predicted Orbital Product15. PODRES : POD Restituted Orbital Product16. NUCTB_ : NUC Table17. MSIIPT : Instrument Paramater Table (FEE ou IPS)

o MTDT :1. METARC : Metadata report containing Archived Products2. METUPD : AI Metadata Update Report3. METDEL : AI Metadata Delete Report4. QCCMPI : MPI generated by the QCC

o REP_:1. STNACQ : For station acquisition reports2. PASS_n : For DFEP Acquistion Report (generaterd by the

DFEP N°n)3. STNUNV : For station unavailability reports4. SUP___ : FOS + Satellite Unavailability Reports5. EDRUNV : EDRS Unavailability report6. NPIFCC : For NPIF Constraints Checker Report7. AIUNV_ : AI Unavailability8. OPcccc : For operational reports of the component cccc


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 59 / 63


9. PERFO_ : For performance reports;10. OAR___: Optical Acquisition Report11. DISCLA: Product disclaimer12. CHF__:TC History Report13. FCHF_: Filter TC History Report14. ARC___: Archive Report15. UPD___: Update Report16. E2EMPR : Mission Performance report generated by

E2ESPM17. E2EPER : Performance report generated by E2ESPM18. QCCPER : Performance Report generated by QCC

----------- ngEO / OLIB -------------------------------------19. METARC : ngEO Metadata report containing Archived

Products20. METUPD : ngEO Metadata Update Report21. METDEL : ngEO Metadata Delete Report22. BRW_NG: ngEO Browse Report23. BRW_OL: OLIB Browse Report24. TCI___: OLIB TCI Report

o STO1. VCDU__ :

o TLM1. EDDSHK : EDDS HKTM response

o PRD_1. DTSTRP: datastrip2. HKTM__: hktm granule3. TORRNT: torrent file

o REQ_1. PRDDWN: user product download

<instance_id> = <File Instance ID mandatory prefix>_<File Instance ID free string><File Instance ID mandatory prefix> = ssss_yyyymmddThhmmsswhere:- ssss is the Site Centre of the file originator (N/A for MRF files provided to the FOS) - yyyymmddThhmmss is the Creation Date

The Creation Date is a 15 characters field defined according to the format:YYYYMMDDThhmmss, composed of:- 8 characters, all digits, for the date: “YYYYMMDD” - 1 uppercase T: “T” - 6 characters, all digits, for the time: “hhmmss” The date-time value contained within the Creation Date must be expressed in UTC androunded down to the second. For instance, a file created on 31st December 2013 at23:59:59.8shall be reported in the filename as “20131231T235959”.

Notes:


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 60 / 63


- It is assumed that the creation frequency of a file of a given File Type is never higherthan one second, hence ensuring the filename uniqueness;- When the same file (i.e. for the same applicability period) is re-generated more thanonce, the last one delivered becomes the official applicable (i.e. with newest CreationDate). This rule avoids the need to define a file version number and therefore simplifies thefile generation itself and the configuration and control activities on both the originator andthe consumer(s).- In case of leap second introduction, ss is allowed to be 60.

Based on every file-type, the File Instance ID free string allows furthercharacterisation of the scope of every file as specifically relevant to the type of file.

<File Instance ID free string> = Ivvvvvvvv…where:- I is a one-character option Identifier (e.g. ‘C’, ‘O’, ‘V’, etc); - vvvvvvvv… contains the trailing Option Value. This approach allows an unambiguous and data-driven recognition of the optional suffixensuring a deterministic parsing.

Possible option Id:C: name of the component originator of the file coded on 5 caracters max (CPDC)V: validity Period (VyyyymmddThhmmss_YYYYMMDDTHHMMSS)S: applicability start (SyyyymmddThhmmss)O: Orbit period (O12734)Other TBD…

- AAA is the file extension forced to EOF

Potential examples for the Site Centre values based are proposed below:- XXXX reserved for testing or files from outside the ground segment- PDMC assigned to the Payload Data Management Centre- MPC_ assigned to the Mission Performance Centre / Coordinating Center- POD_ assigned to the OFL POD Service- EDRS assigned to the EDRS Exploitation Segment- CDS_ assigned to the Coordinated Data-access System- SPS_ assigned to System Performance Simulator- FOS_ assigned to Flight Operation Segment- CGSx assigned to Core ground station where x is one digit, or 4 characters acronym for the CGSname (SVLB for instance)- PACx assigned to the Processing Archive Centre x, where x is one digit- LGSx assigned to the Local Ground Station x, where x is one digit, or 4characters acronym for the LGS name- CDMx assigned to the Collaborative Data-Access Mirror x, where x is one digit, or 4 charactersacronym for the CDAM name- ESL_ assigned to Expert Support Laboratories- MDP_ assigned to the MSI Decompression Software


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 61 / 63


Potential examples for S2PDGS Components values

ACQ_ LTA_DPC_ DAM_IDP_ DC__OLQC MCC_DFEP M_C_AI__ POD_DAGS S2MPDAGB ESPMDAGC QCC_DAX_ OLIBNGEO

5.3 Standards

As far as possible, standards are used to implement the S2 PDGS internal and external interfaces.This section gives a quick overview of such standards:

Exchange of ASCII file: XML and XSD. Image product: INSPIRE, SAFE for Sentinel, JPEG2000, Image masks: GML (Geographic markup Language) Geolocation header in JPEG2000: GML JP2. Inter component interface: Web services.

- The standard Extensible Markup Language (XML) is a set of rules described by the W3C toencode ASCII values in a structured file. The design goals of XML emphasize simplicity, generality,and usability over the Internet.In order to improve the control of validity of the XML files, the W3C completes the XML with anadd-on language: XML Schema Definition (XSD). The aims of XSD are to provide a grammar todescribe syntax and semantics rules to control the XML files.- JPEG 2000 is a wavelet-based image compression standard. It was created by the JointPhotographic Experts Group committee in the year 2000 with the intention of superseding theiroriginal discrete cosine transform-based JPEG standard (created 1992). The standardizedfilename extension is .jp2 for ISO/IEC 15444-1 conforming files and .jpx for the extended part-2specifications, published as ISO/IEC 15444-2. The registered MIME types are defined in RFC3745. For ISO/IEC 15444-1 it is image/jp2. JPEG-2000 supports both lossless and lossycompression and allows JPIP steaming optimizing bandwidth by allowing quick access to therequested area and detail level.- INPIRE is a European directive aiming at increasing interoperability for a Spatial DataInfrastructure (SDI). This means standardization of metadata and data. The European CommissionINSPIRE Geoportal (inspire-geoportal.ec.europa.eu/) provide definition of INSPIRE metadata setand provides means of online INSPIRE compliance validation. The data definition is still a on-goingactivity.


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 62 / 63


- SAFE for Sentinel is a tailoring of the ESA SAFE format, allowing to include in format definitionreference to external standards (GeoTIFF, JP2000 etc…),- GML and JP2 Georeferencing: The Geography Markup Language (GML) is the XML grammardefined by the Open Geospatial Consortium (OGC) to express geographical features. OGC hasdefined too a metadata standard for georeferencing JPEG2000 images with embedded XML usingthe Geography Markup Language (GML) format: GML in JPEG 2000 for Geographic ImageryEncoding (GMLJP2), version 1.0.0, dated 2006-01-18.JP2 and JPX files containing GMLJP2 markup can be located and displayed in the correct positionon the earth's surface by a suitable Geographic Information System (GIS), in a similar way toGeoTIFF images.- A web service is a method of communication between two electronic devices over the web.Several standards are defined to implement these exchanges.Web services (WS-*) publish their features (operations) as remote executable services. Theirspecifications are based on SOAP and WSDL (Web Service Description Language in XML format).A WSDL describes the following elements:Service: It is a container for a set of system functions;Port: It defines the address or connection point to a web service;Binding: Defines the SOAP binding type (RPC/Document) and Transport (SOAP);Port Type: This element defines the operations and the message used to perform the

operations;Operation: This element contains the operations (function) that can be performed;Message: A message corresponds to an operation. It contains all information need to

perform the operationElement: An element contains a unique name and data type. It can consider as a

parameter of the operation.XSD file: XML Schema Definition that can be embedded into the WSDL or in a separate

file. If the XSD is defined in a separate file, the namespace specifies the location (URL)where the XSD is available.

SOAP message is sent over the HTTP protocol means not only that the delivery mechanism iswidely available but also that SOAP is able to get past firewalls that pose problems for othermethods.


& OLQC


ISSUE : 03

DATE : 06/04/12

PAGE : 63 / 63


END OF DOCUMENT