giancarlo rivolta [email protected] esrin, 11 october 2012 rss support to scientific...

Giancarlo [email protected]

ESRIN, 11 October 2012

RSS support to scientific exploitation of EO data

mailto:[email protected]

No. 2© Logica 2012. All rights reserved

Summary

RSS Overview

RSS G-POD process

RSS flexible resources for on-demand processing

Success stories

Future directions

GPOD demo

Questions & Answers

© Logica 2012. All rights reserved No. 3

Research and Service Support

• The ESA Research and Service Support (RSS) service provides resources for supporting Earth Observation (EO) data exploitation

• RSS users are EO Principal Investigators and EO Service Providers

• The environments made available to RSS users are: • Grid Processing On-Demand (G-POD) • Service Support Environment (SSE) • E-collaboration environment (Join&Share) • OGC services (WMS, WFS, WCS) • Virtual Reality Theatre (VRT) • Knowledge-based Earth Observation (KEO) • Reference Data Sets (RDS) • EO Ontology search engine

• More than 3500 people are registered as RSS users

• On the industry side Logica delivers the RSS service as part of the ESRIN O&M Framecontract

• More information available from the RSS Portal (rssportal.esa.int )

Let’s focus on G-POD

RSS Overview


Research Process


Principal Investigators

• EO algorithms delivery • Data type and range indication

• Output validation

• On-demand EO data processing

• Use of produced data (projects delivery, publications, etc)

RSS

• Data are made available in the RSS catalogue

• Algorithm porting

• Integration design

• Implementation

• Test and validation (involving the PI)

• On-demand EO data processing

• Delivery

RSS G-POD process steps

© Logica 2012. All rights reserved

No. 6

Flexible Infrastructure satisfies: - HW requirements - Connectivity requirements - SLA (HA, help desk, ticketing systems, etc.)

On-demand processing service:

Technology

Infrastructure

Application

ESRIN- 172 cores- 400 TB

UK-PAC- 88 cores- 300 TB

Flexible Infrastructure - 10-200 cores- 1-10 TB

EO Scientists Principal Investigators

deliveryProcessEO data

RSS Flexible Resources

© Logica 2012. All rights reservedNo. 7

RSS G-POD Team

On-demand Processing: team composition and responsibility

• The G-POD team is composed by 4 selected highly skilled resources

• Key areas of responsibility are:

• Interaction with Principal Investigators • Algorithm Understanding and proposal evaluation • Resources planning (data, storage, integration, capacity, etc) • Algorithm integration (porting, design, implementation, test) • Data management • Processing environment set-up • User support (for on-demand processing) • Bulk Processing management

• Key skills are:

• Remote sensing background / EO data exploitation expertise • Software development /software integration • Virtualization technology • Grid / Cloud technology • Operations and Maintenance • Scientific goal orientation


Success Story: MGVI

MERIS Global Vegetation Index Cat-1 project

• Objective: Development of MERIS derived products based on the Fraction of Absorbed Photosynthetically Active Radiation (FAPAR) algorithm

• IPR: MGVI algorithm developed and owned by the Joint Research Centre

• Evolution and time span: Several GPOD-MGVI versions integrated since 2007, bulk re-processings over the entire ENVISAT mission (2002-2012)

• Outcomes/Benefits: • Access to online data for bulk re-processing • Systematic global products published on ESA Earthnet Portal • Regional products distributed via RSS SSE and RSS OGC service as well • On-demand MGVI regional available • MGVI Europe / Africa distributed to JRC for creating products for desertification

monitoring• Several scientific papers published • MGVI delivered as complementary information to Fire detection products over

Senegal

From JRC contribution to the paper “A Model for the Scientific Exploitation of Earth Observation Missions”: RSS supported the process of algorithm development, test and deployment over a general purpose processing on demand environment dedicated to the scientific exploitation of EO data generated by ESA missions, as well as the provision of tools and

environments in support to that process.


Success Story: KLIMA-IASIThe KLIMA-IASI ESA Cat-1 project was initially structured along 3 phases1- The KLIMA-IASI application is integrated in G-POD• The data set used consists of: IASI data levels L1B, L1C and L2, in EPS format; for the month of July 2008; • The results of the application on this data set are used to determine if results produced from Level L1B data are better than the

results produced from Level L1C;• The phase is concluded with the decision to use Level L1B or Level L1C data; • Also, during this phase, processing performance is measured and the processing time for the qualification campaign is estimated 2- IASI Level L1 (B or C) and L2 are ingested for the whole year 2009, in EPS format, up to 5Tb • As a "qualification campaign", the KLIMA-IASI application is run upon the year 2009 data and results are used to validate the

quality of the algorithm in relation to Gosat measurements; • Also, during this phase processing performance is measured and the exploitation scenario defined 3- Full IASI Mission data and EumetCast NRT are systematically ingested in G-POD• The KLIMA-IASI application performance is optimized in line with NRT requirements;• The KLIMA-IASI application is enhanced to accept BUFR data format;• KLIMA-IASI application is provided as a service to the User Community

• IPR: KLIMA-IASI algorithm developed and owned by the IFAC-CNR, Florence

• Evolution and time span: Several GPOD-MGVI versions integrated since 2011, bulk processing on dataset from 1 year IASI data (Mar 2010 – Feb 2011)

• Outcomes/Benefits: • On-demand KLIMA-IASI service available to PI for pre-analysis • Access to cloud resources for flexible capacity to meet the deadline • Access to online data for bulk re-processing (11 out of 12 months processed) • Several scientific papers published and presentations at conferences • Process speeded-up by 2 orders of magnitude (flexible)


Success Story: SMOS

SMOS testbed

• Objective: Provide a flexible test environment to support ESA GQ users for L1 calibration and developers for L2 OS

• Evolution and time span: Several SMOS L1 and L2 OS re-processing campaigns with custom configuration (recently processed sample SMOS L1 products over 25 months (from June 2010 to June 2012, 1728 L1A/B + 6576 CRS products), SMOS L2 OS 1 month)

• Outcomes/Benefits:

• Fast integration of new versions (if ICD unchanged) • Access to online data for bulk re-processing • Access to flexible cloud resources for meeting deadlines • On-demand SMOS L1 available for GQ • On-demand SMOS L2 OS available for developer • SMOS L0 NRT ingestion chain set-up for L1 NRT custom re-processing • Testbeds required for other sensors as well

• Open call for proposal published on EOPI


Future directions

In the light of the RSS success story the following should be considered

• RSS model as basis for the definition of a harmonized model of research and service support

• Federation of payload data ground segment services to the scientific community


Contacts & further information

Giancarlo Rivolta [email protected] Team [email protected] Team [email protected] Portal gpod.eo.esa.intJoin&Share wiki.services.eoportal.org




http://gpod.eo.esa.int/

http://wiki.services.eoportal.org/


GPOD demo

Now the GPOD Team will give a 15-minute GPOD demo GPOD Portal: gpod.eo.esa.int

http://gpod.eo.esa.int/


Questions and Answers

Q&A?

Logica is now part of CGI, a leading business and technology services company focused on helping clients achieve results. Since our founding in 1976, we've operated upon the principles of sharing in clients' challenges and delivering quality services to address them. With 72,000 professionals in more than 40 countries, we have the presence, expertise and complete IT services to meet clients’ business needs anywhere, anytime. We provide the responsiveness and accountability of a true local partner while offering the global scale, talent and services needed to meet your evolving needs. Visit www.cgi.com and www.logica.com to learn more.

Thank youGiancarlo Rivolta

Logica xxx Visitor address Street Postcode City Country www.logica.comContact: Name Function T: +XX (0) 000 00 000 000 F: +XX (0) 000 00 000 000 E: [email protected]

http://www.cgi.com/

http://www.logica.com/

giancarlo rivolta [email protected] esrin, 11 october 2012 rss support to scientific...

Documents