a multi-satellite, multi-sensors and multi- temporal ... · muscate processing current status 14...
Post on 23-Sep-2020
7 Views
Preview:
TRANSCRIPT
MUSCATE: A MULTI-SATELLITE, MULTI-SENSORS AND MULTI-
TEMPORAL VERSATILE PLATFORM DESIGNED
FOR THEIA
Joëlle DONADIEU
CNES – Toulouse, France
On behalf of main authors:
C L’HELGUEN, P.LASSALLE, P.KETTIG - CNES
T.SEGUR, D.CLESSE, N.NEROME - CAPGEMINI
WHAT IS THEIA ?
2
• THEIA: French national multi-agency organisation for the studies of continental surfaces
• THEIA promotes the use of satellite data by scientific community and institutional actors.
MUSCATE OVERVIEW
MUSCATE = Initially designed as THEIA processing facility of HR
optical satellite products
3 Families of Satellites:
• SPOT 1 to 5: Spot World Heritage project
• LANDSAT 5-7-8
• SENTINEL-2 (A & B)
Already more than30 years of data !
MUSCATE OVERVIEW
4
MUSCATE = Initially designed as THEIA processing facility of HR
optical satellite products
3 Families of Satellites:
• SPOT 1 to 5: Spot World Heritage project
• LANDSAT 5-7-8
• SENTINEL-2 (A & B)
3 Types of Products (CEOS standard):
• Level 1C: ortho-rectified product in TOA reflectance
• Level 2A: level 1C product in surface reflectance
• Level 3A : temporal synthesis of level 2A products
But designed to be evolutive :
• New Satellites
• New type of Products
DataCatalog
ProcessingOrchestrator(PHOEBUS)
MUSCATE ARCHITECTURE
5
Production facility
Distribution facility
Acquisition facility
Exchange space
THEIA shared services – Websitetheia.cnes.fr
Archiving
SPOT, LANDSAT, SENTINEL2 DATA
Administrators
MUSCATE ARCHITECTURE
6
Production facility
Distribution facility
Catalog HMI PHOEBUS HMIAcquisition facility
Exchange space
Processing on Cluster
PHOEBUS orchestrator
Administrator / Catalog
Catalogdata base
THEIA shared services – Websitetheia.cnes.fr
Archiving
SENTINEL2
DataCatalog
ProcessingOrchestrator(PHOEBUS)
MUSCATE: PRODUCTION FACILITY
EXAMPLE OF SENTINEL 2 PROCESSING
7
MAJA algorithm (multi-temporal) :
• Clouds (and shadows) detection,
• Aerosol Optical Thickness (AOT) & Water Vapour estimation,
• Atmospheric effects Correction
• Cirrus correction
MUSCATE: PRODUCTION FACILITY
EXAMPLE OF SENTINEL 2 PROCESSING
8
MAJA algorithm (multi-temporal) :
• Clouds (and shadows) detection,
• Aerosol Optical Thickness (AOT) & Water Vapour estimation,
• Atmospheric effects Correction
• Cirrus correction
Initialisation phase (backward)
t0
t0 + 5t0 + 10
t0 + 15t0 + 20
t0 + 25
t0t0 + 5
t0 + 10t0 + 15
t0 + 20t0 + 25
Level 1C
t0
Level 2A
MUSCATE: PRODUCTION FACILITY
EXAMPLE OF SENTINEL 2 PROCESSING
9
MAJA algorithm (multi-temporal) :
• Clouds (and shadows) detection,
• Aerosol Optical Thickness (AOT) & Water Vapour estimation,
• Atmospheric effects Correction
• Cirrus correction
Initialisation phase
Nominal phase (recursive)
t0
Level 2A
t0 + 5
Level 1C
t0
Level 2A
t0 + 5
Level 2A
MUSCATE: PRODUCTION FACILITY
EXAMPLE OF SENTINEL 2 PROCESSING
Retrieve of Level-1C products on PEPS platformand ingestion in MUSCATE
catalog
MAJA L2 processing(atmo. correction)
Dispatch to MUSCATE
distribution facility
Archiving
MUSCATE catalog database
REAL
PREDICTED – available inputs
PREDICTED – unavailable inputs
LEGENDE
Level 1C
Level 2A
MUSCATE Catalog(tasks list with all available
inputs)
Level-1C
Level-2A
Level-2A already avail.
PHOEBUS Orchestrator
L2 processing T1L2 processing T2L2 processing T3PHOEBUS pooling
Processingstart
PHOEBUS pooling
Processingstart
End of processing
End of processing
HPC Cluster
10
End of processing
Processingstart
PHOEBUS pooling
Snow L2B processing
Level 2B Snow
Level-2B Snow
11
Snow detection from Sentinel2 Level-2A products
on selected mountain areas defined by THEIA users
MUSCATE: PRODUCTION FACILITY EXAMPLE OF SENTINEL 2 PROCESSING
0
250
500
750
1000
2014 2016 2018 2020 2022 2024 2026
Vo
lum
e in
To
Years
Input
Output
Total
MUSCATE PROCESSING CAPACITY
12
MUSCATE : an Ambitious Project …
• Automatically process up to 2000 products a day (SPOT / LANDSAT /
SENTINEL-2)
• Offer reprocessing capabilities
• Cumulative Volume of MUSCATE data ~1 PBSE
NTI
NEL
-2A
SEN
TIN
EL-2
B
~1 PetaByte in 2024
MUSCATE PROCESSING CAPACITY
13
MUSCATE : an Ambitious Project …
… with “limited budget”!
• Use of CNES HPC centre (resources shared among projects)
• Currently
• Limited to 150 CPU
• 0,5 PB Common storage means
• Extensive re-use of CNES software components
MUSCATE PROCESSING CURRENT STATUS
14
MUSCATE : ramp-up phase in 2018
• Hardware architecture upgraded this year
• Resources and parameters tuning
• Sentinel2A& 2B Level 2A automatically produced on geographic area defined by THEIA users (equivalent to 6 times France area)
• Automatically processed up to 900 SENTINEL-2 L2A products a day with increase production to 1350 products / day
• Cumulative MUSCATE data ~ 140 000 generated data
SEN
TIN
EL-2
A
SEN
TIN
EL-2
B
0
10000
20000
30000
40000
50000
60000
70000
80000
90000
100000
18/08/16 26/11/16 06/03/17 14/06/17 22/09/17 31/12/17 10/04/18 19/07/18
S2-N2B
S2-N2A
SWH-L1C
OUTLOOK AND CHALLENGES
15
Host of new scientific processing chains
Temporal synthesis Level-3A
1st batch : July 2018
Automatic production : end of 2018
OUTLOOK AND CHALLENGES
16
Landsat 8 production :
on France area
Level 2A products (projected on S2 grid) : september 2018
Level 2B-snow products on mountain area
Reprocessing of level 2A Sentinel2A&2B products :
with MAJA 3.1
Using CAMS and GRIB meteo data
Autumn 2018
Venµs production :
Level 2A products
Using CAMS and GRIB meteo data
Level 3A products
OUTLOOK AND CHALLENGES
17
Muscate – SAR
Integration of new processing chains in CNES HPC for
FORM@TER datapole to distribute Sentinel1 interferometry
products
Coregistered SLCs, interferograms and spectral diversity corrections
Stack of coregistered interferograms Velocity maps
Time
OUTLOOK AND CHALLENGES
18
Muscate cloud compatible
Virtualization of MUSCATE environment (compatible
DIAS/AWS/…)
Promote THEIA processing chains in other contexts and other
large areas of interests
Provide massive processing capability deployable outside
CNES in the frame of partnerships and collaborations
top related