Implementing WMS interface on Imagery Database (BDI)

Frédéric GUILLAUD

DSI/DEV

EGOWS Meeting

KNMI, June 2009

2

Overview

Context– Towards SOA for production and workstations (Marie-Françoise talk)

– DWG Meteorology and WMS IE

Soprano imagery database (BDI) WMS implementation Issues Still to do Conclusion

3

Context

IT SOA pilot project Goals

– Improve our skill with SOA architecture and tools (ESB)

– Improve our skill with OGC standards

– Early identify issues

– Make technical decisions

DWG meteorology IE on WMS– Calendars compatibles (Over 6 month by the end of 2009)

– WMS BDI may be a service involved in IE experiments

– To do this, WMS should be deployed in DMZ

– This lead to security issues

4

Soprano Imagery database (BDI) : Before

RDBMS Oracle 3 days real-time data online (“blob” data type). Data is not archived. About 250 datasets Legacy catalogue content not always consistent … Native formats

– BUFR, TIFF-MeteoFrance (TIFF with - open - specific MF tags…)

Native extent depends on dataset origin (MSG, METOP, other geostationary platforms, radar data ..)

Native CRS– “Space View” (One-Point Perspective Projection ?) and “Polar Stereographic”

Data access services– Low level (application binary executable)– No API available– No remote access possible

5

Soprano Imagery database : product overview

Basic radar reflectivity measurements (for each radar) Mosaics

– Precipitation flux (through reflectivity conversion)

– Precipitation accumulation (5’, 15’, …)

– Forecasted intensities (2PIR algorithm)

6

Soprano Imagery database : product overview

Satellite data• From MSG, METOP, GOES platforms …• Composite CMS products, …

7

Soprano Imagery database : products overview

Composite products– Radar precipitation intensity calibrated with ground measurements

(ANTILOPE)

– Satellite-Based fog diagnostics (CARIBOU)

8

WMS BDI features : Now

About 250 Layers GetMap returns only one LAYER at a time LAYER name is an internal identifier (tokens delimited by “_”, in

order to make easier mapping WMS – RDBMS data access Pre-defined STYLE : Each LAYER has a default colour map, which

is the official colour map on operational workstations at Meteo-France.

User defined styles are not yet supported (SLD=) If TIME not specified, GetMap returns the last image Outputs formats : image/gif; image/png Output CRS : only EPSG:4326 and Polar Stereographic at the

moment No support for metadata layers (e.g. pixel timestamp, pixel

quality, ...) but may be WCS scope ?

9

WMS Implementation : Reuse of legacy components

Data access component : “RDImage”– Application Binary Interface (executable)

– Performs data extraction from BDI on criteria :• Data type (radar, satellite …)• Production type (observation, forecast …)• Data origin (sensor platform, system, algorithm …)• Native domain covered by the dataset • Parameter (rain intensity, accumulation, radiance/wave length …)

Image processing component : “TransImage”– Application Binary Interface (executable)

– Performs :• Sampling a sub-domain• Format translation• Coordinate reference system operations (re-projection)

10

WMS Implementation

Meteo-France WMS framework– Implements WMS interface in PHP

– Check on GetMap – GetCapabilities consistency

– WMS 1.1.1 or 1.3.0

WMS BDI driver– Links GetMap parameters to RdImage & TransImage ones

– Depending on GetMap parameters, invoke TransImage if necessary

– Generate GetCapabilities response (XML)

11

WMS implementation overview

BDI (Oracle)

OCI

RDImage (.exe C/C++)

TransImage (.exe C/C++)

Capabilities (C )

Meteo-France WMS Framework (PHP)

GetMap GetCapabilities

BDI Driver (PHP)

WMS

12

Issue : GetCapabilities layering

GetCapabilities response – Mainly product metadata

– Describe product types (characteristics common to product instances)

GetCapabilities tree : Observations & Measurements approach– 3 Levels of layers :

• Process– Product type, Platform, Algorithm e.g Satellite MSG

• Feature Of Interest– Domain e.g Europe-Atlantic …

• Observed property– Parameter e.g IR 108

– Leaf layer is a named product type (queryable)

13

Issue : GetCapabilities layering

Layer granularity – Balance LAYER / Multi-dimensional LAYER (LAYER plus DIM)– E.g :

LAYER=SA__OBS__MSGC__V_EURATL__WV-62Or

LAYER= SA__OBS__MSGC__V_EURATL DIM_Wavelength=62 Or

LAYER=SA_OBS DIM_Platform=MSGC DIM_NativeDomain=V_EURATL DIM_Wavelength=62

Parameter names, units, … for DIMENSION usage– We need controlled vocabularies

E.g : DIM_Wavelength=62, 73, 108 , DIM_AccumulationTime=5, 15, 60 …DIM_BaseTime , DIM_Quality, DIM_PixelTimeStamp

The granularity of the layers will impact : – The efficiency of catalogue searching – The size of GetCapabilities response and performances

14

Issue : Keywords & Titles

Issue : Finding the layer name to perform GetMap request …– Layer names are quite complicated (internal identifiers), but unique :

• Ex : “RD_CPO_P__OBS__RD_ETRANGER__V_PIEMONT__REFLECTIVITE”

– About 250 layers in GetCapabilities– At the moment “Title” == “Name” (that is bad !)

• difficult to be interpreted by a human… Title should be explicit

– No abstracts, no keywords To help catalogue searching, we are looking for keywords, “Controlled

vocabularies”, “Ontology” …– E.g “METEOSAT” “MSG”, “METEOSAT-8” “METEOSAT-9”, …

Keywords should be hierarchic– E.g “TEMPERATURE -- CLOUD -- TOP”

Catalogue searching engine tested : GeoNetwork Possible keywords references :

– Global Change Master Directory (GCMD)– Common WMO code tables (Table C13, …)– AMS Glossary of Meteorology (GMET)– EUMETSAT , HMA (for satellite data) ?– Others ? To be defined ?

15

GeoNetwork evaluation as BDI catalogue

16

ISO 19128 – ISO 19115 relationship

ISO 19128 (WMS) Metadata element

property ISO 19115

Title CI_Citation.title

Abstract MD_DataIdentification.abstract

Keyword element in Keyword list MD_TopicCategoryCode

MD_Keywords

EX_GeographicBoundingBox EX_GeographicBoundingBox

BoundingBox EX_BoundingPolygon

DataURL MD_metadata.dataSetURI

Attribution CI_ResponsibleParty

Identifier and AuthorityURL CI_Citation.identifier

17

Issue : ISO 19128 – ISO 19115 relationship

How to keep consistency between WMS GetCapabilities (ISO 19128) and ISO19115 Metadata common fields ?

Where is the metadata reference or entry point ?– SOPRANO ISO 19115 metadata database

• To be created …• Partially harvested from VGISC metadata (probably not sufficient for BDI

because focussed on GTS products)

– BDImage (or other databases) descriptive schema content • Incomplete (keywords, abstracts are missing …)• Not always consistent (because of the database history…)• Difficult to change (because of backward compatibility)

18

Issue : Time

Time information– It would be useful to include product instance availability information

(ISO 19108) in GetCapabilities response :• Potential availability (time range plus step)• Real time availability (real time list : may be complex and CPU time –

consuming

– There is a TIME parameter in GetMap, but no dedicated tags for TIME information in GetCapabilities response WMS 1.1.1 and 1.3

– GetMap : no support for multi-dimensional time (forecasted imagery)e.g. : Forecasted precipitation intensity = f ( Base time, Validity Time )

– TIME semantics may be ambiguouse.g : Sampling Time, Validity Time, Result Time, Base Time …

Possible approach :– Using “Dimension” / “ExtendedCapabilities” ?

19

Issue : CRS identifiers

There are three methods to identify and describe CRS Namespaces & codes

– We need CRS identifiers not yet define in EPSG database and / or AUTO namespace, others …

• EPSG (not parametric)

– Lot of codes to be defined ( all CRS in use at Meteo-France ? )

• AUTO2 (parametric) :

– Universal Transverse Mercator AUTO2:42001

– Transverse Mercator AUTO2:42002

– Ortographic AUTO2:40003

– Equirectangular AUTO2:40004

– Mollweide AUTO2:40005

• Is it possible to define other codes ?

– Ex Polar Stereographic for each prim meridian, “Space View”, .. ?

Reference (URL) to ISO 19111 documents Proj4 “input parameters – like” ?

20

Issue : Performances

It takes about 1 seconds to process full disk MSGC image “on the fly”– Reprojection appears to be very expensive

– ”On the fly” processing probably not possible if many concurrent requests.

We probably need pre-defined CRS– This would allow pre-calculation of the projection matrix for each CRS

transformation (in TransImage)

– WMTS implementation ? But we’ll have to pre-compute tiles for each layer, CRS, Zoom Level, Format, Time …

– WMTS implementation is obviously a challenge for real-time Met data.

21

Left to do …

Lot of work on Metadata– Keywords, abstracts, ….– Link GetCapabilities response – ISO 19115 metadata

New output CRS New output formats (does GeoTiff make sense in a WMS ?) Support for multi dimensional times Support for metadata layers (pixel time stamp, pixel quality, … ?) Support for client style (SLD=) Improve performances

Limited set of CRS, projection matrix in TransImage component , WMTS Proj4 – GDAL migration of TransImage ( GEDAL is already in use on

SYNERGIE) TransImage as service Security topics to be investigated

22

Conclusion : BDI WMS works but …

To get really interoperable WMS, we need to agree on :

Metadata implementation rules (at least keywords) Colour Maps / Units of Measure

– E.g : Radar Data : Reflectivity (DBZ) – Intensity ( Kg m-2 / h)

CRS policies (EPSG, AUTO2, …) How to manage multiple time dimensions Layer granularity for each product type ( balance LAYER – layer

DIMENSION ) Parameter names, units for DIMENSION usage

Thank you for your attentionquestions ?