ECOOP Data Management System (T2.2/WP2)

Declan Dunne

13th February 2008, Athens

Presentation Topics

Data Management System (DMS)… – work to date

DMS Architecture Framework– overview

Implementation plan– What is practical to implement

ECOOP outline plan

Data Management System (DMS)

DMS focuses on in situ data– Currently no best practice/standard system for in situ data

Needed to look at other initiatives:– INSPIRE– GMES Marine Core Service– MERSEA (FP6)– SeaDataNet (FP6)– SEPRISE (FP6)– InterRisk (FP6)

Data Management System (DMS)

Needed to look at two communities:– Oceanographic community (primary project goal)– GIS community (important for down-streaming)

Therefore, a cross fertilisation of ideas to seek best practice

Deliverables to date

D2.2.1.1 (Mo5): Report on existing regional centres documenting role, data types, and services.

D2.2.2.1 (Mo5): Review of metadata practices at regional centres.

D2.2.1.2 (Mo9): Delivery of a static web portal developed for ECOOP.

– http://ecoop.ucc.ie/static.php

D2.2.1.3 (Mo9): Report on the architectural plan for building the ECOOP DMS based on review of best practices (software patterns and design).

D2.2.2.2 (Mo12): Report on the architecture of the ECOOP metadata catalogue system.

Architecture Framework

INSPIRE service terminology

1. Data discovery services

2. Data download services

3. Data view services

Data discovery services

ECOOP S2.2.2

Data discovery services:tasks

Data discovery concerns metadata

Tasks– “product” metadata catalogue– “platform” metadata catalogue– reuse of existing common vocabularies

Data discovery services:best options

We will use SeaDataNet metadata standards…

– Product metadata CDI (Common Data Index)

– Platform metadata EDIOS (European Directory of the Ocean Observing Systems)

– Common vocabularies

INSPIRE metadata guidelines– Guidelines still draft (not finalised)

Data download services

ECOOP S2.2.1

Data download services:In situ data types

Point / Point Series

Profile

Trajectory

Data download services:tasks

Data download concerns access to data

Tasks– Selection and use of data format option(s)– Selection and use of data transport protocol(s)

– Development of quality assurance protocols S2.2.3 - Charlotte O'Kelly (TechWorks Marine, Ireland) Started January 2008 (links to T2.4)

– Quality Control flags (e.g. SeaDataNet QC flags)

Data download services:best data format options

Selection of data format option(s) for in situ data

No single standard, but best options include…

– ASCII ODV (Ocean Data View) spreadsheet - used in SeaDataNet

– NetCDF Coriolis netCDF

Data download services:best data transport options

Selection of data transport protocol(s)

– FTP Static files

– OPeNDAP More dynamic (can perform sub-setting)

– OGC services (WFS – Web Feature Service) Used by GIS community (can perform sub-setting)

View services

Data view services:best option

Any view services developed will use – OGC Web Map Service (WMS)

OGC Web Map Service recommended by INSPIRE policy

Example map: in situ stations Point/

Point Series– Point in map

Profiles– Point in map

Trajectories– Line in map

Technology available for WMS

Implementation plan

Step 1: FTP sites for in situ data

Progressing Regional Centres: in situ data

In situ data delivery: Step 1

FTP sites: Harmonisation– Ideally, a standard data format

(creates an environment for developing in situ tools & services)

– However, needs to be flexibility (no real right or wrong format)

ODV spreadsheet format (ASCII) Coriolis in situ netCDF format (binary)

Step 1: FTP sites for in situ data

SEPRISE is an option to help data gathering

Step 2: OPEnDAP for in situ data

In situ data delivery: Step 2

OPEnDAP

We will use “Dapper”

– Open source implementation of OPEnDAP for in situ data

– Has been implemented in Coriolis for in situ data

Dapper: view and download services

In situ data delivery: Step 2

Dapper vs THREDDS

– Dapper best used for in situ data products

– THREDDS best used gridded data products

– Dapper does not have a THREDDS type catalogue Therefore, will build a catalogue based on CDI standard

(SeaDataNet)

Step 2: OPeNDAP implementation

Dapper implementation plan

Need to further test and implement in higher detail

Will use SEPRISE system over coming months to test Dapper

Once successful… – Create a “how-to guide” for installing Dapper in regions

Overall implementation (steps 1 & 2)

interface to EuroMISS

Higher Steps

Big Picture

Research topics– e.g. Connecting to OGC WFS (Web Feature Service)

Technical Workshop 5 / 6 March 2008– Look at some of data harmonisation issues– Needs to be consensus/convergence path across various

European marine projects

Future Deliverables

D2.2.2.3 (Mo15): Delivery of the ECOOP metadata catalogue system, including web-based search interface.

D2.2.5.1 (Mo16): Report on the architectures of both the ECOOP DMS and EuroMISS.

D2.2.1.4 (Mo21): Implementation of the ECOOP Data management System - version 1.

D2.2.4.1 (Mo21): Review and design of the WP 2.1 system architecture (the climatological compiling system).

D2.2.4.2 (Mo21): Implement interface between climatological compiling system and the ECOOP DMS.

D2.2.5.2 (Mo21): Implementation of the Interface between the ECOOP DMS and EuroMISS.

D2.2.1.5 (Mo30): Delivery of the ECOOP Data Management System - version 2.

Thanks for your attention

13th February 2008, Athens

Declan Dunne