marine data & information management – lessons learnt from...

1

SCOR/IGBP Meeting onData Management

for Marine Research Projects

Foresight Centre, Liverpool, UK8-10 December 2003

Marine Data & Information Management– Lessons learnt from JGOFS

Bernard Avril(JGOFS International Project Office)

Presentation Outline

PreambuleIntentions and Disclaimer

Science & Data from Joint Global Ocean Flux Study (1)-(3)What is JGOFS?

JGOFS Data Management System – Initial Plans (1)-(2)What were the initial objective and proposed set-up?

JGOFS Data Management – Final Report, 2003 (1)-(4)What were the actual working set-up and achievements?

JGOFS International Data Collection (1)-(2)Contents and benefits for the users

JGOFS Data Management – Lessons LearntRecommendations of improvements for the benefits of all parties

Data & Information Management – some ideas (1)-(3)Objective and subjective evaluations

Idealised D&IM framework in marine projects (1)-(2)An idealised vision or a feasible set-up for the future?

Conclusion

2

Preambule

Intentions and Disclaimer

This presentation is ...... written with the overall idea that the highest quality inD&IM should help and support the PIs, project sponsors, funding agencies and end-users, to achieve and/or benefit from the best science, today and tomorrow.

This presentation is not ...... always (or ever) the “official” viewpoint from the JGOFS community, if such thing can be defined...

... Yet it is a viewpoint, and it could be sometimes useful, and discussed...

... So, please, do not hesitate to interrupt!

• global carbon survey• regional process studies• long-term time-series• data management• synthesis & modelling

Scientific Goal• oceanic biogeochemical processes of carbon & associated

biogenic elements, and response to climate change.

Science & Data of Joint Global Ocean Flux Study (1)

→ 20+ countries→ 15-year project→ SCOR + IGBP

Strategies

3

12.5

25

37.5

0

Annual primary production (mol-C m-2 a-1) based onSeaWiFS data and model. Falkowski et al., IMCS, Rutgers University, USA.

Global map of the average annual exchange

CO2 flux (mol-C m-2 a-1) across the sea surface. Takahashi et al.,. 1999

Global map of seafloor respiration based on benthic O2 fluxes (mol-O m-2 a-1). Jahnke R., 1996. GBC © AGU

Particulate organic carbon export(mol-C m-2 a-1). Schlitzer, AWI, 2001.

Global composite image of surface Chlorophyll a concentration (mg m-3) estimated from SeaWiFS data (NASA / GFSC + ORBIMAGE)

± 1 GtC/yr

D. Karl


(evolving) extensive & intensive, marine biogeochemistry studies,

design of field projects, hypothesis testing and diagnostic & prognostic models

data management (& information) system

Unprecedented, high spatial & temporal resolutions of complex, multidisciplinary ocean data (20+ physical, chemical, biological and sedimentological core parameters)

D. Karl


4

JGOFS Data Management System – Initial Plans (1)

(rewritten after JGOFS Science Plan, 1990 and JGOFS Implementation Plan, 1992)

JGOFS Data Management System:

“To provide all interested scientists with complete and convenient access to the international JGOFS dataset.”

JGOFS policy regarding data submission, access and exchange:

- encourage free & open communication of findings, including raw data, - access without restriction for any interested user, provided that data originator is contacted for permission of further data use,- national (mandatory) submission by project investigators of their data to the project data system in a timely manner (cf. core parameter protocols),- each national project committee should endorse this policy.

JGOFS Data Management System – Initial Plans (2)

(rewritten after JGOFS Science Plan, 1990 and JGOFS Implementation Plan, 1992)

National Project Data Centres (with a data coordinator):

- flexible, online procedures of submission, exchange, retrieval & work- online national project database, including data inventory & metadata- (extra-)national exchange modes and formats for data project centres,- arrange training in using the procedures,- interact with the project DMTT as needed.

Data Management Group (now DMTT) continuously:- co-ordinate the national activities, as needed,- monitor data management compliance, and- reassess periodically needs & performance.

Core Project Office (now IPO) also for Data Management:Goal: Help the investigators to locate and access project data.- project planning & execution, data management, synthesis & modeling,- project resource center on national and regional activities

5

JGOFS Data Management – Final Report, 2003 (1)

JGOFS Data Management Task Team (cf. ToR, 1997):

• initiate, encourage and develop national data management,• develop, implement timely, national data exchange

mechanisms, and international data integration • compile & dissemintate an integrated, international data

and cruise inventory,• monitor international acceptance of, compliance with,

policy (& adapt it),• liaise with other (inter-)national initiatives & data centres,• provide IPO with advice regarding data management, and• report to SSC and implement its recommendations.

Data Management Task Team (DMTT, 1993-2003)

JGOFS Data Management – Final Report, 2003 (2)Australia

Germany

India

Japan

United States

France

United Kingdom

Canada

from within their NODC (CA, IN, JP, UK), or with a funded position in an oceanographic institution (AU, FR, GE, US).

In other countries, data mostly with individual PIs (e.g., IT, PK, SP) and rarely managed nationally (NL, NO).

6

Data Management Task Team (DMTT)

DMTT Modellers

Task Teams &Synthesis Groups

SSCIPO

PANGAEA

IOC IODEMDM

NODCs

CEOS/GCMD

DIFs


National Contact Points

(NCP)North AtlanticIndian Ocean

Southern OceanNorth Pacific

Equatorial Pacific

Regional Synthesis andModelling Groups

Remote SensingGlobal Synthesis and ModellingPhotosynthesis MeasurementsPaleo-JGOFS (w/ PAGES)Continental Margin (w/ LOICZ)Task Teams

SICAP-CO2 IOCCG

Global Synthesis and ModellingOcean Carbon Modelling (w/ GAIM)Global Synthesis Activities


Data Management Task Team summarized achievements:

- high profile for biogeochemical data management in community and outside,

- compile and publish national cruise and data inventories, with added value,

- produce a unique, multinational, biogeochemical data product (DVD vol. 1),

- contributed to WDC-A, NASA’s GCMD, WDC-MARE, and others.

- & multiple CDs, websites, publications (books, special issues, special reports, meeting reports, notes)

7

first large scale, international, discrete data collection in global oceanic biogeochemistry, from ~1000 cruises~80-96% JGOFS-funded science data available (for DMTT-represented countries),~85% datasets/cruises from DMTT-represented

countries,according to JGOFS cruise inventories, ~80% of all JGOFS data are on DVD vol. 1,rough estimate: ~5% [2-8%] dedicated to data management... + ~5-yr half-time IPO supportfair representation of the national JGOFS data managersʹ efforts,

JGOFS International Data Collection (1)

JGOFS International Data Collection.Volume 1: Discrete Datasets DVD

JGOFS International Data Collection. Vol. 2: Integrated

Datasets DVD with WDC-MARE, funded by AWI and

BMBF (Germany).

JGOFS International Data Collection (2)

About 40’000 JGOFS data entries already in PANGAEA (www.pangaea.de/)

8

JGOFS Data Management – Lessons Learnt

(rewritten after JGOFS International Data Collection. Vol. 1 – readme fileand presentation during the 3rd IGBP Congress)

Lessons learned from JGOFS: Recommendations for future Data and Information Management (to ensure the rapid dissemination of data and its long-term preservation and accessibility)

Establish a coherent, credible, semi-distributed and scalable, end-to-end D&IM plan with:clear D&IM objectives, activities and timeline,policy (e.g., delivery and exchange standards), widely agreed upon by the scientists and sponsors, and supported by funding agencies to insure compliance,clearly identified core project science and associated core parameters, to be revised regularly,an International Project Data Centre designed to establish guidelines, provide advice, and facilitate exchange of knowledge and expertise,experienced, full-time national data managers / coordinators,increase overall value of scientific research and derived outputs...

Issues / Questions to be addressed about Data & Information Management, through few keywords

describing the overall D&IM system

Data & Information Management – some ideas (1)

Flexibility

Credibility

Adaptability

Support

Focus

Productivity

Manageability

Integration

9

Data & Information Management – some ideas (2)

Focus: definition of the project scope (business plan ?)(e.g., identified project (inter)-national activities, core parameters, expected data volume and data management & product requirements)

Credibility: Coherent & credible data management framework(“An attractive carrot is worth ten big sticks”)

Support: full and adequate support of the PIs, project managers, project sponsors and other end-users..., and reciprocally!

Flexibility: levels of granularity, of stability and of efficiency(e.g., DM organised by PIs, projects, nations, parameters, communities?)

Issues / Questions to be addressed about Data & Information Management, through few keywords describing the overall D&IM system

Data & Information Management – some ideas (3)Issues / Questions to be addressed about Data & Information

Management, through few keywords describing the overall D&IM system

Adaptability: roles of already existing N(O)DCs?(e.g., long-term archival / on-demand, multi-disciplinary, multi-strategic

project support)

Integration: added-value for all integrated activities and data product, for all parties

(e.g., no duplication of works, standards, easier synthesis and modelling)

Manageability: Objective tools to review, evaluate and improve the D&IM efficiency and achievements

(e.g., data delivery, data usability, capacity building)

Productivity: create highly visible, international deliverables to maintain the D&IM dynamics

10

Schematic diagram representing the interactions between the project partners andcollaborative bodies, and the main fluxes of data & information.

Idealised D&IM framework in marine projects (1)After Avril & Dittert (2003)

Each element represents a

function rather than an office

• a range of services and products, taking full advantage of best practices, standards or innovative approaches,

• pro-active, ”bottom-up” strategies addressing all needs and requirements,• establish & support experienced, full-time national or regional data

managers, through existing infrastructures, capacity building, new international synergies,

• fair ”top-down” strategies to insure compliance, better integration, • policy fully agreed upon and widely disseminated at the (inter)-national

levels, for delivery, quality-control, referencing, exchange, integration, dissemination, preservation..., to facilitate interoperability,

• project knowledge resource centre (DIMO), a Data Management Committee (DMC, with data managers, observationalists and modellers) and a set of Data Evaluation Units (DEUs), where needed.

Idealised D&IM framework in marine projects (2)(rewritten after Avril & Dittert (2003) and other texts)

+ There is a large number of marine data policies already available. A rough compilation of unofficial comments from data managers is also available.

11

Conclusion(rewritten after H. Ducklow, Final JGOFS OSC, May 2003)

JGOFS was created as a visionary programme, especially through its union of biology, chemistry and physics, (and sedimentology) and its collaborative science works including more than 20 nations, in a global, marine biogeochemistry (V. Vernadsky, “The Biosphere”, 1926) project. This is a major achievement of JGOFS.

And behind all those science works and approaches, there is also a revolution in the way data and information have been managed! ... and it is not finished!

JGOFS experienced some drastic changes in the way oceanography is done and managed. Now, “things” are changing even faster: increased overall commitment and focus for data management, information management and even knowledge management; and the international collaboration and standards are better promoted and accepted... Yet sometimes, it works pretty differently than anticipated on this side of the world... Within the early plans, “great” ideas and principles were presented but they were not always realistic, and were not applicable on into solutions pratical enough, or sometimes with a commitment inadequate for the task.

marine data & information management – lessons learnt from...

Documents