experience from the data collection exercise for update of the wfd priority substances 2011 d.preux...

Experience from the data collection exercise for

update of the WFD priority substances 2011

D.PREUX and B.FRIBOURG-BLANC, IOW

D. Preux, B. Fribourg-Blanc, Norman data exchange Workshop, 20-21/04/2011 slide 2

International Office for Water

Non profit making association (NGO with 150 members), State approved

Created in 1991 (20 years!)

Missions : international networking: INBO services in the water domain (Training centre, Information and data

Centre) European and international cooperation: twinning, CIS-SPI,

6th WWF Marseille 2012

http://www.oieau.org/


A Motto: “Capacity building for better water management!”

THREE DEPARTMENTS

Information management

Institutionalcooperation

Vocational training

International Office for Water


WFD priority substances: the project

• assistance to DGENV on WFD priority substances• One main activity: monitoring based prioritisation• Prioritisation methodology (INERIS) + data collection (IOW)

EU relevant, scientifically based and pragmatic From universe of substances to a manageable list (30-50 substances) Minimum data requirements

• Target: pollution of surface water compartments (Water, sediment and biota)

(Adapted from V.Bonnomet, INERIS)


WFD PS: the data collection steps

1.Define the needs: minimum mandatory and optional fields, voluntary basis

2. Create the End user tool (MS Access) for data collection

3. Data collection + helpdesk

4. Gather data, analyse, define the corrections and quality checks, test and implement

5. Import data in a central database

6. Treat data for prioritisation


Tools to support the collectionThe Microsoft Access End user tool (+template)

The How to use guide

A Web page www.oieau.fr/WISE-end-user-tool and a helpdesk


WFD PS: Data collection 2008 and 2009

• A strictly defined template with 22 mandatory fields

• Flexibility introduced on 8 fields (longitude/latitude, LoD/LoQ, type of station, date of sampling, biota species, laboratory name)

• A flexible data collection tool, and detailed documentation for non experts, to: minimise reporting burden: except on some validity

checks, any fraction, any unit, any substance even with missing mandatory fields is accepted

allow decentralised distribution of the tool (experienced by RO)

maximise quantity of data collected

• Interaction between data provider and consultant for Quality checking before selection of relevant dataset(s) correction of missing fields and coherence


WFD PS: Preliminary data treatment

• Only datasets for surface water River, Lake, Transitional, Coastal, Marine

• Elimination of : data measured before year 2000 non relevant parameters (e.g. P(tot), Nitrates, etc.) datasets for which neither LOD nor LOQ were provided Insufficient precision of matrix or fraction (all ”other”)

• Correction station location if missing substance name (in national language) or CAS code if detected wrong unit (in national language) fraction when it can be gathered with existing fractions date of sampling or date of analysis format or when one is missing

• Harmonisation of the measurement units (130 units on the 1130 substances provided and 102 units on 21 Tin compounds) Water: µg/l, Sediment: µg/kg dw, Biota: µg/kg ww


WFD PS: Central Database creation

• A central database similar to the End User tool (Station, Sampling, Analysis)

structured database (PostGre/PostGIS, SQL Language) on-line management tools for import of datasets (SQL based interface) Maintenance tools for storing and saving (SQL calculation modules)

• All individual files imported (more than 50 files, up to 40 GBytes)

export

XML file

End User Tool : one for each data

provider

Use for prioritisation

Internal checks and validation

import

Central database

Discarded (cat. 1 incomplete)


Source: GIS layer : Official WFD Districts

WFD PS: Summary situation

• Data 2000-2008• Surface water • 26 Member States + CH and NO• almost 20 000 stations

- 4 water body types • 547 000 sampling• 14,6 million analyses

• 1 150 substances or groups (PCB and Tin compounds)


WFD PS: Apportionment of analyses

River Water81%

Transitional Water4%

Coastal Water9%

Lake Water6%

0

500000

1000000

1500000

2000000

2500000

3000000

3500000

2000 2001 2002 2003 2004 2005 2006 2007 2008

by year

by water category


WFD PS: Apportionment of analyses

Biota0,5%

Sediment6,3%

Water93,2%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Wa

ter

Se

dim

en

t

Bio

ta

X>LoQ

X>Lod but no LoQ

X<=LoQ but no LoD

LoD<X<=LoQ

X=LoD

X<LoD

by matrix

along analytical result


WFD PS: The data quality aspects-Example: performance criteria set in Directive 2009/90/EC: LoQ < 0,3 x EQS

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%10

4-40

-510

7-06

-211

5-29

-711

7-81

-711

8-74

-112

002-

48-1

120-

12-7

122-

34-9

127-

18-4

140-

66-9

1582

-09-

815

972-

60-8

1912

-24-

920

6-44

-029

21-8

8-2

3253

4-81

-933

0-54

-134

123-

59-6

470-

90-6

50-2

9-3

56-2

3-5

608-

73-1

608-

93-5

67-6

6-3

71-4

3-2

7439

-92-

174

39-9

7-6

7440

-02-

074

40-4

3-9

75-0

9-2

79-0

1-6

8553

5-84

-887

-68-

387

-86-

591

-20-

3

LoQ <= 0,3xEQS

LoQ > 0,3xEQS

no LoQ available


WFD PS: Conclusion

• The first data collection of regular chemical monitoring data under WFD and a huge database

• A collaborative exercise involving EU and MS authorities

• A complete report on quality assessments (restricted, accessible to CIS

expert) concluded future data collection will require: Adaptation of the data collection template and tool additional collection rules (more mandatory fields, checks…) A checking mechanism on the data content (station location, …) more quality information (LoD and LoQ, accreditation…) Use of WISE and INSPIRE reference lists

• Possible development could include: Use of a shared EU wide structured water language (Reportnet) Use of shared reference lists (Reportnet Data Dictionary, ROD…) Use of shared scenarios

for example the French system: SANDRE and EDILABO


SANDRE, French data and metadata for waterA cornerstone of French water information system

Water Agencies

RBD authorities

Industries

Municipalities

Laboratories

Ministry of Environment

I do not understand the data sentI can not read the format

I need to develop a specific mappingfor each data collection

He used codes I do notknow

SANDRE

Data interchange in an evolving context: WFD implementation, progress of scientific knowledge… Numerous actors with different needs and tools


0..*

0..*Commemoratif

0..*

0..*Commemoratif

0..*

0..*Commemoratif

0..*

0..*Commemoratif

<<complexType>>

DEMANDE : 4

++++++++++

Code de la demande chez le commanditaireCode de la demande chez le prestataireType de demandeContexte de codificationDate de la demandeLibellé de la demandeDate de début d'application de la demandeDate de fin d'application de la demandeRéférence du marchéCommentaires sur la demande

: IdentifierType: TextType: CodeType: CodeType: DateType: TextType: DateType: DateType: TextType: TextType

CaractèreCaractèreCaractèreCaractèreDateCaractèreDateDateCaractèreTexte

<<complexType>>

PRELEVEMENT : 5

+++++++++++++++++

Code du prélèvementNuméro d'ordre du prélèvementRéférence du prélèvement chez le préleveurDate du prélèvementDélai de réalisation du prélèvementHeure du prélèvementDurée du prélèvementLocalisation exacte du prélèvementConformité du prélèvementAccréditation du prélèvementPrélèvement sous réserveCommentaires sur le prélèvementRisque du produit de prélèvementUsage du produit de prélèvementNature du produit de prélèvementNorme appliquée au produit de prélèvementFinalité du prélèvement

: IdentifierType: TextType: TextType: DateType: NumericType: TimeType: TextType: TextType: CodeType: CodeType: CodeType: TextType: TextType: CodeType: CodeType: CodeType: CodeType[]

CaractèreCaractèreCaractèreDateNumériqueHeureCaractèreCaractèreCaractèreCaractèreCaractèreTexteTexteCaractèreCaractèreCaractèreCaractère

<<complexType>>

ECHANTILLON : 3

++++++++

Référence de l'échantillon chez le commanditaireRéférence de l'échantillon chez le préleveurRéférence de l'échantillon chez le laboratoireComplétude de l'échantillonAcceptabilité de l'échantillonDate de réception de l'échantillonHeure de réception de l'échantillonCommentaires sur l 'échantillon

: IdentifierType: TextType: TextType: CodeType: CodeType: DateType: TimeType: TextType

CaractèreCaractèreCaractèreCaractèreCaractèreDateHeureTexte

<<complexType>>

ANALYSE : 2

+++++++++++++++++

Numéro d'ordre d'analyseDate de l'analyseHeure de l'analyseRésultat de l'analyseCode remarque de l'analyseLimite de détectionLimite de quantificationLimite de saturationAccréditation de l'analyseConfirmation du résultat d'analyseAnalyse sous réserveIncertitude analytiqueRésultat de référenceAnalyse in situ / en laboratoireRendement de l'extractionCommentaires sur l 'analyseVolume filtré

: NumericType: DateType: TimeType: TextType: CodeType: NumericType: NumericType: NumericType: CodeType: CodeType: CodeType: NumericType: TextType: CodeType: NumericType: TextType: NumericType

NumériqueDateHeureCaractèreCaractèreNumériqueNumériqueNumériqueCaractèreCaractèreCaractèreNumériqueCaractèreCaractèreNumériqueTexteNumérique

<<complexType>>

COMMEMORATIF

++++

Code du commémoratifLibellé du commémoratifDescriptif du commémoratifValeur du commémoratif

: IdentifierType: TextType: TextType: TextType[]

CaractèreCaractèreTexteTexte

SANDRE, a collaborative systemfor water professionals

Surface water

Waste water

Drinking water

Transitional and coastal water

Rainwater Groundwater

1 2 3

From natural water cycle to anthropogenic water cycle: a number of themes

From professional language to database interoperability

Experts needs Conceptual model Database


SANDRE, the approach

1st STEP: A COMMON SEMANTIC

3rd STEP: SCENARIOS

DATA DICTIONARIESFOR EACH THEME

2nd STEP: COMMON REFERENCES

CODE LISTS, REFERENCE MAP LAYERS

DATA EXCHANGE BY SCENARIOSAND FORMATS

XML-XSD

Scenariodocumentation

All the SANDRE results are freely availablewww.eaufrance.fr


Extract of data model


SANDRE: code list management

PARAMETER

UNIT

METHOD

MATRIX

FRACTION ANALYSED

µg/L

T 90-121

WATER

raw water 23

1107

133

132

3

The principle: evolving code lists freely available to water community to ease data interchange.

http://www.sandre.eaufrance.fr


EDILABO, a scenario for water analysis

Client – public administration

or private operatorLaboratory

Samplerin situ

information

Asks for analyses

Asks for

samplings in

situ

in

form

atio

nSampl

es

Analyses results

3 main actors, electronic exchange of sectoral data.


EDILABO: secured data exchange in water field

<Analysis><Parameter>1107</Parameter><Matrix>3</Matrix><Fraction>23</Fraction>

<Result>0.72</Result> <Unit>133</Unit> <Method>132</Method> </Analysis>

• Started 2001, based on EDIFACT standard and SANDRE,• Target water physico-chemical and microbiological analyses,

• Example:


Main conclusions

Based on WFD PS first experience: the first voluntary exercise proved feasibility of collecting big datasets Tool known: can be used for future

Quality of data expected to improve: QA/QC Directive entered into force Aug. 09 and transposed Aug. 11 (art 4 minimum performance, Art 5 calculation of mean, Art 6 ISO 17025) EQS Directive

a need to improve robustness and reliability of data collections More mandatory fields a need for more automatic data interchanges French SANDRE proves the feasibility of automating the process

A collaborative work: crucial for the success !!!


Thanks for your attention…

[email protected]

experience from the data collection exercise for update of the wfd priority substances 2011 d.preux...

Documents

data provider

quantity of data

water slide

data collection helpdesk

norman data exchange

data collection exercise

data collection steps

flexible data collection