gmes marine service my ocean cross cutting cal/val activities & product quality working group...
TRANSCRIPT
GMESMarineService
MY OCEAN
Cross Cutting Cal/Val Activities & Product Quality
Working GroupScientific Assessment of
External Products in MyOcean
Second annual meeting
F. Hernandez, chair PQCV-WG, QuARG
Marine Core Service
MyOcean : Second Annual Meeting / 27-28 April 2011 / Roma
Cal/Val activities for the second MyO year
Cal/Val teams in Production Centres (X.5) strongly involved in:
– Test and implementation of Cal/Val tools for V1
– Design Calibration and Validation strategies and plans (ScCP, ScVP)
– Perform scientific calibration, routine scientific validation, and report (ScCR, ScVR, ScVR-Rea) for reviews
Prepare ScCP, ScVP…
….. V1S1 devlpt.
PC Acceptancereview of ScCP, ScVP,
ScCR
June 2010 End September 2010
VAlid. Read. Rev.Start V1 S1 operation
15/12/2010
Test op. validation
V1S1 scientific assessment ….. V1S1 pre-operated ….. V1S1 in operation
Operate routinely PQ validation: reg. reporting
29/4/2011
P. Read. Rev.Review V1 S1 ScVR
Calibration
….. V1S2 devlpt. / REA V1S2 & REA scientific assessment
PC Acceptancereview of ScCP, ScVP, ScCR, ScVR-Rea
March 2011 June 2011 Sept. 2011
VARRStart V1S2 operation
PRRV1S2 ScVR
….. V1S2 in operation
Validation
Marine Core Service
MyOcean : Second Annual Meeting / 27-28 April 2011 / Roma
Cal/Val activities for the second MyO year
Product Quality addressed at the project level:
– Reviews, where ScCR, ScVR are examined
– Link with users: feedbacks from User Forum (Stockholm, April 2011)
– Designing MyOcean-2 Product Quality activities: focusing on cross-cutting integrated actions and methods, regular monitoring, link between Production Centres and Users
Marine Core Service
MyOcean : Second Annual Meeting / 27-28 April 2011 / Roma
Organising Product Quality for V1:plans and reporting for reviews
PC Acceptance Stream1 V1, Roma, September 2010
VARR Stream1 V1, Brussels, December 2010
PC Acceptance Stream2 V1, Copenhagen, March 2011
User Forum, Stockholm, April 2011
PRR Stream1 V1, Roma, April 2011
VARR Stream2 V1, June 2011
PRR Stream2 V1, September 2011
Prod. Centre©
ScCP/ScVP ScCR ScVR-Rea ScVR (PRR)
GLO x Stream 1 5 docs 1 doc + 2 docs
ARC x Stream 1 1 doc 1 doc
BAL x Stream 1
NWS x Stream 2 2 docs
IBI x Stream 2
MED x Stream 1 1 doc
BLS x Stream 2 1 doc
SL x Stream 1 1 doc
OC x Stream 1/2 1 doc 1 doc
SST no Stream 1 in 2 OSTIA doc 1 doc
SIW no for Wind Stream 1 7 docs no Wind
INS x NO (in prep?) 1 doc
Marine Core Service
MyOcean : Second Annual Meeting / 27-28 April 2011 / Roma
Organising Product Quality for V1:plans and reporting for reviews
Overall good level of information in ScCR and ScVR
Permanent improvements of reporting, from stream1 to stream2:– Upgrading and extending Cal/Val methodologies
– Analysing benefit of V1 compared to V0
– Analysing added values of regional products
– Starting to address « estimated accuracy » values compared to user requirements (if exists)
From V1 to V2, plans to upgrade cal/val strategies and associated documentation with more consistency among TACs and MFCs
Plans to use existing scientific cal/val to provide usefull information for users
Marine Core Service
MyOcean : Second Annual Meeting / 27-28 April 2011 / Roma
Flavour of Product Quality assessment, from reporting
Different difficulties, management, history, heritage, and many positive aspects:
– SST TAC reporting: not all docs, but validation strategies already described in existing operational activities. ScVR, really complete, and usefull for users (estimated confidence level per products)
– INS TAC slightly different from other PC: provide Real Time Quality Control, instead of Cal/Val phasing, and in a continous framework
– BAL MFC: a complete reporting, plus a long annex (+550p)
– BLS MFC: enormous lack of observations, but alternative use of climatology
– ARC MFC: A strong dedication to Sea Ice scientific assessment.
– …..
Marine Core Service
MyOcean : Second Annual Meeting / 27-28 April 2011 / Roma
Flavour of Product Quality assessment, from reporting:V1 vs V0
From IBI ScCR
Marine Core Service
MyOcean : Second Annual Meeting / 27-28 April 2011 / Roma
Flavour of Product Quality assessment, from reporting:V1 vs V0
From NWS ScCR
Marine Core Service
MyOcean : Second Annual Meeting / 27-28 April 2011 / Roma
Flavour of Product Quality assessment, from reporting: proposing new approach: OC TAC and MED MFC validating Chl-a
From MED ScVR
Marine Core Service
MyOcean : Second Annual Meeting / 27-28 April 2011 / Roma
Flavour of Product Quality assessment, from reporting:organize and standardize cal/val (metrics) definitions
ScCP, ScVP provide description of metrics Next phase: harmonisation
Marine Core Service
MyOcean : Second Annual Meeting / 27-28 April 2011 / Roma
Flavour of Product Quality assessment, from reporting: organize product quality synthesis
From ARC ScVR, ScVR-REA
Marine Core Service
MyOcean : Second Annual Meeting / 27-28 April 2011 / Roma
Tranverse Cal/Val at MYO level
Organize more homogeneous Cal/Val procedures btw PC/PU:
– X.5 experts asked to review ScCR: « share good ideas »
– Neighboring dynamics:
• WP6 / WP7 working already together add WP8
• WP8 / WP9 and WP4 / WP9 in contact
• WP10 / WP9 starting
• WP4 / WP8 « natural contacts »
• Existing inside each TACs
– Indirect benefit from Lisbon 1st PQCV-WG meeting: lot’s of similarities in ScCR and ScVR metrics
Second PQCV-WG meeting yesterday to strengthen transverse Cal/Val
Marine Core Service
MyOcean : Second Annual Meeting / 27-28 April 2011 / Roma
PQCV-WG 2nd meeting outcomes
Improvements of ScCP, ScVP, technical definitions of metrics
– Upgrade definitions before V2, start with moorings
Marine Core Service
MyOcean : Second Annual Meeting / 27-28 April 2011 / Roma
Upgrade definition: adopt common names of moorings…..
Marine Core Service
MyOcean : Second Annual Meeting / 27-28 April 2011 / Roma
PQCV-WG 2nd meeting outcomes
Improvements of ScCP, ScVP, technical definitions of metrics
– Upgrade definitions before V2, start with moorings
Product Quality expected accuracy level (static numbers to be associated to products):
– Starting from list of V1 products (eg EPST excel sheet)
– End of June 2011: every PC provide numbers to PQCV-WG
– End of August 2011: After PQCV-WG consistency check, numbers provided to WP17
Marine Core Service
MyOcean : Second Annual Meeting / 27-28 April 2011 / Roma
Last year: PQCV-WG proposition for link with Service Desk
information V1 V2
Static:
“precision” level given as meta-data
Dynamic
“flags”
Error bars
Extreme values
? YES
<e>
no YES
ENSTaf BQMMPP
Marine Core Service
MyOcean : Second Annual Meeting / 27-28 April 2011 / Roma
PQCV-WG 2nd meeting outcomes
Improvements of ScCP, ScVP, technical definitions of metrics
– Upgrade definitions before V2, start with moorings
Product Quality expected accuracy level (static numbers to be associated to products):
– Starting from list of V1 products (eg EPST excel sheet)
– End of June 2011: every PC provide numbers to PQCV-WG
– End of August 2011: After PQCV-WG consistency check, numbers provided to WP17
QUID: Quality Information Document for users:
– Baseline: provide ScCR, ScVR
– Instead provide dedicated documents from ScCR, ScVR
– Template discussed at PQCV-WG meeting
– Late September 2011 (phased with end of Stream2 Validation Period) QUID draft delivered by each PC
– Mid-October 2011: after a 2 week PQCV review, QUID provided to PMO, WP17
Marine Core Service
MyOcean : Second Annual Meeting / 27-28 April 2011 / Roma
PQCV-WG 2nd meeting outcomes
Improvements of ScCP, ScVP, technical definitions of metrics
– Upgrade definitions before V2, start with moorings
Product Quality expected accuracy level (static numbers to be associated to products):
– Starting from list of V1 products (eg EPST excel sheet)
– End of June 2011: every PC provide numbers to PQCV-WG
– End of August 2011: After PQCV-WG consistency check, numbers provided to WP17
QUID: Quality Information Document for users:
– Baseline: provide ScCR, ScVR
– Instead provide dedicated documents from ScCR, ScVR
– Template discussed at PQCV-WG meeting
– Late September 2011 (phased with end of Stream2 Validation Period) QUID draft delivered by each PC
– Mid-October 2011: after a 2 week PQCV review, QUID provided to PMO, WP17
Transverse Cal/Val activities: internal website:
– Start with selected metrics for SL, SST, Sea-Ice, Chl-a, Water masses
– First integration aspects (tbd) with WP2 in September 2011
Marine Core Service
MyOcean : Second Annual Meeting / 27-28 April 2011 / Roma
Marine Core Service
MyOcean : Second Annual Meeting / 27-28 April 2011 / Roma
Cal/Val activities in MyOcean, summary
All PC/PU involved
Project rules (document, reviews) adopted, with obvious benefits
Non expected yet, but User request for quality (strongly reminded during the User Forum) can start to fullfiled due to the Cal/Val work achieved
The Cal/Val transverse strategy is accepted, even if additional work is requested
This strategy and difficulties encountered have been a direct input for MYO-2 Cal/Val definition and organisation