WP9/JRA3: ESM Evaluation: developing an InfrastructureParticipants: AA, CNRS-IPSL, DLR, FMI, SMHI, MPG

OBJECTIVES

Create an interdisciplinary infrastructure that will facilitate the scientific evaluation of complex ESMs

•Standardize, harmonize and simplify the tools and methodologies used in existing or past model intercomparison projects to evaluate the basic quality of an ESM model.

•Create a public archive of easy-to-use toolkits that will include software to prepare required model diagnostics as well as links to pre-assembled most recent state-of-the-art observational datasets.

•Add to the IS-ENES portal an evaluation component that can be used as a ‘one-stop-shop’ for modellers who want to evaluate the performance of their ESM.

TASKSTask 1. Survey of methods and problems in model intercomparison projectsIdentify synergies among the different established model intercomparisons in the field(CFMIP, AEROCOM, PMIP, CCMVal, C4MIP…). Investigate the common problems involved in documenting model quality and model progress. Develop a joint implementation plan to improve the functioning of these intercomparisons. Organize a workshop to examine the implementation plan in collaboration with international groups such as PCMDI, NASA, and NCAR.

Task 2. Creation of support structure to facilitate model evaluation testsProvide support structures to ESM modelling teams to implement the appropriate model diagnostics and formatting tools (including CMOR, instrument simulators, variable meaning and unit standards), so that standard evaluation tests can be easily applied. Develop interfaces to bridge the format gaps between model and observational output.

Task 3. Provision of cataloguing and access services to observational datasets and evaluation toolsProvide catalogue and access services for the evaluation toolkits together with the corresponding observational datasets. Define standards for documentation of a toolkit. Implement these documentation standards for a set of previously used evaluation procedures in the above mentioned intercomparisons.

Task 4. Pilot study on the implementation of evaluation toolkits on ESM outputPerform a pilot study of the application of the evaluation toolkits to one ESM.

FIRST ACTIONS

T.1 Survey of methods and problems in model intercomparison projects

AA: Clouds, radiation and precipitation (e.g. CFMIP) CNRS-IPSL: Aerosols (e.g. AEROCOM), paleoclimate (e.g. PMIP), and carbon-climate couplings (e.g. C4MIP) SMHI: Ocean property and circulation projects (e.g. OMIP, AOMIP) MPG(MPIMET): Surface fluxes (e.g. SEAFLUX)DLR: Atmospheric chemistry (e.g. CCMVal)

Larger interdisciplinary model evaluation projects such as CMIP and CCMVal will be surveyed in collaboration with international groups like PCMDI and NCAR.

M1. Workshop on synergies and differences between model intercomparison projects.

D1. Report on differences, synergies and potential methodology transfers between the surveyed model intercomparison projects. Joint implementation plan to improve the functioning of ESM intercomparisons.