![Page 1: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/1.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
RT5: EvaluationAlbert Klein Tank & Elisa Manzini
![Page 2: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/2.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
RT5: EvaluationAlbert Klein Tank & Elisa Manzini
Objective: to perform a comprehensive and independent evaluation of the performance of the ensemble simulation/prediction system against observations/analyses
![Page 3: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/3.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
Achievements
• With some delay, all RT5 deliverables and milestones are met
![Page 4: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/4.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
Achievements
• With some delay, all RT5 deliverables and milestones are met
![Page 5: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/5.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
Achievements
• With some delay, all RT5 deliverables and milestones are met
globalmultidecadal
globals2d
regionalmultidecadal
impactmodels
![Page 6: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/6.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
• Evaluation of global models used in multidecadal simulations (RT2a)– D5.15: impact of improved land-surface physics
onthe simulation of the Boreal summer
surfaceclimate (INGV)
Achievementsglobal
multidecadal
![Page 7: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/7.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
• Evaluation of forecast quality with global models in s2d simulations (RT2a)– D5.11: representation of the tropical intra seasonal
variability and its impact on seasonalpredictability (IPSL)
– D5.12: predictability in perfect-model mode over theNorth Atlantic/European sector (IfM)
Achievementsglobal
multidecadalglobals2d
![Page 8: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/8.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
• Evaluation of regional models used in multidecadal simulations, with focus on extremes (RT2b and RT3)– D5.18: ENSEMBLES daily gridded dataset for
Europe(UEA)
– D5.20: observational station data collected for theENSEMBLES daily gridded dataset for
Europe(KNMI)
Achievementsglobal
multidecadalglobals2d
regionalmultidecadal
![Page 9: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/9.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
• Evaluation of impact models (RT6)– D5.13: challenges of interpreting and
application of anensemble of imperfect models on
seasonaltime scales (LSE)
Achievementsglobal
multidecadalglobals2d
regionalmultidecadal
impactmodels
![Page 10: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/10.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
• Some examples:
Achievementsglobal
multidecadalglobals2d
regionalmultidecadal
impactmodels
![Page 11: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/11.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
GCM representation of vertical circulation regimes (ω500 dynamical proxy) for tropical eastern Pacific
-> double ITCZ syndrome (INGV)
![Page 12: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/12.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
Evaluation of DEMETER Asian summer monsoon intraseasonal variability ISV (Xavier, LMD)
![Page 13: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/13.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
KNMI Climate Explorer ENSEMBLES data server(climexp.knmi.nl by Van Oldenborgh, KNMI)
Select data in the Climate Explorer
Data is retrieved from ECMWFs2d server with OpenDAP
Available for analysis, e.g. skill scores
![Page 14: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/14.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
-1
ECMWF
UKMO
Multi-model (equal weights)
1-0.5 0.50
RPSSD
T2m JJA forecast, ini: May, DEMETER, 1960-2001 cross-validation using modified ranked probability skill score
RPSS (Weigel et al., MeteoSwiss)
![Page 15: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/15.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
-1
ECMWF
UKMO
Multi-model (weighted + climatology) (more orange + no blue)
1-0.5 0.50
RPSSD
T2m JJA forecast, ini: May, DEMETER, 1960-2001 cross-validation using modified ranked probability skill score
RPSS (Weigel et al., MeteoSwiss)
![Page 16: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/16.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
ECA dataset for gridding (KNMI and MeteoSwiss)
![Page 17: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/17.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
ECA dataset for gridding (KNMI and MeteoSwiss)
![Page 18: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/18.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
ECA dataset for gridding (KNMI and MeteoSwiss)
![Page 19: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/19.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
Best gridding method for daily rainfall(Hofstra et al., Oxford Univ.
and Haylock et al., Univ. of East Anglia)
![Page 20: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/20.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
Comparison with existing daily datasets(Hofstra et al., Oxford University)
Compound relative error score CRE
Daily precipitation
![Page 21: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/21.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
Comparison with ERA40(Hofstra et al., Oxford University)
Daily Tmax
RMSE
![Page 22: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/22.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
ECMWF seasonal forecast storms, %total insured value SwissRe portfolio (Della-Marta & Liniger, MeteoSwiss)
![Page 23: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/23.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
Verification of wheat yield at the Bologna University experimental farm forecasted with DEMETER
downscaled input (ARPA)
![Page 24: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/24.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
Plans
• Another 16 deliverables until the end of the project, including all-GCM evaluation efforts
• Start of more regional work now that the RT5 observational data set and the RCM data are available
• Note that good access to all data remains essential for RT5!
![Page 25: ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007 RT5: Evaluation Albert Klein Tank & Elisa Manzini](https://reader036.vdocuments.site/reader036/viewer/2022062720/56649f145503460f94c28576/html5/thumbnails/25.jpg)
ENSEMBLES General AssemblyPrague, Czech Republic, 14 November 2007
RT5 review
• Taken up the recommendation to further integrate RT5 work within ENSEMBLES:– Monday workshop on RCM and SDS evaluation
(together with RT2b and RT3)– Thursday workshop on model error (with RT4)– Thursday workshop on weighting, credibility and
reliability (with RT3; Geert Lenderink)