development of an ensemble prediction system on …nvd = var(eps1) – var(eps2) var(eps1) +...
TRANSCRIPT
![Page 1: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/1.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
COSMO-DE EPS construction, diagnosis and verification
of a limited-area ensemble prediction system on the convective scale
Marcus Paulat, Susanne Theis, Christoph Gebhardt,Michael Buchhold, Zied Ben Bouallègue
08 June 2009
4th Verification Workshop, Helsinki
Deutscher Wetterdienst
![Page 2: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/2.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Overview
construction and current status of COSMO-DE EPS
- perturbation strategy
results with focus on precipitation
- diagnosis of different EPS experiments
- verification of ensemble forecasts with “PACprove”
summary and future work
![Page 3: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/3.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Model configuration of deterministic COSMO-DE
grid-spacing: 2.8 km → convection permitting model
forecast lead time: 0-21 hours
model start: every 3 hours
operational set-up at DWD COSMO-EU
GME
COSMO-DE
![Page 4: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/4.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Model configuration of deterministic COSMO-DE
grid-spacing: 2.8 km → convection permitting model
forecast lead time: 0-21 hours
model start: every 3 hours
operational set-up at DWD COSMO-EU
GME
COSMO-DE
Ensembles based on high-resolution models are a new field of research!
Question: Which perturbations have really an effect on the forecast?
![Page 5: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/5.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
perturbation of initial conditions
COSMO-DE EPS: general perturbation strategy
perturbation of model physics
perturbation of boundary conditions
![Page 6: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/6.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
perturbation of initial conditions
perturbation of boundary conditions
combination of perturbations X X
COSMO-DE EPS: general perturbation strategy
perturbation of model physics
![Page 7: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/7.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
perturbation of initial conditions
perturbation of boundary conditions
combination of perturbations X X
COSMO-DE EPS: general perturbation strategy
perturbation of model physics
![Page 8: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/8.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Perturbation of the model
alter parameters in parameterization schemes- alteration should lead to a different forecast
- alteration should not deteriorate forecast quality (on average)
![Page 9: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/9.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Perturbation of the model
alter parameters in parameterization schemes- alteration should lead to a different forecast
- alteration should not deteriorate forecast quality (on average)
12 different fixed configurations of model physics
i.e. 12 slightly different model versions
“PHY” EPS1 2
312
. . .
![Page 10: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/10.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Perturbation of boundary conditions
nesting procedure
![Page 11: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/11.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Perturbation of boundary conditions
COSMO-DE EPS(DWD, Germany)
COSMO-DE (2.8 km)
nesting procedure
![Page 12: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/12.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
COSMO model (10 km)
Perturbation of boundary conditions
COSMO-DE EPS(DWD, Germany)
Part of COSMO-SREPS(ARPA-SIM, Bologna)
COSMO-DE (2.8 km)
nesting procedure
![Page 13: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/13.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
COSMO model (25 km)
COSMO model (10 km)
Perturbation of boundary conditions
COSMO-DE EPS(DWD, Germany)
Part of COSMO-SREPS(ARPA-SIM, Bologna)
Part of AEMet Ensemble(INM, Spain)
COSMO-DE (2.8 km)
nesting procedure
![Page 14: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/14.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
COSMO model (25 km)
COSMO model (10 km)
Perturbation of boundary conditions
COSMO-DE EPS(DWD, Germany)
Part of COSMO-SREPS(ARPA-SIM, Bologna)
Part of AEMet Ensemble(INM, Spain)
GME IFS UKMO NCEP
COSMO-DE (2.8 km)
nesting procedure
![Page 15: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/15.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
COSMO model (25 km)
Perturbation of boundary conditions
COSMO-DE EPS(DWD, Germany)
Part of COSMO-SREPS(ARPA-SIM, Bologna)
Part of AEMet Ensemble(INM, Spain)
GME IFS UKMO NCEP
COSMO-DE (2.8 km)
COSMO model (10 km)
“LBC” EPS
nesting procedure
physics variations in COSMO-SREPS leads to 12 members (in this study)
![Page 16: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/16.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Combination of physics and boundary perturbations
- all global models should be included
- 3 physics perturbations with highest impact on spread are chosen
1 2 3
GME
IFS
UM
NCEP
“COMB” EPS
![Page 17: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/17.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Results with focus on precipitation
time period: 15 days of August 2007
3 EPSs with 12 members: PHY, LBC, COMB
starting point: 00 UTC, lead time 0-24 hours
- Normalized Variance Difference
ensemble verification: „PACprove“- quality of single members
- probabilistic verification
ensemble diagnoses
![Page 18: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/18.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Ensemble diagnoses: Normalized Variance Difference
investigation of the impact of different EPSs on the spread.
NVD = Var(EPS1) – Var(EPS2)
Var(EPS1) + Var(EPS2)
-1 ≤ NVD ≤ +1
![Page 19: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/19.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Ensemble diagnoses: Normalized Variance Difference
investigation of the impact of different EPSs on the spread.
NVD = Var(EPS1) – Var(EPS2)
Var(EPS1) + Var(EPS2)
> 0 higher impact of EPS1
= 0 equal impact of both EPSs
< 0 higher impact of EPS2
NVD
-1 ≤ NVD ≤ +1
![Page 20: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/20.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
EPS1 = LBC EPS2 = PHY
Ensemble diagnoses: Normalized Variance Difference
![Page 21: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/21.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
EPS1 = LBC EPS2 = PHY EPS1 = COMB EPS2 = LBC
Ensemble diagnoses: Normalized Variance Difference
![Page 22: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/22.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Following verification results focussing on the COMB experiment
EPS1 = LBC EPS2 = PHY EPS1 = COMB EPS2 = LBC
Ensemble diagnoses: Normalized Variance Difference
![Page 23: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/23.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Ensemble verification with „PACprove“
PACprove = Package for probabilistic verification
Why we are developing our own Package and not „R“ for example?
• actually, the R-package is not officially supported at DWD
• main work has to be done on data handling (outside „R“)
• perfromance problems using „R“ for large data amounts
![Page 24: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/24.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Ensemble verification with „PACprove“
PACprove = Package for probabilistic verification
Why we are developing our own Package and not „R“ for example?
• deterministic: standard error scores like FBI, ETS, ...
• probabilistic: Brier score, rank histogram, reliability diagram, ROCcurve/area
Scores and measures:
• actually, the R-package is not officially supported at DWD
• main work has to be done on data handling (outside „R“)
• perfromance problems using „R“ for large data amounts
![Page 25: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/25.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Ensemble verification with „PACprove“
PACprove = Package for probabilistic verification
Why we are developing our own Package and not „R“ for example?
• deterministic: standard error scores like FBI, ETS, ...
• probabilistic: Brier score, rank histogram, reliability diagram, ROCcurve/area
Scores and measures:
Meteorological parameters:
• in general: all relevant parameters
• focus here: verification of precipitation with RADAR estimations
• actually, the R-package is not officially supported at DWD
• main work has to be done on data handling (outside „R“)
• perfromance problems using „R“ for large data amounts
![Page 26: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/26.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Ensemble verification with „PACprove“
PACprove = Package for probabilistic verification
Why we are developing our own Package and not „R“ for example?
• deterministic: standard error scores like FBI, ETS, ...
• probabilistic: Brier score, rank histogram, reliability diagram, ROCcurve/area
Scores and measures:
Meteorological parameters:
• in general: all relevant parameters
• focus here: verification of precipitation with RADAR estimations
• actually, the R-package is not officially supported at DWD
• main work has to be done on data handling (outside „R“)
• perfromance problems using „R“ for large data amounts
Keep in mind: development stage How far we get with our perturbations?
![Page 27: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/27.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Verification results: single member quality check
Is our perturbation strategy the right way?
![Page 28: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/28.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Verification results: single member quality check
- Frequency Bias Index
- 24 hours accumulated precipitation
- red line shows the deterministic COSMO-DE
Is our perturbation strategy the right way?
![Page 29: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/29.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Verification results: single member quality check
- Frequency Bias Index
- 24 hours accumulated precipitation
- red line shows the deterministic COSMO-DE
- all members are within a reasonable range
- we are on the right way!
Is our perturbation strategy the right way?
![Page 30: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/30.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Verification results: Brier Skill Score
24 hours accumulated precipitation
reference: deterministic COSMO-DE
![Page 31: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/31.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Verification results: Brier Skill Score
24 hours accumulated precipitation
reference: deterministic COSMO-DE
6 hours accumulated precipitation
![Page 32: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/32.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Verification results: rank histogram
Fre
qu
ency
24 hours accumulated precipitation
1 2 3 4 5 6 7 8 9 10 11 12 13
Rank
![Page 33: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/33.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Verification results: rank histogram
Fre
qu
ency
24 hours accumulated precipitation
1 2 3 4 5 6 7 8 9 10 11 12 13
Rank
ensemble is underdispersive
in development stage
![Page 34: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/34.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Summary
• perturbations in physics and boundaries
• perturbations of initial conditions is work in progress
construction of 3 different EPS experiments based on COSMO-DE
![Page 35: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/35.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Summary
• perturbations in physics and boundaries
• perturbations of initial conditions is work in progress
construction of 3 different EPS experiments based on COSMO-DE
• boundaries have the dominant effect (except the first 3 hours)
• combination of physics and boundary perturbations have the highest impact on EPS variance
diagnosis of the EPS variances (for precipitation)
![Page 36: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/36.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Summary
• perturbations in physics and boundaries
• perturbations of initial conditions is work in progress
construction of 3 different EPS experiments based on COSMO-DE
• boundaries have the dominant effect (except the first 3 hours)
• combination of physics and boundary perturbations have the highest impact on EPS variance
diagnosis of the EPS variances (for precipitation)
verification of the COMB ensemble (for precipitation) with PACprove• all members are within a reasonable range (no outliers)
• ensemble forecasts have some skill – our perturbation strategy is a useful way
• ensemble forecasts are underdispersive - the COSMO-DE ensemble is in development stage
![Page 37: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/37.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Summary
• perturbations in physics and boundaries
• perturbations of initial conditions is work in progress
construction of 3 different EPS experiments based on COSMO-DE
• boundaries have the dominant effect (except the first 3 hours)
• combination of physics and boundary perturbations have the highest impact on EPS variance
diagnosis of the EPS variances (for precipitation)
verification of the COMB ensemble (for precipitation) with PACprove
most of these results are presented in an article (submitted to Atmos. Res.)
• all members are within a reasonable range (no outliers)
• ensemble forecasts have some skill – our perturbation strategy is a useful way
• ensemble forecasts are underdispersive - the COSMO-DE ensemble is in development stage
![Page 38: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/38.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Future work
• include the initial conditions (increase of the spread) promising ideas
• increase duration of experiments more data
the COSMO-DE ensemble needs further developments
![Page 39: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/39.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Future work
• include the initial conditions (increase of the spread) promising ideas
• increase duration of experiments more data
the COSMO-DE ensemble needs further developments
• aim: improve the quality of ensemble products
• first step: logistic regression for calibration of probabilities
• final plan: Bayesian Model Averaging (BMA)
apply statistical postprocessing
![Page 40: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/40.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Future work
• include the initial conditions (increase of the spread) promising ideas
• increase duration of experiments more data
the COSMO-DE ensemble needs further developments
• aim: improve the quality of ensemble products
• first step: logistic regression for calibration of probabilities
• final plan: Bayesian Model Averaging (BMA)
apply statistical postprocessing
solve some technical problems
• running a high resolution ensemble needs large computer resources
• handling of large data amount
![Page 41: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/41.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Future work
• include the initial conditions (increase of the spread) promising ideas
• increase duration of experiments more data
the COSMO-DE ensemble needs further developments
• aim: improve the quality of ensemble products
• first step: logistic regression for calibration of probabilities
• final plan: Bayesian Model Averaging (BMA)
apply statistical postprocessing
• running a high resolution ensemble needs large computer resources
• handling of large data amount
solve some technical problems
COSMO-DE EPS is planned to be operational in 2011
![Page 42: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/42.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
![Page 43: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/43.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Extra Slides
![Page 44: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/44.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Verification results for 2m-temperature 12UTCVerification results for 2m-temperature 12UTC
ROC
area=0.94
threshold: 25°C
![Page 45: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/45.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
Perturbation of initial conditions
first experiments: perturb “nudgecast”- correlation length of observation increments
- geostrophic balance
- divergent flow correlations
current work:
use differences between control and COSMO-SREPS as IC perturbations
long-term plan:
Ensemble Transform Kalman Filter (COSMO project KENDA)
![Page 46: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/46.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
no perturbationof initial conditions
1 2 3
GME
IFS
UM
NCEP
Experiments – physics and boundary perturbations
![Page 47: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/47.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
mm/24h
24h accumulations of precipitation [mm]
2 July 2007, 00 UTC + 24h
![Page 48: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/48.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
no perturbationof initial conditions
1 2 3
GME
IFS
UM
NCEP
Experiments – physics and boundary perturbations
![Page 49: Development of an ensemble prediction system on …NVD = Var(EPS1) – Var(EPS2) Var(EPS1) + Var(EPS2)-1 ≤ NVD ≤ +1 08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue](https://reader033.vdocuments.site/reader033/viewer/2022043001/5f798c5a2f947123d229de43/html5/thumbnails/49.jpg)
08 June 2009 Marcus Paulat, Theis, Gebhardt, Buchhold, Ben Bouallègue 4th Verification Workshop, Helsinki
COSMO-DE EPS: construction, diagnosis and verification of a limited-area ensemble prediction system on the convective scale
mm/24h
24h accumulations of precipitation [mm]
2 July 2007, 00 UTC + 24h