issues in the verification of eps - umr-cnrm.fr fileeps Åke johansson smhi . spread-skill t2m from...
TRANSCRIPT
Issues in the
verification of
EPS
Åke Johansson
SMHI
Spread-Skill T2M from HIRLAM_K for JAN 2016
CU
The Commonly Used
Spread-Skill Relationship
MRMSEE
2
fo
MRMSEE E.Dev.StdE
2
fo 2
ff
CU SKILL – SPREAD RELATION
VAREEMSEE
.Dev.StdERMSEE
M
EM
CU SKILL – SPREAD RELATION
VAREEMSEE
.Dev.StdERMSEE
M
EM
CU Skill-Spread relation
is only valid for an
Ideal ensemble
4 reasons why
it is inappropriate for
Real World EPS
CU SKILL – SPREAD RELATION
VAREEMSEE M
Real World Models
usually have a
BIAS
VAREESDEE
VAREEMSEE
2
M
M
Real World Models
usually have members that
are not statistically equal
P
2
MP
2
M
VAREESDEE
VAREESDEE
Using U-Statistics for estimating
Skill and Spread
gives more realistic estimates
SKILL
SPREAD
SPREAD
K
1k
2
fofo
1K
1k
P ),t,t(f)k,t,t(f)1K(K
2SPRE
SPREAD
K
1k
2
fofo
1K
1k
P ),t,t(f)k,t,t(f)1K(K
2SPRE
U-statistic
A U-statistic is defined as the average –
across all combinatorial selections of the given size from the
full set of observations –
of the basic estimator applied to the sub-samples.
U-statistics, where the letter U stands for unbiased, arise
naturally in producing minimum-variance unbiased
estimators.
SPREAD
K
1k
2
fofo
1K
1k
P ),t,t(f)k,t,t(f)1K(K
2SPRE
U-statistic
A U-statistic is defined as the average –
across all combinatorial selections of the given size from
the full set of observations –
of the basic estimator applied to the sub-samples.
U-statistics, where the letter U stands for unbiased, arise
naturally in producing minimum-variance unbiased
estimators.
SPREAD
K
1k
2
fofo
1K
1k
P ),t,t(f)k,t,t(f)1K(K
2SPRE
2
fofoP )t,t(f)k,t,t(fVARE
PPPP VAREE2SPREEVARE
1K
K2SPRE
SKILL
2
PP
2
P MEMSESDE
SKILL
U-statistic
P
2
PSPREESDEE
U.UI SKILL – SPREAD CONDITION
P
2
PVAREESDEE
2
1
P
2
PSPREESDEE
U.UI SKILL – SPREAD CONDITION
P
2
PVAREESDEE
2
1
P
2
P
P
2
MP
VAREESDEE2
1
VAREESDEE
VAREESDEE
VAREEMSEE
2
M
M
CU U.UI
U
I
U
Example for T2M from HIRLAM_K for JAN 2016
Example for T2M from HIRLAM_K for JAN 2016
Why U-Statistics
Theoretical justification - Spread
aFE
22'FEaFE
2
2
ji
2
ji
2
ji
'FE2
'F'FE
'Fa'FaEFFE
Theoretical justification - Spread
aFE
22'FEaFE
2
2
ji
2
ji
2
ji
'FE2
'F'FE
'Fa'FaEFFE
Theoretical justification - Spread
22
ji
22
'FE2FFE
'FEaFE
Theoretical justification - Skill
aFE,0OE
222a'OEaOE
222
2
22
a'OE'FE
'O'FaE
'O)O(E'F)F(EEOFE
Theoretical justification - Skill
aFE,0OE
222a'OEaOE
222
2
22
a'OE'FE
'O'FaE
'O)O(E'F)F(EEOFE
Theoretical justification - Skill
222
222
a2
1'FE2aFE
a'FEaOE
Spread Skill
222
222
a2
1'FE2'FE2
a'FE'FE
Real World Models
are usually not verified against
the ”Truth”
(F, O)
Inflation = 1.0
Inflation = 1.1
Inflation = 1.16
Example for T2M from HIRLAM_K for JAN 2016
Conclusions
Over-dispersiveness – not under-dispersiveness
– seems to be the problem with EPS
Therefore one can wonder whether the following
really is necessary
• Parameter perturbations
• Stochastic physics
• Stochastic kinetic energy backscatter