ASSESSING FORECAST UNCERTAINTY FROM

SYNOPTIC TO SUB-SEASONAL SCALES.

Celeste Saulo and Juan Ruiz

CIMA (CONICET/UBA) – DCAO (FCEN –UBA)

Motivation and general context Many meteorological services run operational

ensemble prediction systems (EPS), which provide estimates of the uncertainty of the forecast.

Many of these outputs are readily available to the scientific community through, e.g. TIGGE (THORPEX Interactive Grand Global Ensemble).

Obtaining useful (valuable) information from EPS requires statistical post-processing and specific research depending on the variable/problem/region.

There is growing interest in obtaining useful information from EPS on time scales between 2 weeks and 2 months.

Motivation and general context

Active research is being pursued in numerous places on the definition of initial ensembles, multimodel (or stochastic physics) as well as on the evaluation of ensemble predictions.

During the first half of THORPEX it was realized that model error diagnosis is one area where universities and research institutions can make substantial contributions to the further development of models (and hence forecast skill), thereby supporting the relatively small community of model developers.

THORPEX = The Observing System Research and Predictability Experiment

Potential areas of research under UMI-IFAECI

Predictability studies Ensemble generation (including data

assimilation) Probabilistic forecasts Verification strategies

Related ongoing studies How sensitive are probabilistic precipitation

forecasts to the choice of calibration algorithms and the ensemble generation method? Part I: Sensitivity to calibration methods (Ruiz

and Saulo, Meteorol. Appl., 2011)Part II: sensitivity to ensemble generation

method (Ruiz, Saulo and Kalnay, Meteorol. Appl. 2011)

Three different ensemble generation strategies, using WRF regional model as the basis: Breeding (11 members)Multi-model (11 members)Pragmatic= spatially shifted ((2*m + 1)2 members, e.g.,

121)

In order to correct the effect of the ensemble systematic errors, several techniques have been developed, all of them based in the study of the relationship between error and forecasted value and in the development of statistical models to compute a calibrated probability given the forecasts of the ensemble members

A logistic regression is used to represent h(y>0|f) and a GAMMA function is used to represent h(y>tr|f,y>0)BMA → weighted + calibrated probability for each memberGAMMA-ENS →all weights are equal + calibrated probability

for each memberGAMMA→ no weights + calibration applied to the ensemble

mean WMEAN →weighted ensemble mean and then calibration is

applied

Weights associated to each member of the spatially shifted ensemble as a function of the corresponding shift in the south–north (y axis) and the west–east (x axis) directions. Negative shift values indicate southward and westward shifts respectively.

GAMMA calibration has been adopted

Continuous rankedprobability score (CRPS)

The computation of a weighted ensemble mean can lead to moderate better results; however the best choice for a weight computation algorithm is still an open question. The PQPF derived from the un-weighted ensemble mean produces, if not the best results, almost as good results as any other approach.

MM

Breeding

Shifted

24 hours forecast

Combined

48 hours forecast

Shifted combined

Shifted-Breeding

Shifted-MM

shifted multimodel 1331 membersshifted breeding 1331 members shifted combined 2541 members

The spatially shifted ensemble proves to be quite competitive at short forecast ranges,

yet its skill drops rapidly with increasing lead times

Precipitation uncertainty at these ranges is mostly related with the location of rain areas

uncertainties associated with the existence, or intensity of pp, tend to become more important with increasing lead times.

multimodel ensemble (physics) outperformed the breeding ensemble (I&BC). Still, the improvement combining both is modest

most of the PQPF limitations during summer arise from errors in model physics rather than problems in the initial and/or boundary conditions

Among the alternatives that have been evaluated, the most important improvement has been obtained with the combination of the multimodel ensemble approach (and/or the combined approach) and the spatial shift technique even at 48-hours lead time. This approach is particularly interesting and promising for implementing high resolution ensembles in small operational or research centers for which computational costs largely restrict ensemble size.

Ensemble Forecast Object Oriented Verification Method Work in progress Juan Ruiz (postdoc at LMD) and

Olivier Talagrand The method has been designed to be applied to the

500 hPa field, however it can be easily extended to other fields as well (and probably other “objects” i.e. jet streak position, low level jet maximum possition, etc).

It is based in the identification of local minima and the system associated with each local minima.

As in 500 hPa, usually low pressure systems appear in the form of troughs rather than in the form of closed systems, the geopotential height anomaly is used instead of the full 500 hPa field.

Cyclone trajectories at 500 hPa, for a particular day derived from the NCEP ensemble system

Questions for future research How much information can be obtained from the ensemble spread

about the forecast skill? Are there specific scores to quantify this relationship in terms that it becomes useful for particular applications?

Which is the most convenient way to combine different ensemble members? Is it necessary to take into account the different skill of each member? (i.e. Bayesian model averaging trying different weights against simpler techniques like logistic regression for precip)

Which kind of information/type of scores could be used to provide valuable information about weather states with more than two weeks in advance?

How can we use model error statistics to understand which processes are strongly affecting forecast quality so that key problems can be isolated and models improved?

Which methodologies should we apply to forecast probability of extreme events?