assessment of flood predictive
DESCRIPTION
Assessment of flood predictive uncertainty in using meteorological ensembles to hydrological forecastingTRANSCRIPT
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Assessment of flood predictive uncertainty in using meteorological ensembles to hydrological
forecasting
Prof. Ezio Todini
ISSAOS 2005, ISSAOS 2005, L’AquilaL’Aquila, August 29 , August 29 –– September 2, 2005 September 2, 2005
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Flood emergency managers require simple decision rules to be applied in real time.
Unfortunately these rules are strongly affected by the uncertainty on the future evolution of events.
This implies the necessity of using forecasting tools with the aim of reducing their uncertainty on future events.
The basic problem
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
A Flood Forecasting System is a tool aimed at reducing uncertainty on
the future evolution of a flood event..!!!
Do we all agree on what isa Flood Forecasting System?
It is the reduction of uncertainty thatallows for more reliable decisions.
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Expected value= Forecast
Probability of overflow
Leve
l
Costs
Cross section
An Example:The Flood Warning Problem
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
When using such forecasting tools, as pointed out by Krzysztofowicz (1999), it is essential to asses the uncertainty of the future state of the quantity of interest (level, discharge, etc.) conditional to its forecasted value in order to improve decisions.
This approach is a much simpler alternative to theoverwhelming computational effort required by a fullunconditional uncertainty assessment, which would imply marginalisation of the forecasting density with respect to all possible forecasting models, model parameters, initial states, input measurement and forecasting errors.
The big question: are we actually describing it?
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
( ) ( ) XdddIdMdXXMIXyfyf ˆ,,,,ˆ, ϑϑ∫∫ ∫ ∫ ∫=
( ) ( ) ( ) ( ) ( ) ( ) XdddIdMdXXfXfMfXMIfXMIfXMIXyf ˆˆ,,,,,,,ˆ ϑϑϑ∫∫ ∫ ∫ ∫=
The full unconditional uncertainty assessment
the quantity of interest, the predictand
the input data
the forecasted inputs
the model
the model parameters
the initial status====
==
I
MX
Xy
ϑ
ˆ
with
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Different levels of conditional uncertainty 1/5
( )XMIXyf ,,,,ˆ ϑ
Although incorrect, it is quite common to estimate the model parameter values and to use theconditional density to represent
our predictive uncertainty.
The full conditional density
expresses our uncertainty conditional to the forecastprovided by a given model with given parameter values,given initial status, given inputs and given input forecasts.
ϑ( )XMIXyf ,,,ˆ,ˆ ϑ
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Different levels of conditional uncertainty 2/5
( ) ( ) ( )∫= ϑϑϑ dXMIYfXMIXyfXMIYXyf ,,,",,,,ˆ,,,,ˆ
A more appropriate way is to derive a “posteriordensity” of the parameters using Bayesian Inference
using a set of past observations and tomarginalise with respect to the parameters, to obtain:
( )XMIYf ,,," ϑ Y
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Different levels of conditional uncertainty 3/5
If a density representing the uncertainty of the forecasted inputs is available,then one can obtain:
( ) ( ) ( )[ ] ( ) ϑϑϑ dXMIYfXdXfXMIXyfXMIYyf ,,,"ˆˆ,,,,ˆ,,, ∫ ∫=
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
NOTE that, the previous slides show that no action is generally taken to eliminate the conditioning
- on the assumed model structure,
- on the initial value of the state variables,
- on the input measurement errors.
Different levels of conditional uncertainty 4/5
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Different levels of conditional uncertainty 5/5As a matter of fact the model is generally taken as ourprior knowledge, concentrating all the uncertainty in itsparameter values.
The initial value of the state variables heavily affectsthe predictions, but its effect can be reduced using continuous-time models and starting at the end of dry periods.
Input measurement errors will certainly affect parameterestimation, but if they are not eccessive, they will not strongly affect the predictive uncertainty.
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
ASSESSING PARAMETER ESTIMATION ERRORIn order to overcome the problem of formulating “formal” likelihoods, correctly representing the statistical properties of the error terms Beven and Binley, (1992) introduced the Generalised Likelihood Uncertainty Estimation (GLUE), which follows in principle the Bayesian inference scheme, but uses “less formal likelihoods”as defined by the authors.
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
The introduction of less formal likelihood functions overcomes the need for formulating precise distribution functions for the observable variablesand/or for the errors in complex situations originated by the presence of many sources of errors, the complexity of the explicative models considered and the high number of parameters with which to build the learning process. Unfortunately, as it was shown by Mantovan and Todini (paper in preparation), the use of lessformal likelihood functions leads to paradoxical inferential results.
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
( )( ) ( )
( )⎪⎩
⎪⎨
⎧
>
≤⎥⎦
⎤⎢⎣
⎡−
=22
222
2
1
0
1
nn
nn
N
n
n
n
ss
sss
sL
ϑ
ϑϑϑ
( ) ( )[ ] 122 ≥=− NsL N
nn ϑϑ
( ) ( )[ ] 123 ≥−= NNsExpL nn ϑϑ
Simulation results using three well knownGLUE “less formal likelihoods”
Nash-Suttcliffe
Inverse error variance
Exponential
( ) ( )[ ]nnnn yyVars X,ˆ2 ϑϑ −=
[ ]nn yVars =2
where
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
A two dimensional first order autoregressive process based on two parameters, which known true value was set to ( ) ( )1,1, *
2*1 =ϑϑ
was generated with the addition of a non central Student-t distributed autoregressive noise. The samples were of length n=m=336.The results using the three non formal likelihoodsare compared with the ones obtained using the Gaussian (wrong, but in this case relatively robust)assumption
Synthetic data testing of GLUE
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Bayesian Inference: Gaussian Likelihood
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
GLUE: Nash-Suttcliffe Likelihood
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
GLUE: Inverse Error Variance Likelihood
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
GLUE: Exponential Likelihood
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Bayesian Inference: true likelihood(Posterior to Prior distance)
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Less formal likelihoods (Posterior to Prior distance)
0 2 4 6 8 10 12 14 16 18 200
0.005
0.01
0.015
0.02
0.025
0.03
0.035
0.04Figure 5. Distance between posterior and prior p.f.: Nash-Sutcliffe l.f.l..
Sample size (weeks)
Frob
eniu
s no
rm
5-th percentile 50-th percentile 95-th percentile
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Bayesian Inference: true likelihood(Posterior expected square error loss)
0 2 4 6 8 10 12 14 16 18 200
0.02
0.04
0.06
0.08
0.1
0.12Figure 8. Posterior expected squared error loss: Exact likelihood.
Sample size (weeks)
Frob
eniu
s no
rm
5-th percentile 50-th percentile 95-th percentile
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Less formal likelihoods (Posterior expected square error loss)
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Similar results were found in the Po river case study (37,000 km2 ) by comparing:
- GLUE using a less formal likelihood based on Nash-Sutcliffe statistics
- Bayesian inference assuming a χ2
density for the sum of squared errors
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
GLUE: Nash-Sutcliffe
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
GLUE: Nash-Sutcliffe
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Bayesian Inference: χ2
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Bayesian Inference: χ2
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
GLUE: Nash-Sutcliffe (4 x 400)
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
GLUE: Nash-Sutcliffe (2 x 800)
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
GLUE: Nash-Sutcliffe (1 x 1600)
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Bayesian Inference: χ2 (4 x 400)
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Bayesian Inference: χ2 (2 x 800)
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Bayesian Inference: χ2 (1 x 1600)
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Another example using GLUE on a Chinese 10,000 km2 catchment
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
The Bayesian inference using the Normal Quantile Transform on the same Chinese catchment
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
If we are able of determining a relativelypeaky posterior density for the parameterswe can resonably question ourselves whetherthe use of a set of parameters can stronglydistort the estimation of the model predictive density. In other words shouldwe use the conditionalor the unconcitional density?
Inequifinality as opposed to equifinality
( )XMIXyf ,,,ˆ,ˆ ϑ
( ) ( ) ( )∫= ϑϑϑ dXMIYfXMIXyfXMIYXyf ,,,",,,,ˆ,,,,ˆ
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Unconditional
Conditional
Predictive uncertainty in hindcast model(A Chinese catchment)
As one can see from this slide and from the following one, the difference between unconditional and conditional densities canbe relatively small if the parameter posterior density is dense around the modal value.
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Predictive uncertainty in hindcast mode(A Chinese catchment)
Solid: UnconditionalDashed: Conditional
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
In order to provide the probability densityof the future values of our predictand (stage,discharge, etc.) conditional on our forecasts,we need to answer the following questions:
Which are the means we have to express the input (mostly future rainfall) predictive uncertainty?
How do we operate in practice ?
Predictive uncertainty in forecast mode
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Presently, we can hardly make direct use of the members of the Ensemble Forecasts provided by the Numerical Weather Prediction models.
For instance, the followingexample on the Po river in Italy shows that while the“deteministic” run (dashed line)provides a relatively goodrainfall forecast, the “spaghetti”ensemble produces extremelybiased estimates (possibly dueto the coarser model mesh).Similar results obtained in project EFFS using different NWP models confirm this situation.
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
At NOAA a post-processor for rainfall productsgenerated by NWP models has been developed using the NQT and the ensemble mean(Slide: John Schaake).
Observed0
Y
X
Archived data
Noassumptionof normalityfor observed
& forecastdistributions
ZX
Joint distribution
ZY
zX0
zY0
Normal Space
NQTObserved
Forecast
For a given forecast
Normal Space
P(ZX zX0 | ZY = zY)
ObservedzX0 ZX
0
1
Conditional Distribution
PQPFgiven a
QPFInverse
NQT
NQT = Normal Quantile Transform
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Ensemble Member Climatologies
0
0,2
0,4
0,6
0,8
1
0 10 20 30 40
Daily Precipitation (mm)
Prob
abili
ty ObsRawSyn
Which allows to correctly reproduce the future precipitation distribution (Slide: John Schaake).
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
John Shaake has shown that using the NQT and the ensemble mean it is possible to derive a probability density of future rainfall conditional to the ensemble mean, namely
In real time, one can then derive one conditional density for each member of the ensemble andmarginalise out the uncertainty caused by the precipitation forecasts.
( )XXf ˆ
( )XXf ˆ
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
NormalQuantileTransform
Gaussian Space
Original Space
The Normal QuantileTransform (NQT)
Both measured andmodeled values aretransformed into a Standard Normal spaceby quantile matching.In the Normal space the derivation of the conditional densities isrelatively simple. Theconditional densities are Then reconverted back into the original space.
Conversion of data in the Gaussian space
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
Conversion of data in the Gaussian space
The NQT fully preserves the rank correlation. Moreoverin the Gaussian space, the joint probability distributionof the observed and modelled variables is a multi-dimensional Gaussian distribution which densitycan be easily estimated.
In hindcast mode one can directly derive the predictive density, while in forecasting mode the predictive density can be derived following the work of Krzysztofowicz (1999).
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
y = 0,7403xR2 = 0,4958
-4
-3
-2
-1
0
1
2
3
4
5
-4 -3 -2 -1 0 1 2 3 4
y
y
The conditional density in hindcast mode
( )ikiy x,ϑˆ
( )iif ηη ˆ
η
η
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
If we accept the conditionality on the chosen model, onthe initial state and on the measurement errors,the problem of predictive uncertainty, can besolved by marginalising the joint density with respectto the parameters and to the forecasted input.Therefore, in practice the original equation
( ) ( ) ( )[ ] ( ) ϑϑϑ dXMIYfXdXfXMIXyfXMIYyf ,,,"ˆˆ,,,,ˆ,,, ∫ ∫=
must be discretised to be used with an ensemble ofparameter values drawn from the parameter space and an ensemble of rainfall forecasts produced bythe NWP models.
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
One possible approach can thus be summarised as follows:
1) Assume a multi-uniform prior on the parameters;
2) Generate, a large number of parameter setsby means of a Monte Carlo approach;
3) Estimate the posterior density of the parametersusing the Bayesian Inference process
on historical records;
4) Estimate, for each generated parameter set, itsposterior probability of occurrence ;
si Ni ,1=∀ϑ si Ni ,1=∀ϑ
si Ni ,1=∀ϑsN
( )XMIYf ,,," ϑ
( )ϑ'f
( )XMIYf i ,,," ϑ
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
5) For each ensemble member and each parameter setestimate the predictive density ;
6) Marginalise out the ensemble uncertainty by uniformlyweighting the ensembles , with
the number of ensemble member used;
7) Marginalise out the parameter uncertainty:
( )XMIXyf ij ,,,,ˆ ϑ
( ) ( )∑=
=eN
jij
ei XMIXyf
NXMIyf
1,,,,ˆ1,,, ϑϑ
eN
eN
( ) ( ) ( )∑=
=sN
iii XMIYfXMIyfXMIYyf
1,,,",,,,,, ϑϑ
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
One has to realise that the required computationaleffort can be quite substantial given that Ne ,the numberof ensemble members is generally around 50 and Ns the number of parameter sets is generally of the order of several thousands (t10000).
Therefore, given that one must repeat the procedurefor each future value, two alternatives can be conceived,that can reduce the computational effort as well as therequired computation time.
Possible alternatives
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
The first alternative, similarly to what is done in meteorology, is to cluster the parameter sets in groups (possibly into equiprobable groups) and use a representative set per each group. The number of these groups could be of the order of 100, thusreducing by two orders of magnitude the computational burden.
Alternative 1
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
If one can prove the substantial coincidence of the predictive density marginalised with respect to the parameters and of the one conditional to an estimatedparameter set value (for instance a ML estimated value)as in the Chinese catchment case, it is possible to furtherreduce the computational effort by repeting steps 5 and 6as follows.
Alternative 2
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
5) For each ensemble member estimate the predictivedensity ;
6) Marginalise out the ensemble uncertainty by uniformlyweighting the ensembles , with
the number of ensemble member used;
eN
eN
( )XMIXyf j ,,,ˆ,ˆ ∗ϑ
( ) ( )∑=
∗∗ =eN
jj
e
XMIXyfN
XMIyf1
,,,ˆ,ˆ1,,,ˆ ϑϑ
Alternative 2
Department of Earth and Geo-Environmental SciencesUniversity of Bologna
CONCLUDING REMARKSThe objective of this lecture was not to propose solutions, rather to set forth the perception of the main unresolvedproblems in the assessment of Real Time FloodForecasting Uncertainty and to propose and discuss a number of possible alternative approaches.
It must be clear that these are ideas and research linesfollowing which we hope to find the appropriate solutionswithin the frame of HEPEX and of a number of Europeanfunded research projects.