Data assimilation in mesoscale modeling and numerical weather
predictionNils Gustafsson
Croatian – USA Workshop on Mesometeorology – June 2012
Perspective:What are the important issues for
development of operational data assimilation when we are in the move:
From hydrostatic models with a grid resolution of 5-20 km (HIRLAM)
To non-hydrostatic models with a grid resolution of 1-3 km (HARMONIE)
Taking also the needs for probabilistic forecasting into account ??
+ work in a wider community (HIRLAM, ALADIN, Meteo-France + use of the IFS-code (ECMWF)
Outline of talk
Relative importance of different model variables in the initial conditions
Candidate assimilation methods Host model coupling issues Moisture assimilation Radar data assimilation Initialization Surface assimilation
OUTCRY: ‘‘There are too many types of data, too many new ones appearing every year, and there are too few people to work on their assimilation!’’
Could there be a way to prioritize data assimilation efforts?
Importance of different model variables in the initial conditions – convection permitting scales
Questions (for mesoscale foreasting of
convection in this study): What is the relative importance of different model
variables in the initial conditions? What is the time-scale for how perturbations in one model
variable influence other model variables? (Tadjust
)
What is the time-scale for non-linearities to have important effects for different model variables? (T
nl)
What is the “signal strength” of different instruments measuring different variables?
Important: Tadjust
< assimilation window < Tnl
Fabry & Sun:
Approach and tools Identical twin model simulation
experiments Perturb initial data of different variables,
check how perturbations influence other variables
WRF, 4 km grid resolution Use different amplitudes of initial
perturbations, check for how long time linearity holds
Fabry & Sun:
Linear, non-linear and contradictory regimes
Fabry & Sun:
Rainfall accumulation from 16 control cases
Fabry & Sun:
Horizontal spectra and variances of simulated initial errors
Fabry & Sun:
Experimentation strategy
Fabry & Sun
RESULTS – error growthFabry & Sun:
Error spread to different model variables 3 hours
15 minutes
Fabry & Sun:
Non-linearity index Contradictory index
Fabry & Sun:
What can we learn from Fabry & Sun?
Exept for low-level moisture and soil properties, most initial condition errors propagate quickly to other variables. Uncertainties in mid-level moisture caused the greatest uncertainties in the forecasts. For mesoscale forecasts longer than 6h, the uncertainties in all other variables had comparable effects. For forecasts of a few hours, uncertainties in temperatures, low-level moisture and mid-level winds stood out as the greatest cause of forecast uncertainties after mid-level moisture. There are limits on the assimilation interval that are different for different variables, in particular condensates have shorter predictability and can be used with shorter windows only.
Remember: Identical twin experiment !!!
The assimilation method
• Mesoscale balance constraints, in particular involving moisture? Analytical or statistical constraint? Utilization of ensembles?
• How to handle inhomogeneities and an-isotropy? • Flow-dependent structure functions? 4D-Var, ensemble
techniques or synthesis?• How to handle model errors?
Candidate data assimilation techniques for the mesoscale
Variational techniques (3D-Var and 4D-Var) Ensemble Kalman Filters Hybrid variational ensemble methods Nudging (will not be discussed) Warping or alignment to correct for phase errors
Basic idea of 4D-Var: Fit model trajectory to observations over a time window
4D-Var
Possibilities:
• Handle weak non-linearities via outer loops
• Provide flow-dependencies within
assimilation window
Problems:
• Strong non-linearities
• Static influence of observations at the start of the window
• Maintain tangent linear and adjoint codes
• Scalability on parallell computers
Comparison of HIRLAM 3D-Var and 4D-Var for the stormy month of December 1999
(Gustafsson et al. 2012)
The Danish storm, 3 December 1999 18 UTC
3D-Var+30h
4D-Var +30 h
3D-Varanalysis
4D-Varanalysis
Gustafsson et al. 2012)
Why does 4D-Var make a better job than 3D-Var in storm cases?
Improved implicit flow-dependent structure functions. Better quality control decisions due to a better
background and better implicit structure functions Use of more observation? In general, it is difficult to point to a specific reason
for improvements, the improvements generally occur
gradually through the assimilation cycles
Assimilation increments
3D-Var06UTC
3D-Var12UTC
4D-Var06UTC
4D-Var12UTC
Surface pressure increments for the Danish mesoscale storm
(Single simulated observation experiment)
3D-Var
4D-Var, spectral TL prop. of incr
4D-Var; gp model prop. of incr.
Mesoscale data assimilation and the use of ensembles
• Will assumptions on weak non-linearities at 10 km resolution break down at the km scale ? Most likely yes !
• Can ensembles provide information on uncertainties and balances at the mesoscale ? Most likely Yes !
• Is it too risky to put all our efforts on 4D-Var, that works on the 10 km scale but may fail at the km scale ? Yes !
• Should we drop our investments in variational data assimilation and start with ensemble Kalman Filters or even particle filters? No, EnsKF and particle filters still have weaknesses !
• Can we develop hybrid data assimilation schemes that combine the best of the two worlds? We believe so !
Synoptic variation of covariances as calculated from 3 days of 12h minus 36 h forecast differences
20060116-20060118
20060116-20060118
500 hPa geopotential height
Wavelet representation of 500 hPa temperature covariances
From Landelius
Ensemble Kalman Filters
Basic idea: Estimate forecast error covariances from an ensemble of forecasts
Ensemble generation:Perturbed observationsModel physics perturbationsModel tendency perturbations
Advantages:Flow-dependenciesNatural together with EPS
Problems:50-100 members needed; CostlyLow rank of covariance matrix (ensemble size) makes some type of localization necessary
From Whitaker
HIRLAM first approach to use ensembles in 3D-Var and 4D-Var
• Use the ETKF algorithm for re-scaling of a 6h forecast ensemble to an analysis ensemble (estimation of the analysis error covariance).
• Use ensemble of 6h forecasts to estimate the background error covariance and blend it with the static background error covariance.
First version of HIRLAM implementation
Ensemble weights a have horizontal variations only (horizontal localization) and is controlled in spectral space with the assumption of isotropy – this is equivalent to a horizontal localization of covariances with a Schur product based on a horizontal correlation function. The localization is applied for vorticity, divergence, temperature, surface pressure and specific humidity for better balancing.
Forecast verification scores for a first winter case
(12 ensemble members only)
3D-Var (red curve)Hybrid (green curve)Hybrid + tuning of obs. error s.dev. (blue curve)
Difficult to show the same improvement for the hybrid over 4D-Var!
Model level around 800 hPa
Notice increments along a front
Example of assimilation increments (temperature and wind)
(Landelius et al., 2011)
Landelius et al. 2011)
Host model coupling issues
Large scales are poorly assimilated on a small regional domain – one may wish to handle these large scales differently
In case lateral boundary conditions are coming from an earlier host model run, one may wish to update also the inner domain when new LBC conditions are available
One may also wish to control lateral boundary conditions in a mesoscale 4D-Var
Large scale error constraint in 3-4D-Var
(from Dahlgren and Gustafsson, 2012)
Add large-scale constraint to 4D-Var cost function:
Large scales of vorticity are constrained by a short range host model forecast at the start of the 4D-Var assimilation window.
Impact of Jk on surface pressure forecast verification scores
(Dahlgren and Gustafsson, 2012)
WHY?+0h
+6h
In case an object that we want to assimilate is outside the lateral boundaries at the start of the assimilation window, but described by observations inside the lateral boundaries later during the assimilation window, we must control it by lateral boundary conditions. Let it come in!Similarly observations may tell us to let objects go out even if this is not described by the host model LBCs.
Control of lateral boundary conditions
Control of Lateral Boundary Conditions
(1) Introduce the LBCs at the end of the data assimilation window as assimilation control variables (full model state = double size control vector)(2) Introduce the adjoints of the Davies LBC relaxation scheme and the time interpolation of the LBCs(3) Introduce a “smoothing and balancing” constraint for the LBCs into the cost function to be minimized
J = Jb + Jo + Jc + Jlbc
whereJlbc = (Xlbc- (Xlbc)b)T B-1 (Xlbc- (Xlbc)b)
And, as a first approximation, B is taken as identical to B for the background constraint
Single simulated observation impact studyWith control of lBC
Simulated observation SW 9 m/s at 32N 12W3Dec 1999 11 UTC
3 December 1999Assimilation window06UTC – 11UTC
Strong SW inflow in the background field
06 UTC
09 UTC
11 UTC
(Gustafsson, 2012)
Specific problems in assimilation of moisture
Non-Gaussian behavour of water vapour close to zero humidity and saturation
Flow-dependent structures, for example in “rain” and “no-rain” regimes
Sharp vertical gradients (inversions, PBL top)
Handling of model errors (biases)
Effects of a model dry bias (700 hpa RH)
A new moisture control variable (Elias Holm)
Pseudo-relative humidity:
Normaliztion with a standard-deviation depending also on the increment (non-linear transform):
Effect on PDF of new moisture control variable
Standard deviation of relative humidity forecast differences as a function of “background” relative
humidity
Effect on assimilation increments near saturation
Q control variableRH* control variableRH* control variable + relinearization
BUT, no effect on forecast verification scores!
Simulated observation experiments
Shown to be very important for assimilation of radar data in a mesoscale model (AROME)!
From Montmerle & Berre
Smoothing effect of climatological vertical correlations
Single observation experiment with HIRLAM 4D-Var Cloud-affected SEVIRI IR data (water vaour channel) Observation operator includes diagnosis of clouds
(ECMWF simplified physics) and radiative transfer Elias Holm moisture control variable
From Stengel et al. (2012) in preparation
Cloud layer
Jacobians IncrermentsWithout SP With SP
q
T
Impact of clear SEVIRI radiances on forcast scores (relative RMS reduction)
Additional impact of cloud-affected radiances
Assimilation of radar data
As an example, the approach to assimilation of radar reflectivity data within the HIRLAM and ALADIN communities are briefly described.
Meteo-France has been the leading partner (Caumont, Ducrocq, Wattrelot et al.).
HIRLAM and ALADIN have ongoing trials to collect, pre-process and assimilate radar data from an extended European network.
The slides have been “borrowed” from a presentation by Caumont et al. (2006) and Mahfouf (2011)
From Caumont et al. (2006)
From Mahfouf (2011)
From Mafouf (2011)
From Mahfouf (2011)
Weak digital filter constraint
Try to come close to the “slow manifold” at the same time as minimization of Jo helps us to come close to the observations! Note the un-known coefficient γ
df.
Does this help to prevent “noise” in the non-linear model integration?
Horizontal average of the absolute value of the surface pressure tendency.
Scale-selective DFI
(Termonia, 2008)
Also: Scale-selective DFI => less “noise”!
Validation and tuning of HARMONIE surface assimilation
1. Validate the present OI scheme using pre-calculated coeffficients for the soil variables2. Compare the performance of the new EKF scheme for soil variables with the OI scheme – check that everything is correctly implemented3. Tune if needed the EKF scheme (beat the OI scheme !)4. Use of satellite data (ASCATT and SMOS)?
Work started with 1-3.
Example of time-series in SYNOP stations - Gardermoen
Without OI With OI
Reduced wg2
Improveddaily cycle!
Improved RH2
Verification – T2 and RH2
Verification – PMSL !!
Summary – Priority areas for HIRLAM, ALADIN in mesoscale DA
Use of high-resolution observations: - Radar reflectivity and radial winds; - Cloudy radiances; - Ground-based GPS; - Low-level winds (10 met, scattereometer)
Flow-dependencies: - Heterogenious structure functions (rain-no rain, cloud-no cloud); Hybrids; 4D Ens Var; 4D-Var (as reference); – Alignment, warping and dynamical transform;
Balances: - Flow-dependent structure functions; - Mesoscale DF; – Spinup (consistency between DA and physics)
Improve large scales: Jk & Control of LBC in 4D-Var
Surface assimilation: - EKF (=> flow and model dependency); – satellite observations; snow, lakes; - interpolation for heterogenious surfaces
Meteorologisk institutt met.no
powerfulLAM EPS
efficientDA scheme
skillfulcontrol forecast
realisticLAM perturbations
Solution for long term perspective
Transversary issue: interaction of initialisation/dynamics/physics/model error
Greetings from my student Lena Bojarova, 2012