localization and correlation in ensemble kalman filters
TRANSCRIPT
![Page 1: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/1.jpg)
Localization and Correlation in Ensemble Kalman Filters
Jeff Anderson
NCAR Data Assimilation Research Section
The National Center for Atmospheric Research is sponsored by the National Science Foundation.
![Page 2: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/2.jpg)
Sequential ensemble filter assimilation
Impact of single ob y on scalar state component x. Describes problem without loss of generality.
2
![Page 3: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/3.jpg)
Sequential ensemble filter assimilation
Impact of single ob y on scalar state component x.
Compute increments for prior y ensemble. (Details irrelevant for this talk)
3
![Page 4: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/4.jpg)
Sequential ensemble filter assimilation
Impact of single ob y on scalar state component x.
Regress y increments onto x.
Δxi=bΔyi, i=1,…N
N is ensemble size.
b is regression coefficient.
4
![Page 5: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/5.jpg)
Ensemble filters are optimal and exact when…
• Model is linear, • Observation operators are linear, • Observation error is gaussian, • Ensemble size N > Ncrit , • Filter is EAKF (or similar deterministic).
Violating any of these may lead to suboptimal solution.
5
![Page 6: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/6.jpg)
Most commonly observed errors are…
• Too little variance (often solved by inflation). • Too much correlation (often solved by localization).
We will treat all errors as sampling errors.
Introduce sampling error correction that acts as a localization.
6
![Page 7: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/7.jpg)
Working definition of localization
• Multiply regression coefficient by factor α Δxi=αbΔyi , i=1,…N.
• α often a function of distance between observation y and state variable x.
• Gaspari Cohn compact pseudo-gaussian function. • Traditionally, α<1, correlation never too small.
7
![Page 8: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/8.jpg)
Using additional prior information
Need for localization implies that standard ensemble is suboptimal.
Need additional prior assumptions to improve it.
It’s nice to know what these assumptions are, but not always possible.
8
![Page 9: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/9.jpg)
Sampling error correction as a localization
Let be ensemble sample regression. Assume that there is sampling error. View as draw from . Find value of α that minimizes expected RMS
error of after regression. This can be done analytically…
where
€
α = b 2 σ b2 + b 2( ) = Q2 1+ Q2( )
€
Q = b σ b
€
ˆ b
€
ˆ b
€
Normal b ,σ b( )
€
x
9
![Page 10: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/10.jpg)
But, all I have is single estimate of .
Group filter (old work): • Run several ensembles. • Get set of estimates. • Estimate and from this set.
• But, extra cost of group often too high.
€
ˆ b
€
b
€
ˆ b
€
σ b
10
![Page 11: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/11.jpg)
Solution: Use a prior distribution for correlation r
• Try U(-1, 1), all correlations equally likely. • Given this prior and ensemble sample use
Monte Carlo to estimate… • Most likely value of ‘true’ correlation r, • Underlying values of and , • Optimal value of α.
Replace with r , and localize with α. Only a function of and N so we can precompute
r and α.
€
ˆ r €
b
€
σ b€
ˆ r
€
ˆ r
11
![Page 12: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/12.jpg)
Localization is function of N and sample correlation .
This is precomputed for table look-up. 12
€
ˆ r
![Page 13: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/13.jpg)
Correction for r is also function of N and .
This is precomputed for table look-up. 13
€
ˆ r
![Page 14: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/14.jpg)
Test I: Separable linear model, identity obs.
j = 1, 200. Observation error variance is N(0, 1). Has 200 degrees of freedom. N>200 EAKF converges to optimal solution. N<201 diverges without sampling error correction. For N<201, optimal localization is delta function. Compute time mean RMS and α for various N.
Note: Adaptive inflation used to stabilize.
€
x j,t+1 = 1+ c( )x j,t
14
![Page 15: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/15.jpg)
Test I: Separable linear model, identity obs.
40 60 80 100 120 140 160 180 2004
5
6
7
8
9
10
Ensemble Size
Erro
r; Sp
read
Ensemble ErrorSpread
Time mean RMSE and spread as function of N. N=201 is nearly optimal. N<201 fails without sampling error correction localization.
15
![Page 16: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/16.jpg)
Test I: Separable linear model, identity obs.
Time mean localization of observation 101. Should be delta function. Ability to detect noise degrades as N decreases.
16
![Page 17: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/17.jpg)
Test II: Low-order dry dynamical core
5 levels, 30 latitudes, 60 longitudes. Has baroclinic instability. Observations at 300 randomly located ‘radiosondes’: Every 12 hours, Obs errors: PS 1 hPa, T 1K, U and V 1 m/s. 1000 assimilation days.
Adaptive spatially-varying inflation. Background Gaspari Cohn localization.
17
![Page 18: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/18.jpg)
Test II: Low-order dry dynamical core
0.2 0.4 0.6 0.8 1 1.2 Inf10
15
20
25
30
35
40
45
50
Background Localization (Radians)
Tim
e M
ean
PS R
MSE
Sampling Error CorrectedBase
Time mean prior RMSE for PS (Pa) for N=80. No sampling error correction with no GC diverges.
18
![Page 19: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/19.jpg)
Test II: Low-order dry dynamical core
0.2 0.4 0.6 0.8 1 1.2 Inf10
15
20
25
30
35
40
45
50
Background Localization (Radians)
Tim
e M
ean
PS R
MSE
Sampling Error CorrectedBase
Time mean prior RMSE for PS (Pa) for N=40. No sampling error correction with no GC diverges.
19
![Page 20: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/20.jpg)
Test II: Low-order dry dynamical core
0.2 0.4 0.6 0.8 1 1.2 Inf10
15
20
25
30
35
40
45
50
Background Localization (Radians)
Tim
e M
ean
PS R
MSE
Sampling Error CorrectedBase
Time mean prior RMSE for PS (Pa) for N=20. Sampling error correction better for large GC.
20
![Page 21: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/21.jpg)
Test II: Low-order dry dynamical core
Time mean localization for observation of level 3 U on level 3 T state variables. N=80, no Gaspari Cohn. Nearly gaussian but, maximum is not co-located with observation.
21
![Page 22: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/22.jpg)
Test II: Low-order dry dynamical core
Time mean localization for observation of level 3 U on level 3 U state variables. N=80, no Gaspari Cohn. Localization is tripole (similar to group filter results).
22
![Page 23: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/23.jpg)
Test II: Low-order dry dynamical core
Time mean localization for observation of level 3 U on level 3 V state variables. N=80, no Gaspari Cohn. Localization is quadrupole (similar to group filter results).
23
![Page 24: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/24.jpg)
Test II: Low-order dry dynamical core
Time mean localization for observation of level 3 U on level 3 V state variables. N=40, no Gaspari Cohn. Maximum localization is smaller. Background minimum localization is larger (0.3 vs. 0.2).
24
![Page 25: Localization and Correlation in Ensemble Kalman Filters](https://reader031.vdocuments.site/reader031/viewer/2022012513/618c8cfdd0cb9a6e55457835/html5/thumbnails/25.jpg)
Implemented sampling error correction dynamic localization.
Localization is function of correlation and ensemble size.
Improves performance for low-order examples.
Can produce time mean estimates of localization.
Could build statistical models of localization from output.
Can reduce noise, increase balance potentially.
Can be used in concert with other a priori localizations.
Helps when localization is not known a priori.
Conclusions
25