ibrahim hoteit
DESCRIPTION
Examples of Four-Dimensional Data Assimilation in Oceanography. Ibrahim Hoteit. University of Maryland October 3, 2007. Outline. 4D Data Assimilation 4D-VAR and Kalman Filtering Application to Oceanography Examples in Oceanography 4D-VAR Assimilation Tropical Pacific, San Diego, … - PowerPoint PPT PresentationTRANSCRIPT
Ibrahim Hoteit
Examples of Four-Dimensional Data Examples of Four-Dimensional Data Assimilation in OceanographyAssimilation in Oceanography
University of Maryland
October 3, 2007
2
OutlinOutlinee
4D Data Assimilation 4D Data Assimilation 4D-VAR and Kalman Filtering4D-VAR and Kalman Filtering Application to OceanographyApplication to Oceanography
Examples in OceanographyExamples in Oceanography 4D-VAR Assimilation4D-VAR Assimilation Tropical Pacific, San Diego, …Tropical Pacific, San Diego, …
Filtering MethodsFiltering Methods Mediterranean Sea, Coupled models, Mediterranean Sea, Coupled models,
Nonlinear filtering ...Nonlinear filtering ...
Discussion and New ApplicationsDiscussion and New Applications
3
Data AssimilationData Assimilation
GoalGoal: : Estimate the state of a dynamical systemEstimate the state of a dynamical system
InformationInformation: : Imperfect dynamical ModelImperfect dynamical Model:
state vector, model error transition operator form to
Sparse observationsSparse observations:
observation vector, observation error
observational operator
A priori KnowledgeA priori Knowledge: and its uncertainties
x
( ) ok k k ky H x
1 1, 1( ) k k k k kx M x
kH
1,k kM k 1k(0,Q) N
oy (0,R) N
bx B
4
Data AssimilationData Assimilation
Data assimilationData assimilation: Use all available information to : Use all available information to determine the best possible estimate of the system determine the best possible estimate of the system statestate Observations show the real trajectory to the model Model dynamically interpolates the observations
3D assimilation:3D assimilation: Determine an estimate of the state at a given time given an observation by minimizing
4D assimilation:4D assimilation: Determine given
4D-VAR and Kalman Filtering4D-VAR and Kalman Filtering
ax
1 , ,a aNx x
oy
1 , ,o oNy y
1 1( ) R B To o b bx y Hx y Hx x x x xJ
5
4D-VAR Approach4D-VAR Approach Optimal Control:Optimal Control: Look for the model trajectory Look for the model trajectory
that best fits the observations by adjusting a set that best fits the observations by adjusting a set of “control variables” of “control variables” minimize minimize
with the model as constraintwith the model as constraint: :
is the control vector and may include any model parameter (IC, OB, bulk coefficients, etc) … and model errors
Use a gradient descent algorithm to minimize Most efficient way to compute the gradients is to
run the adjoint model backward in time
1 1, 1( ) k k k k kx M x
J
1 1
1( ) R B
TN o o b bk k k k k kk
c y H x y H x c c c cJ
c
6
Kalman Filtering ApproachKalman Filtering Approach
Bayesian estimationBayesian estimation: Determine : Determine pdfpdf of given of given
Minimum Variance Minimum Variance (MV) estimate (minimum error (MV) estimate (minimum error on average)on average)
Maximum a posterioriMaximum a posteriori (MAP) estimate (most likely) (MAP) estimate (most likely)
Kalman filter (KF)Kalman filter (KF): : provides the MV (and MAP) provides the MV (and MAP) estimate for linear-Gaussian Systemsestimate for linear-Gaussian Systems
1:o
k kx yP
1:maxa
k ok kx y
x Arg P
1:/a ok k kx E x y
kx 1: 1 , , o o ok ky y y
7
Analysis Step (observation)Analysis Step (observation)
The Kalman Filter (KF) AlgorithmThe Kalman Filter (KF) Algorithm
Initialization StepInitialization Step::
Forecast Step (model)Forecast Step (model)
0 0and Px
Kalman Gain Kalman Gain
Analysis stateAnalysis state
Analysis Error Analysis Error covariancecovariance P P P a f f f
k k k k kG H
Forecast stateForecast state
Forecast Error Forecast Error covariancecovariance , 1 1 1P P Qf a k
k k k k k kM M
1P [ P R ] f T f Tk k k k k k kG H H H
[ ]a f o fk k k k k kx x G y H x
, 1 1 f ak k k kx M x
8
Application to OceanographyApplication to Oceanography 4D-VAR and the Kalman filter lead to the same 4D-VAR and the Kalman filter lead to the same
estimate at the end of the assimilation window when estimate at the end of the assimilation window when the system is linear, Gaussian and perfectthe system is linear, Gaussian and perfect
Nonlinear system:Nonlinear system: 4D-VAR cost function is non-convex multiple minima Linearize the system suboptimal Extended KF (EKF)
System dimension ~ 10System dimension ~ 1088:: 4D-VAR control vector is huge KF error covariance matrices are prohibitive
Errors statistics:Errors statistics: Poorly known Non-Gaussian: KF is still the MV among linear
estimators
R, Q, B
9
4D Variational 4D Variational Assimilation Assimilation
ECCO 1ECCO 1oo Global Assimilation System Global Assimilation System
Eddy-Permitting 4D-VAR AssimilationEddy-Permitting 4D-VAR Assimilation
ECCO Assimilation Efforts at SIO ECCO Assimilation Efforts at SIO Tropical Pacific, San Diego, …Tropical Pacific, San Diego, …
In collaboration with the ECCO group, especiallyIn collaboration with the ECCO group, especially
Armin KArmin Kööhl*, Detlef Stammer*, Patrick Heimbach**hl*, Detlef Stammer*, Patrick Heimbach***Universitat Hamburg/Germany, **MIT/USA*Universitat Hamburg/Germany, **MIT/USA
10
ECCO 1ECCO 1oo Global Assimilation Global Assimilation SystemSystem Model:Model:
Data:Data:
Assimilation scheme:Assimilation scheme: 4D-VAR with control of the 4D-VAR with control of the initial conditions and the atmospheric forcing (with initial conditions and the atmospheric forcing (with diagonal weights!!!)diagonal weights!!!)
ECCO reanalysis:ECCO reanalysis: 11oo global ocean state and global ocean state and atmospheric forcing from 1992 to 2004, …and from atmospheric forcing from 1992 to 2004, …and from 1952 1952 2001 (Stammer et al. …) 2001 (Stammer et al. …)
MITGCM (TAF-compiler enabled)MITGCM (TAF-compiler enabled) NCEP forcing and Levitus initial conditionsNCEP forcing and Levitus initial conditions
Altimetry (daily):Altimetry (daily): SLA TOPEX, ERS SLA TOPEX, ERS SST (monthly):SST (monthly): TMI and Reynolds TMI and Reynolds Profiles (monthly) :Profiles (monthly) : XBTs, TAO, Drifters, SSS, ... XBTs, TAO, Drifters, SSS, ... ClimatologyClimatology (Levitus S/T) and (Levitus S/T) and GeoidGeoid (Grace (Grace
mission) mission)
11
ECCO Solution FitECCO Solution Fit
ECCECCOO
JohnsonJohnson
Equatorial Under Current Equatorial Under Current (EUC)(EUC)
12
RegionalRegional: 26: 26ooS S 26 26ooN, 1/3N, 1/3oo, 50 layers, ECCO O.B., 50 layers, ECCO O.B.
Data:Data: TOPEX, TMI SST, TAO, XBT, CTD, ARGO, Drifters; TOPEX, TMI SST, TAO, XBT, CTD, ARGO, Drifters;
all at roughly daily frequencyall at roughly daily frequency Climatology: Levitus-T and S, Reynolds SST and Climatology: Levitus-T and S, Reynolds SST and
GRACEGRACE
ControlControl: Initial conditions , 2-daily forcing, and weekly : Initial conditions , 2-daily forcing, and weekly O.B. O.B.
Smoothing:Smoothing: Smooth ctrl fields using Laplacian in the Smooth ctrl fields using Laplacian in the horizontal and first derivatives in the vertical and in time horizontal and first derivatives in the vertical and in time
First guessFirst guess: Levitus (I.C.), NCEP (forcing), ECCO : Levitus (I.C.), NCEP (forcing), ECCO (O.B.)(O.B.)
ECCO Tropical Pacific ECCO Tropical Pacific Configuration Configuration
OB = (U,V,S,T)
MITGCM MITGCM
Tropical PacificTropical Pacific
13
Eddy-Permitting 4D-VAR Eddy-Permitting 4D-VAR AssimilationAssimilation
The variables of the adjoint model exponentially increase in timeThe variables of the adjoint model exponentially increase in time Typical behavior for the adjoint of a nonlinear chaotic modelTypical behavior for the adjoint of a nonlinear chaotic model Indicate unpredictable events and multiple local minimaIndicate unpredictable events and multiple local minima Correct gradients but wrong sensitivitiesCorrect gradients but wrong sensitivities Invalidate the use of a gradient-based optimization algorithm Invalidate the use of a gradient-based optimization algorithm
Assimilate over short periods (2 months) where the adjoint is Assimilate over short periods (2 months) where the adjoint is stablestable
Replace the original unstable adjoint with the adjoint of a Replace the original unstable adjoint with the adjoint of a tangent linear model which has been modified to be stable (Köhl tangent linear model which has been modified to be stable (Köhl et al., Tellus-2002)et al., Tellus-2002)
Exponentially increasing gradients were filtered out using Exponentially increasing gradients were filtered out using larger viscosity and diffusivity terms in the adjoint modellarger viscosity and diffusivity terms in the adjoint model
14
Visc = 1e11 & Diff = 4e2
HFL gradients after 45 days with HFL gradients after 45 days with increasing viscositiesincreasing viscosities
10*Visc & 10*Diff
30*Visc & 30*Diff
20*Visc & 20*Diff
15
Initial temperature gradients after 1 year Initial temperature gradients after 1 year (2000)(2000)
10*Visc & 10*Diff
20*Visc & 20*Diff
16
Data Cost Function TermsData Cost Function Terms
1/3;39
1;39
1/6;39
1;23
17
Control Cost Function TermsControl Cost Function Terms
1/3;39
1;39
1/6;39
1;23
18
Fit to DataFit to Data
1/3;39
1;39
1/6;39
1;23
19
Assimilation Solution (weekly field end of August)Assimilation Solution (weekly field end of August)
20
What Next …What Next …
Fit is quite good and assimilation solution is Fit is quite good and assimilation solution is reasonablereasonable
Extend assimilation period over several years Extend assimilation period over several years Add new controls to enhance the Add new controls to enhance the
controllability of the system and reduce errors controllability of the system and reduce errors in the controlsin the controls
Improve control constraints …Improve control constraints … Some referencesSome references Hoteit et al. (QJRMS-2006)Hoteit et al. (QJRMS-2006)
Hoteit et al. (JAOT-2007)Hoteit et al. (JAOT-2007) Hoteit et al. (???-2007) Hoteit et al. (???-2007)
21
Other MITGCM Assimilation Efforts Other MITGCM Assimilation Efforts at SIOat SIO
1/101/10oo CalCOFI 4D-VAR assimilation system CalCOFI 4D-VAR assimilation system Predicting the loop current in the Gulf of Mexico …Predicting the loop current in the Gulf of Mexico …
San Diego high frequency CODAR assimilationSan Diego high frequency CODAR assimilation● Assimilate hourly HF radar data and other dataAssimilate hourly HF radar data and other data
Adjoint effectiveness at small scaleAdjoint effectiveness at small scale Information content of surface velocity dataInformation content of surface velocity data
● MITGCM with 1km resolution and 40 layersMITGCM with 1km resolution and 40 layers● ControlControl: I.C., hourly forcing and O.B.: I.C., hourly forcing and O.B.● First guessFirst guess: one profile T, S and TAU (no U, V, S/H-: one profile T, S and TAU (no U, V, S/H-
FLUX)FLUX)● Preliminary resultsPreliminary results: 1 week, no tides: 1 week, no tides
22
Time evolution of the Time evolution of the normalized radar costnormalized radar cost
1/3;39
1;39
1/6;39
1;23
Model Domain and Model Domain and Radars CoverageRadars Coverage
23
Assimilation Solution: SSH / (U,V) & Wind Adj. Assimilation Solution: SSH / (U,V) & Wind Adj.
1/3;39
1;39
1/6;39
1;23
24
What Next …What Next …
Assimilation over longer periodsAssimilation over longer periods Include tidal forcingInclude tidal forcing Coupling with atmospheric modelCoupling with atmospheric model Nesting into the CalCOFI modelNesting into the CalCOFI model
25
Filtering MethodsFiltering Methods Low-Rank Extended/Ensemble Kalman Low-Rank Extended/Ensemble Kalman
FilteringFiltering SEEK/SEIK Filters SEEK/SEIK Filters Application to Mediterranean SeaApplication to Mediterranean Sea Kalman Filtering for Coupled ModelsKalman Filtering for Coupled Models Particle Kalman FilteringParticle Kalman Filtering
In collaboration withIn collaboration with
D.-T. Pham*, G. Triantafyllou**, G. Korres**D.-T. Pham*, G. Triantafyllou**, G. Korres***CNRS/France, **HCMR/Greece*CNRS/France, **HCMR/Greece
26
Reduced-order Kalman filters:Reduced-order Kalman filters: Project on a low-dim Project on a low-dim
subspacesubspace
Kalman correction along the directions ofKalman correction along the directions of
Reduced error subspace Kalman filtersReduced error subspace Kalman filters: : has low-rankhas low-rank
Ensemble Kalman filters:Ensemble Kalman filters: Monte Carlo approach toMonte Carlo approach to
Correction along the directions ofCorrection along the directions of
Low-rank Extended/Ensemble Low-rank Extended/Ensemble Kalman FilteringKalman Filtering
P P k k
Tk kL L
P P Tk k k kx Lx L L
11
P ( )( )N i i i i T
k k k k kN ix x x x
Pk
( ); ( )
( , ); P ( , )
x n x r
L n r r r
L
1 1[ ] i Nk k k kx x x x
x
27
1
1
( )
M
f ak k k
k k k
x M x
L L
1 1 11
1
H R H
H R [ H ( )]
T
k k k k i k k
Ta f o fk k k k k k k k k k
U U L L
x x L U L y x
AnalysiAnalysiss
ForecasForecastt
Low-rank (r) error subspace Kalman filters:Low-rank (r) error subspace Kalman filters:
Singular Evolutive Kalman Singular Evolutive Kalman (SEEK) Filters(SEEK) Filters
0 0 0 0 0,P Tx L U L
SEIK:SEIK: Ensemble variant with (r+1) members only! (~ETKF)
SFEK: SFEK: Fixed variant
A “collection” of SEEK filters:A “collection” of SEEK filters:M Μ
d 0M I L L
SEEK:SEEK: Extended variant
Inflation and LocalizationInflation and Localization
28
The Work Package WP12 in The Work Package WP12 in MFSTEPMFSTEP
EU project between several European institutesEU project between several European institutes
Assimilate physical & biological observations into Assimilate physical & biological observations into coupled ecosystem models of the Mediterranean coupled ecosystem models of the Mediterranean Sea:Sea:
Develop coupled physical-biological model for Develop coupled physical-biological model for regional and coastal areas of the Mediterranean regional and coastal areas of the Mediterranean SeaSea
Implement Kalman filtering techniques with the Implement Kalman filtering techniques with the physical and biological modelphysical and biological model
… … Investigate the capacity of surface observations Investigate the capacity of surface observations (SSH, CHL) to improve the behavior of the (SSH, CHL) to improve the behavior of the coupled systemcoupled system
29
2 2n, u, v, u, v, T, S, q , q l px
The Coupled POM – BFM The Coupled POM – BFM ModelModel
One way coupled: Ecology does not affect the physics
1,2,3,4 4,5,6
1,6,7
O2o,O3o,N1p,N3n,N5s,N4n,P (c,n,p,s,i),Z (c,n,p),R 6(c,n,p,s),
R1(c,n,p),B1(c,n,p),Q (c,n,p,s)
ex
30
A Model SnapshotA Model Snapshot
1/10o Eastern Mediterranean configuration 25 layers
Elevation and Mean Velocity
Mean CHL integrated 1-120m
31
Assimilation into POMAssimilation into POM
Model Model = = 1/10o Mediterranean configuration with 25 layers
ObservationsObservations = = Altimetry, SST, Profiles T & S profiles, Argo data, and XBTs on a weekly basis
SEIK FilterSEIK Filter with rank 50 (51 members)
InitializationInitialization = = EOFs computed from 3-days outputs of a 3-year model integration
Inflation factorInflation factor = 0.5 = 0.5
LocalizationLocalization = 400 Km = 400 Km
32
Assimilation into Assimilation into POMPOM
Free-Run
Forecast
Analysis
Obs Error = 3cm
Mean Forecast RMS Mean Forecast RMS ErrorError
Mean Free-run RMS ErrorMean Free-run RMS Error
Mean Analysis RMS Mean Analysis RMS ErrorError
SSH RMS MisfitsSSH RMS Misfits
33 22/04/2322/04/23 ECOOP KICK-OFFECOOP KICK-OFF
Salinity RMS Error Salinity RMS Error Time SeriesTime Series
FerryBox data at Rhone FerryBox data at Rhone RiverRiver
07/12/05: SATELLITE SSH
SSH 07/12/2005SSH 07/12/2005
FREE RUNFORECASTANALYSIS
34
Assimilation into BFMAssimilation into BFM
Model Model = 1/10= 1/10oo Eastern Mediterranean with 25 Eastern Mediterranean with 25 layers with perfect physicslayers with perfect physics
ObservationsObservations = SeaWiFS CHL every 8 days in = SeaWiFS CHL every 8 days in 1999 1999
SFEK FilterSFEK Filter = SEEK with invariant correction = SEEK with invariant correction subspacesubspace
Correction subspaceCorrection subspace = 25 EOFs computed from = 25 EOFs computed from 2-days outputs of a one year model integration2-days outputs of a one year model integration
Inflation factorInflation factor = 0.3 = 0.3
LocalizationLocalization = 200 Km = 200 Km
35
Assimilation into Assimilation into BFMBFM
Analysis
Forecast
Free-Run
CHL RMS MisfitsCHL RMS Misfits
36
CHL Cross-Section CHL Cross-Section at 34at 34ooNN
Ph Cross-Section at Ph Cross-Section at 2828ooEE
Kalman Filtering for Coupled Kalman Filtering for Coupled ModelsModels
1( )
( )
p p p pk k k k
p p p pk k k k
x M x
y H x
1( , )
( )
e e e p ek k k k k
e e e ek k k k
x M x x
y H x
p
e p pe ex x y y
xMax P
x
Physical System
Ecological System
MAPMAP: : Direct maximization of the joint Direct maximization of the joint conditional density conditional density
standard Kalman filter estimation standard Kalman filter estimation problemproblem
Joint approach:Joint approach: strong coupling and same strong coupling and same filter (rank) !!!filter (rank) !!!
Dual ApproachDual Approach Decompose the joint density into marginal densities p p p p p pe e e e ex x y y x x y y x y y
P P P
max & maxp ep p p pe e ex y y x x y y
x Arg P x Arg P
Compute MAP estimators from each marginal density
Separate optimization leads to two Kalman filters …
Different degrees of simplification and ranks for each filter significant cost reduction
Same from the joint or the dual approach
The physical filter assimilate and
exeypy
39
RRMS for state vectorsRRMS for state vectors
Physics Physics BiologyBiology
RefDualJoint
Ref DualJoint
Twin-Experiments 1/10o Eastern Mediterranean (25 layers)
Joint: SEEK rank-50 Dual: SEEK rank-50 for physics SFEK rank-20 for biologyREF FILTER
REF
X XRRMS
X X
..
40
What Next …What Next …
Joint/Dual Kalman filtering with real dataJoint/Dual Kalman filtering with real data
State/Parameter Kalman estimationState/Parameter Kalman estimation
Better account for model errorsBetter account for model errors
Some referencesSome references Hoteit et al. (JMS-2003), Triantafyllou et al. Hoteit et al. (JMS-2003), Triantafyllou et al.
(JMS-2004),(JMS-2004),
Hoteit et al. (NPG-2005), Hoteit et al. (AG-2005),Hoteit et al. (NPG-2005), Hoteit et al. (AG-2005),
(Hoteit et al., 2006), Korres et al. (OS-2007)(Hoteit et al., 2006), Korres et al. (OS-2007)
41
Nonlinear Filtering - MotivationsNonlinear Filtering - Motivations
The EnKF is “semi-optimal”; it is analysis step is linear The optimal solution can be obtained from the optimal
nonlinear filter which provides the state pdf given previous data
Particle filter (PF) approximates the state pdf pdf by mixture of Dirac functions but suffers from the collapse (degeneracy) of its particles (analysis step only update the weights )
Surprisingly, recent results suggest that the EnKF is more stable than the PF for small ensembles because the Kalman correction attenuates the collapse of the ensemble
1,
i
N i
xiw
iw
42
The Particle Kalman Filter The Particle Kalman Filter (PKF)(PKF)
The PKF uses a Kernel estimator to approximate the pdfpdfss of the nonlinear filter by a mixture of Gaussian densities
The state pdfspdfs can be always approximated by mixture of Gaussian densities of the same form: Analysis Step: Kalman-type: EKF analysis to update and Particle-type: weight update (but using instead of
) Forecast Step: EKF forecast step to propagate and
Resampling Step: …
1 1 111: 1/ G ;N i i i
k k k k k kikx yP w x x
R
ix Pi
ix Pi
43
Particle Kalman Filtering in Particle Kalman Filtering in OceanographyOceanography
It is an ensemble It is an ensemble ofof extended Kalman filters with extended Kalman filters with weights!! weights!!
Particle Kalman Filtering Particle Kalman Filtering requires simplification of the particles error requires simplification of the particles error
covariance matrices covariance matrices
The EnKF can be derived as a simplified PKFThe EnKF can be derived as a simplified PKF Hoteit et al. (MWR-2007) successfully tested one low-Hoteit et al. (MWR-2007) successfully tested one low-
rank PKF with twin experiments rank PKF with twin experiments What Next …What Next …
Derive and test several simplified variants of the PKF Derive and test several simplified variants of the PKF Assess the relevance of a nonlinear analysis step: Assess the relevance of a nonlinear analysis step:
comparison with the EnKFcomparison with the EnKF Assimilation of real data …Assimilation of real data …
44
Discussion and New ApplicationsDiscussion and New Applications Advanced 4D data assimilation methods can be now Advanced 4D data assimilation methods can be now
applied to complex oceanic and atmospheric problemsapplied to complex oceanic and atmospheric problems More work is still needed for the estimation of the error More work is still needed for the estimation of the error
covariance matrices, the assimilation into coupled models, covariance matrices, the assimilation into coupled models, and the implementation of the optimal nonlinear filter and the implementation of the optimal nonlinear filter
New ApplicationsNew Applications:: ENSO prediction using neural models and Kalman ENSO prediction using neural models and Kalman
filtersfilters Hurricane reconstruction using 4D-VAR ocean Hurricane reconstruction using 4D-VAR ocean
assimilation!assimilation! Ensemble sensitivities and 4D-VAREnsemble sensitivities and 4D-VAR Optimization of Gliders trajectories in the Gulf of MexicoOptimization of Gliders trajectories in the Gulf of Mexico ……
THANK YOU
45
46
4D-VAR or (Ensemble) Kalman 4D-VAR or (Ensemble) Kalman Filter?Filter?
Easier to understandEasier to understand More portable More portable (easier to implement?)(easier to implement?)
No low-rank deficiencyNo low-rank deficiency Support different degrees Support different degrees of simplificationsof simplifications
Easier to incorporate a Easier to incorporate a complex background complex background covariance matrix covariance matrix
Low-rank estimates of Low-rank estimates of the error cov. matrices the error cov. matrices (better forecast!)(better forecast!)
Dynamically consistent Dynamically consistent solutionsolution
Still room for Still room for improvement …improvement …
4D-VAR4D-VAR EnKFEnKF
4D-VAR or EnkF? …4D-VAR or EnkF? …
47
Sensitivity to first guess (25 Iterations)Sensitivity to first guess (25 Iterations)
48
Comparison with TAO-Array RMSComparison with TAO-Array RMS
1/3;39
1;39
1/6;39
1;23
RMS Meridional Velocity (m/s)
RMS Zonal Velocity (m/s)
49
San Diego HF Radar Currents San Diego HF Radar Currents AssimilationAssimilation
Assimilate hourly HF radar data and other data Assimilate hourly HF radar data and other data Goals:Goals:
Adjoint effectiveness at small scaleAdjoint effectiveness at small scale Information content of surface velocity dataInformation content of surface velocity data Dispersion of larvae, nutrients, and pollutantsDispersion of larvae, nutrients, and pollutants
MITGCM with 1km resolution with 40 layersMITGCM with 1km resolution with 40 layers ControlControl: I.C., hourly forcing, and O.B.: I.C., hourly forcing, and O.B. First guessFirst guess: one profile, no U and V, and no : one profile, no U and V, and no
forcing!forcing! Preliminary resultsPreliminary results: 1 week, no tides: 1 week, no tides
50
Cost Function termsCost Function terms
1/3;39
1;39
1/6;39
1;23
51
Assimilation Solution: U & VAssimilation Solution: U & V
1/3;39
1;39
1/6;39
1;23
52
Assimilation SolutionAssimilation Solution
1/3;39
1;39
1/6;39
1;23
53
Gulf of MexicoGulf of Mexico Loop current predictionLoop current prediction
Observations HF radar, Gliders, ADCP, … Observations HF radar, Gliders, ADCP, … Adjoint effectiveness …Adjoint effectiveness …
1/101/10oo with 50 layers with 50 layers
Ctrl: I.C. (S,T), daily forcing, and weekly Ctrl: I.C. (S,T), daily forcing, and weekly
O.B.O.B. Proof of concept assimilating SSH, Levitus Proof of concept assimilating SSH, Levitus
and Reynolds …and Reynolds …
54
What Next …What Next …
Ensemble forecastingEnsemble forecasting Ensemble Kalman filteringEnsemble Kalman filtering Optimization of observation systems Optimization of observation systems
55
Twin-Experiments Twin-Experiments SetupSetup
Spin up EOFs REF – OBS2 years
Pseudo-obs:Pseudo-obs: SSH and CHL surface data every 3 days SSH and CHL surface data every 3 days Initialization:Initialization: start from mean state of the 2 years run start from mean state of the 2 years run Free-run:Free-run: run without assimilation starting from run without assimilation starting from
mean statemean state Evaluation:Evaluation: RMS misfit relative to the misfit from RMS misfit relative to the misfit from
mean statemean state REF FILTER
REF
X XRRMS
X X
Model Model = = 1/10o Eastern Mediterranean with 25 layers
3 months4 years
05/03/02
05/03/02
1996
2000
56
Reduced-order Kalman filters:Reduced-order Kalman filters: Project x on a low-dim Project x on a low-dim
subspacesubspace
Analysis along the directions ofAnalysis along the directions of
Reduced error subspace Kalman filtersReduced error subspace Kalman filters: : has low-rankhas low-rank
Ensemble Kalman filters:Ensemble Kalman filters: Monte Carlo approach toMonte Carlo approach to
Analysis along the directions ofAnalysis along the directions of
Low-rank Extended/Ensemble Low-rank Extended/Ensemble Kalman FilteringKalman Filtering
P P k k
Tk kS S
P P Tk k k kx Sx S S
11
P ( )( )
N i i i i Tk k k k kN i
x x x x
Pk
( ); ( )
( , ); P ( , )
x n x r
S n r r r
P k
S
1 1[ ] i Nk k k kx x x x
●
●
,1
a ikx
●●
●●
●
●
●
●●
●
●
●
●
●
● ●
ModeModell
,f ikx
●
●●
●
●●
●● ●
●●●
●
●
●
, , ,
1
,
[ ]
P H [H P H ]
P cov( )
a i f i f ik k k
f T f T
a a ik
x x G y Hx
G R
x
EnKFEnKF
,
1
[ ]
P H [H P H ]
P P H P
a f f ik k k
f T f T
a f f
x x G y Hx
G R
G
SPFSPF,
,P cov( )
f f ik kf f i
k k
x x
x
●●
●
●
●
ResamplResamplinging
DataData
57
Low-Rank DeficiencyLow-Rank Deficiency
Issues Issues Error covariance matrices are underestimated Error covariance matrices are underestimated Few degrees of freedom to fit the dataFew degrees of freedom to fit the data
Amplification by an inflation factorAmplification by an inflation factor
LocalizationLocalization of the covariance matrix (using Schur product)of the covariance matrix (using Schur product)
P Pf f
58
Joint ApproachJoint Approach
Direct maximization of the joint conditional Direct maximization of the joint conditional density density
standard Kalman filter estimation standard Kalman filter estimation problem acting onproblem acting on
1
1
( )
( )
p p p pk k k k
k e e e ek k k k
x M xx
x M x
and assimilating
( )
( )
p p p pk k k k
k e e e ek k k k
y H xy
y H x
Issues:Issues: Strong coupling and same filter Strong coupling and same filter (rank)(rank)
Dual Approach – Some FactsDual Approach – Some Facts
Only the second marginal density depends on , this means same from the joint or the dual approach
does not depend on , more in line with the one-way coupling of the system
The physical filter assimilates both and : assimilation of guaranties consistency between the two subsystems
The ecological filter assimilates only , but it is forced with the solution of the physical filter
The linearization of the observational operator in the physical filter is a complex operation because of the dependency of the ecology on , it was neglected in this preliminary application
ex
px
eypyey
ey
px
ex
ex
60
Twin-Experiments Twin-Experiments SetupSetup
Spin up EOFs REF – OBS2 years
Pseudo-obs:Pseudo-obs: SSH and CHL surface data every 3 days SSH and CHL surface data every 3 days Initialization:Initialization: start from mean state of the 2 years run start from mean state of the 2 years run Free-run:Free-run: run without assimilation starting from run without assimilation starting from
mean statemean state Evaluation:Evaluation: RMS misfit relative to the misfit from RMS misfit relative to the misfit from
mean statemean state REF FILTER
REF
X XRRMS
X X
Model Model = = 1/10o Eastern Mediterranean with 25 layers
3 months4 years
05/03/02
05/03/02
1996
2000
61
As the Kalman filter, it operates as a succession of As the Kalman filter, it operates as a succession of forecast forecast and analysis steps to update the state and analysis steps to update the state pdfpdf::
Forecast Step:Forecast Step: Integrate the analysis Integrate the analysis pdfpdf with the model with the model
Analysis Step:Analysis Step: Correct the predictive Correct the predictive pdfpdf with the new with the new datadata
Particle Filter approximates the state Particle Filter approximates the state pdf pdf by mixture of by mixture of Dirac functions but suffers from degeneracy.Dirac functions but suffers from degeneracy.
1: 11 1: 1 1( | ) ( ); ( | )
n k kk k k k kp x y x M u Q p u y du
11: 1 1: 1( | ) ( | ) ( );
kk k k k k k k kbp x y p x y y H x R
1,
i
N i
xiw
The Optimal Nonlinear FilterThe Optimal Nonlinear Filter
62
New Directions/ApplicationsNew Directions/Applications New ApplicationsNew Applications
ENSO prediction using surrogate models and ENSO prediction using surrogate models and
Kalman filtersKalman filters Hurricane reconstruction using 4DVAR ocean Hurricane reconstruction using 4DVAR ocean
assimilation!assimilation! Ensemble sensitivities and 4DVAREnsemble sensitivities and 4DVAR
Other InterestsOther Interests Optimal ObservationsOptimal Observations Estimate Model and Observational Errors Estimate Model and Observational Errors Estimate Background Covariance Matrices in 4DVAREstimate Background Covariance Matrices in 4DVAR Study the behavior of the different 4DVAR methods Study the behavior of the different 4DVAR methods
with highly nonlinear modelswith highly nonlinear models