pod/deim strategies for reduced data assimilation...

POD/DEIM Strategies for reduced data assimilationsystems

Razvan Stefanescu 1 Adrian Sandu 1 Ionel M. Navon 2

1Computational Science Laboratory, Department of Computer Science, Virginia PolytechnicInstitute and State University

2Department of Scientific Computing,The Florida State University

Razvan Stefanescu , Adrian Sandu , Ionel M. Navon

POD/DEIM Strategies for reduced data assimilation systems 1/55

Outline

1 Motivation

2 Reduced Order Modelling

3 ROM 4D-Var DA systems - Choice of bases

4 4D-Var SWE DA reduced order systems

5 Numerical resultsChoice of POD basesReduced order POD based SWE 4D-Var DA systems

6 Conclusions and future research



Why do we need data assimilation?

By construction, numerical weather forecasts are imperfect

discrete representation of the atmosphere in space and time(horizontal and vertical grids, spectral truncation, time step)

subgrid-scale processes (e.g. turbulence, convective activity) need tobe parametrized as functions of the resolved-scale variables.

errors in the initial conditions.

Physical parametrizations used in NWP models are constantly beingimproved: a. more and more prognostic variables (cloud variables,precipitation, aerosols), b. more and more physical processesaccounted for (e.g. detailed microphysics).

However, they remain approximate representations of the trueatmospheric behaviour.




The goal of data assimilation is to periodically constrain the initialconditions of the forecast using a set of accurate observations thatprovide our best estimate of the local true atmospheric state.

This is achieved by combining in an optimal statistical way all theinformation on the atmosphere, available over a selected time window(usually 6 or 12h)

Observations with their accuracies (error statistics)

Short-range model forecast (background) with associated errorstatistics

Atmospheric equilibria (e.g. geostrophic balance)

Physical laws (e.g. perfect gas law, condensation)




Data assimilation techniques: OI, Nudging, 3D-Var, 4D-Var,Ensemble DA.




The objective function J to be optimized is defined based onmodel-data misfit penalty terms as:

J(x0) =1

2

(xb − x0

)TB−1

0

(xb − x0

)+

1

2

N∑i=1

(yi − H(xi )

)TR−1i

(yi − H(xi )

),

(1)

subject toxi+1 =Mixi , i = 0, ..,N − 1, (2)




The optimality conditions:

Adjoint model: λi = MTi λi+1 + HTR−1

i

(yi − H(xi )

), i = N − 1, 1;

λN =HTR−1N

(yN − H(xN)

)and λ0 = MT

0 λ1.

(3)

Cost function gradient: ∇x0J = −B−10

(xb− x0

)−λ0 = 0;

(4)



Why do we need reduced order data assimilation?




The operational configuration at ECMWF

Deterministic forecast model: T1279L91 (≈ 16km)

Outer loop of 4D-Var T1279L91

inner loops T159/T255/T255 (≈125km/80km/80km)

The Mesoscale and Microscale Meteorology (MMM) Division of NCARcurrently maintains and supports WRFDA system

Same resolutions in outer loops and inner loops

Data Assimilation of lightning using 4D-Var WRFDA system andnonlinear observation operators

Spatial resolution of 9km and only a 2h assimilation windows




Figure: 1 h precipitation (mm) ending at 2000 UTC 15 June from the controlrun, various assimilation procedures, and stage IV precipitation.


Figure: RMSE of precipitation (mm) for both study days compared with stage IVobservations. Assimilation was not performed after 2000 UTC for the1D+4D-VAR simulation, and not after 0000 UTC for the 3D-VAR and1D+3D-VAR approaches.




Replace the current linearized cost function to be minimized in theinner loop

Low-rank surrogate models that accurately represent sub-grid-scaleprocesses

Highly non-linear and non-smooth observation operators

Increased space and time resolutions

Reduced computational complexity



Reduced Order Modelling

dependent and independent of input

For linear models we are able to produce input-independent highlyaccurate reduced models: balanced truncation, moment matching

For general nonlinear systems, the transfer function approach is notyet applicable and input-specified semi-empirical methods are usuallyemployed

Application of generalized transfer functions and generalized momentmatching (Benner and Breiten [2]) for nonlinear model order reduction




dependent and independent of input

For linear models we are able to produce input-independent highlyaccurate reduced models: balanced truncation, moment matching

For general nonlinear systems, the transfer function approach is notyet applicable and input-specified semi-empirical methods are usuallyemployed

Application of generalized transfer functions and generalized momentmatching (Benner and Breiten [2]) for nonlinear model order reduction



Proper Orthogonal Decomposition

The desired simulation is well approximated in the input collection.

Data analysis is conducted to extract basis functions, fromexperimental data or detailed simulations of high-dimensional systems

Galerkin projections that yield low dimensional dynamical models.

Standard POD models: Its nonlinear reduced terms still have to beevaluated on the original state space making the simulation of thereduced-order system too expensive.

Tensorial POD - Stefanescu et al. [8] - ”Comparison of POD reducedorder strategies for the nonlinear 2D Shallow Water Equations”




Three POD based reduced order models will be considered forderiving reduced order SWE data assimilation systems: standardProper Orthogonal Decomposition (SPOD), tensorial POD (TPOD)and standard POD/Discrete Empirical Interpolation Method(POD/DEIM)

The reduced Jacobians are obtained analytically for all three ROMsand their computational complexity depends only on k the dimensionof POD basis.

We assume a Petrov-Galerkin projection for constructing the reducedorder models. U denotes the POD basis and the test functions arestored in W . W TU = Ik , Ik being the identity matrix of order k . Forsimplicity we assume a POD expansion of x = U x.

The methods differ in the way the nonlinear terms are treated -polynomial quadratic nonlinearity x2.



Reduced Order ModellingStandard POD

N(x) = W T︸︷︷︸k×n

U x�U x︸︷︷︸n×1

, N(x) ∈ Rk (5)

where � is the componentwise multiplication Matlab operator and n isusually the number of spatial mesh points.

Tensorial POD

N(x) =[Ni

]i=1,..,k

∈ Rk ; Ni =k∑

j=1

k∑l=1

Ti ,j ,l xj xl . (6)

N(x) = T︸︷︷︸k×k2

X︸︷︷︸k2×1

T =(Ti ,j ,l

)i ,j ,l=1,..,k

∈ Rk×k×k , Ti ,j ,l =n∑

r=1

Wi ,rUj ,rUl ,r .




Standard POD/DEIM

N(x) ≈W TV (PTV )−1︸︷︷︸k×m

((PTUx)� (PTUx)

)︸︷︷︸

m×1

(7)

where m is the number of interpolation points, V ∈ Rn×m gathers the firstm POD basis modes of the nonlinear term while P ∈ Rn×m is the DEIMinterpolation selection matrix (Chaturantabut [4], Chaturantabut andSorensen [6, 5]).



x(km)

y(km

)

0 2000 4000 60000

500

1000

1500

2000

2500

3000

3500

4000

−6

−4

−2

0

2

4

6

8

10x 10−6

(a) Nonlinear term F11

x(km)

y(km

)

0 2000 4000 60000

500

1000

1500

2000

2500

3000

3500

4000

−4

−3

−2

−1

0

1

2

3

4x 10−7

(b) Nonlinear term F12

x(km)

y(km

)

0 2000 4000 60000

500

1000

1500

2000

2500

3000

3500

4000

−2

−1

0

1

2

3x 10−7

(c) Nonlinear term F21

x(km)

y(km

)

0 2000 4000 60000

500

1000

1500

2000

2500

3000

3500

4000

−2

−1.5

−1

−0.5

0x 10−5

(d) Nonlinear term F22

x(km)

y(km

)

0 2000 4000 60000

500

1000

1500

2000

2500

3000

3500

4000

−4

−2

0

2

4

6

x 10−6

(e) Nonlinear term F31

x(km)

y(km

)

0 2000 4000 60000

500

1000

1500

2000

2500

3000

3500

4000

−5

0

5x 10−6

(f) Nonlinear term F32

Figure: 100 DEIM interpolation points. The background consists in isolines of themaximum values of the nonlinear terms over time.


F11(u,φ) = u� Axu +1

2φ� Axφ, F12(u, v) = v � Ayu

F21(u, v) = u� Axv,F22(v,φ) = v � Ayv +1

2φ� Ayφ,

F31(u,φ) =1

2φ� Axu + u� Axφ, F32(v,φ) =

1

2φ� Ayv + v � Ayφ.

Full ADI SWE Standard POD Tensorial POD POD/DEIM m=180 POD/DEIM m=70

CPU time 950.0314s 161.907 2.125 0.642 0.359

u - 5.358e-5 5.358e-5 5.646e-5 7.453e-5

v - 2.728e-5 2.728e-5 3.418e-5 4.233e-5

φ - 8.505e-5e 8.505e-5 8.762e-5 9.212e-5

Table: CPU time gains and the root mean square errors for each of the modelvariables at tf = 3h for a 3h time integration window. Number of POD modeswas k = 50 and two tests with different number of DEIM points m = 180, 70were simulated.103, 776 spatial points.




103

104

105

10−1

100

101

102

103

No. of spatial discretization points

CP

U t

ime

(sec

on

ds)

SPODTPODPOD/DEIM m=70POD/DEIM m=180FULL

(a) On-line stage

103

104

105

100

101

102

103

No. of spatial discretization points

CP

U t

ime

(sec

on

ds)

SPODTPODPOD/DEIM m=70POD/DEIM m=180

(b) Off-line stage

Figure: Cpu time vs. the number of spatial discretization points for tf = 3h ;number of POD modes = 50; two different numbers of DEIM points 70 and 180have been employed.



ROM 4D-Var DA systems - Choice of bases

The “reduced adjoint” (RA) approach projects the first orderoptimality equations of the full system onto the POD reduced spaces

Accurate low-order surrogate models; Its not clear what informationshould be included in the reduced basis used for full space gradientequation projection

The “adjoint of reduced” (AR) model approach formulates the firstorder optimality conditions from the forward reduced order model

Consistent KKT discrete optimality conditions; Reduced adjointmodel approximates poorly its full counterpart and POD bases relyonly on forward dynamics information.




To guide the POD bases snapshots selection process forPetrov-Galerkin based reduced data assimilation systems governed bynon-linear models

Consistent reduced Karush Kuhn Tucker (KKT) optimality conditionsand accurate reduced POD adjoint model solutions with respect tothe full adjoint model outputs

Every type of reduced optimization involving adjoint models andprojection based reduced order methods including reduced basisapproach will benefit.




We derive the optimality conditions as in the AR approach

Forward POD manifold Uf is computed using snapshots of the fullforward model solution only x ≈ Uf x

Petrov-Galerkin (PG) projection; the test functions POD basis Wf isdifferent than the trial functions POD manifold Uf

JPOD(x0) =1

2

(xb − Uf x0

)TB−1

0

(xb − Uf x0

)T+

1

2

N∑i=1

(yi − H(Uf xi )

)TR−1i

(yi − H(Uf xi )

)T,

(8)

subject to xi+1 = Mi xi , Mi = W Tf MiUf , i = 0, ..,N − 1. (9)




Reduced adjoint model

λi = UTf MT

i Wf λi+1 + UTf HTR−1

i

(yi − H(Uf xi )

), i = N − 1, 1;

λN = UTf HTR−1

N

(yN − H(Uf xN)

)and λ0 = UT

f MT0 Wf λ1

(10)

Reduced Cost Function gradient

∇x0JPOD = −UT

f B−10

(xb − Uf x0

)− λ0 = 0; (11)




RA approach: the full forward and adjoint models are projected ontoseparate reduced manifolds

Uf and Ua are the trial POD reduced subspaces and Wf and Wa arethe test functions POD manifolds, xi ≈ Uf xi , λi ≈ Uaλi , i = 0, ..,N.

Reduced forward model:

xi+1 = Mi xi , Mi = W Tf MiUf , i = 0, ..,N − 1. (12)

Reduced adjoint model:

λi = W Ta MT

i Uaλi+1 + W Ta HTR−1

i

(yi − H(Uf xi

), i = N − 1, 1

λN = W Ta HTR−1

N

(yN − H(Uf xN)

)and λ0 = W T

a MT0 Uaλ1,

(13)




AR adjoint model:

λi = UTf MT

i Wf λi+1 + UTf HTR−1

i

(yi − H(Uf xi )

), i = N − 1, 1;

λN = UTf HTR−1

N

(yN − H(Uf xN)

)and λ0 = UT

f MT0 Wf λ1

(14)

RA adjoint model:

λi = W Ta MT

i Uaλi+1 + W Ta HTR−1

i

(yi − H(Uf xi

), i = N − 1, 1

λN = W Ta HTR−1

N

(yN − H(Uf xN)

)and λ0 = W T

a MT0 Uaλ1,

(15)

For Petrov Galerkin and Galerking projections:

Wf = Ua and Wa = Uf , and Uf = Ua. (16)



4D-Var SWE DA reduced order systems

Algorithm 1 Standard and Tensorial POD SWE DA systems

Off-line stage

1: Generate background state u, v and φ.2: Solve full forward ADI SWE model to generate state variables snapshots.

3: Solve full adjoint ADI SWE model to generate adjoint variables snap-shots.

4: For each state variable compute a POD basis using snapshots describingdynamics of the forward and its corresponding adjoint trajectories.

5: Compute tensors as T required for reduced Jacobian calculations. Cal-culate other POD coefficients corresponding to linear terms.




Algorithm 1 Standard and Tensorial POD SWE DA systems

On-line stage - Minimize reduced cost functional JPOD (8)

1: Solve forward reduced order model (9)2: Solve adjoint reduced order model (10)3: Compute reduced gradient (11)

Decisional stage

4: Project the suboptimal reduced initial condition generated by the on-line stage and perform steps 1 and 2 of off-line stage. Using full forwardinformation evaluate J in (1). If ‖J‖ > ε3 then continue the off-linestage from step 3, otherwise STOP.




Algorithm 2 POD/DEIM SWE DA system

Off-line stage

1: Generate background state u, v and φ.2: Solve full forward ADI SWE model to generate nonlinear terms and

state variables snapshots.3: Solve full adjoint ADI SWE model to generate adjoint variables snap-

shots.4: For each state variable compute a POD basis using snapshots describing

dynamics of the forward and its corresponding adjoint trajectories. Foreach nonlinear term compute a POD basis using snapshots from theforward model.

5: Compute discrete empirical interpolation points for each nonlinear term.6: Calculate other linear POD coefficients and POD/DEIM coefficients as

E11.7: Compute tensors such as T using algorithm described in Stefanescu

et al. [8, p.7] required for reduced Jacobian calculations.


The on-line stage - minimization of the cost function JPOD performedon a reduced POD manifold

The stoping criteria are

‖∇JPOD‖ ≤ ε1, ‖JPOD(i+1) − JPOD

(i) ‖ ≤ ε2, MXFUN ≤ iterMax (17)

The off-line stage - outer iteration - general stopping criterion

‖J‖ ≤ ε3



Numerical Results

ADI SWE model

10% uniform perturbations on the initial conditions of Grammeltvedt[7] and generate twin-experiment observations at every grid spacepoint location and every time step

Background state is computed using a 5% perturbations of the initialconditions

The length of the assimilation window: 3h.

BFGS optimization method (CONMIN)

We use ε1 = 10−14 and ε2 = 10−5.



Numerical Results - Choice of POD basisThe ”adjoint of reduced” (AR) approach is compared with ”adjoint ofreduced + reduced of adjoint” (AR+RA) method using tensorialPOD 4D-Var DA system

AR - no need for implementing the full adjoint SWE model since thePOD basis relies only on forward trajectories snapshots.

We select 31× 23 mesh points 91 time steps and use 50 POD basisfunctions. MXFUN is set to 25 and ε3 = 10−16

||uTPODAR − u|| 2.39e-15 ||uTPOD

AR+RA − u|| 1.39e-6

||vTPODAR − v|| 8.16e-16 ||vTPOD

AR+RA − v|| 7.54e-7

||φTPODAR − φ|| 2.75e-13 ||φTPOD

AR+RA − φ|| 1.59e-6

||λuTPODAR − λu|| 207.38 ||λu

TPODAR+RA − λu|| 1.71e-6

||λvTPODAR − λv || 138.40 ||λv

TPODAR+RA − λv || 7.91e-7

||λφTPODAR − λφ|| 199.377 ||λφTPOD

AR+RA − λφ|| 1.23-6

Table: Average absolute errors of forward (up) and adjoint (down) tensorialPOD SWE initial conditions using AR and AR+RA approaches.Razvan Stefanescu , Adrian Sandu , Ionel M. Navon


Numerical Results - Choice of POD basis

0 25 50 75 100 125 150 175 200−15

−13

−10

−7

−4

−1

2

5

8

log

arit

hm

ic s

cale

Number of eigenvalues

uvφ

(a) Forward model snapshots only

0 25 50 75 100 125 150 175 200−13

−10

−7

−4

−1

2

5

8

log

arit

hm

ic s

cale

Number of eigenvalues

uvφ

(b) Forward and adjoint models snapshots

Figure: The decay around the singular values of the snapshots solutions foru, v , φ for ∆t = 960s and integration time window of 3h .




0 10 20 30 40 50 60 70 80 9010

−25

10−20

10−15

10−10

10−5

100

J/J(

1)

Number of iterations

0 10 20 30 40 50 60 70 80 9010

−25

10−20

10−15

10−10

10−5

100

105

Gra

d/G

rad(

1)

0 10 20 30 40 50 60 70 80 9010

−25

10−20

10−15

10−10

10−5

100

105

0 10 20 30 40 50 60 70 80 9010

−25

10−20

10−15

10−10

10−5

100

105

Grad red. of adj. + adj. of red.Grad adj. of red.

Grad full

Cost func − red. of adj. + adj. of red.Cost func − adj. of red.Cost func − full

Figure: Tensorial POD/4DVAR ADI 2D Shallow water equations – Evolution ofcost function and gradient norm as a function of the number of minimizationiterations. The information from the adjoint equations has to be incorporatedinto POD basis.


0 20 40 60 80 10010

−25

10−20

10−15

10−10

10−5

100


J/J(

1)

red. of adj. + adj. of red.adj. of red.full

(a) Evolution of cost function

0 20 40 60 80 10010

−25

10−20

10−15

10−10

10−5

100


Gra

d J/

Gra

d J(

1)

red. of adj. + adj. of red.adj. of red.full

(b) Evolution of gradient

Figure: Tensorial POD/4DVAR ADI 2D Shallow water equations. Theinformation from the adjoint equations has to be incorporated into POD basis.

POD/DEIM SWE 4D-Var DA systemMesh of 31× 23 and 17× 13 points, a POD basis dimension ofk = 50, and various number of DEIM interpolation points are used.MXFUN is set to 100 and ε3 = 10−16

0 200 400 600 800 100010

−5

10−4

10−3

10−2

10−1

100

J/J(

1)


0 200 400 600 800 100010

−8

10−6

10−4

10−2

100

102

Gra

d/G

rad(

1)

0 200 400 600 800 100010

−8

10−6

10−4

10−2

100

102

Grad m=50

Grad m=180

Cost func − m = 50Cost func − Cost func − m = 180

(a) Number of mesh points 31 × 23

0 50 100 150 20010

−25

10−20

10−15

10−10

10−5

100

J/J(

1)


0 50 100 150 20010

−25

10−20

10−15

10−10

10−5

100

Gra

d/G

rad(

1)

0 50 100 150 20010

−25

10−20

10−15

10−10

10−5

100

0 50 100 150 20010

−25

10−20

10−15

10−10

10−5

100

Grad m=50

Grad m=135

Grad m=165

Cost func − m=50Cost func − m=135Cost func − m=165

(b) Number of mesh points 17 × 13

Figure: Standard POD/DEIM ADI SWE 4D-Var system – Evolution of costfunction and gradient norm as a function of the number of minimizationiterations for different number of mesh points and various number of DEIMpoints.

Reduced order POD based SWE 4D-Var DA systems

0 200 400 600 800 100010

−5

10−4

10−3

10−2

10−1

100


J/J(

1)

m=50m=180


0 200 400 600 800 100010

−8

10−6

10−4

10−2

100

102


Gra

d J/

Gra

d J(

1)

m=50m=180


Figure: Standard POD/DEIM ADI SWE 4D-Var system – Evolution of costfunction and gradient norm as a function of the number of minimizationiterations for different number of mesh points and various number of DEIMpoints. n = 31× 23

Reduced order POD based SWE 4D-Var DA systems

0 50 100 150 20010

−25

10−20

10−15

10−10

10−5

100


J/J(

1)

m=50m=135m=165


0 50 100 150 20010

−25

10−20

10−15

10−10

10−5

100


Gra

d J/

Gra

d J(

1)

m=50m=135m=165


Figure: Standard POD/DEIM ADI SWE 4D-Var system – Evolution of costfunction and gradient norm as a function of the number of minimizationiterations for different number of mesh points and various number of DEIMpoints. n = 17× 13

Hybrid POD/DEIM SWE 4D-Var DA system

10 1 0.1 0.01 0.001 0.0001 1e−005 1e−006 1e−007−7

−6

−5

−4

−3

−2

−1

0

1

(J(x

+δx

) −

J(x

))/ <

∇J(

x),δ

x>

Perturbation ||δx||

0.94

0.96

0.98

1

||M(x

+δx

)(N

T)

− M

(x)(

NT

)||/|

|δ M

(δx)

(NT

)||

Adjoint TestTangent Linear Test

(a) Tangent Linear and Adjoint test

0 10 20 30 40 50 60 7010

−30

10−20

10−10

100

J/J(

1)


0 10 20 30 40 50 60 7010

−30

10−25

10−20

10−15

10−10

10−5

100

0 10 20 30 40 50 60 7010

−30

10−25

10−20

10−15

10−10

10−5

100

Gra

d/G

rad(

1)

Grad Standard POD/DEIM

Grad Hybrid POD/DEIM

Cost func − Hybrid POD/DEIMCost func − Standard POD/DEIM

(b) Cost function and gradient decays

Figure: Tangent linear and adjoint test for standard POD/DEIM SWE 4D-Varsystem. Optimization performances of Standard POD/DEIM and HybridPOD/DEIM 4DVAR ADI 2D shallow water equations for 50 DEIM points and -n = 17× 13 space points.


0 10 20 30 40 50 60 7010

−25

10−20

10−15

10−10

10−5

100


J/J(

1)

Hybrid POD/DEIMStandard POD/DEIM


0 10 20 30 40 50 60 7010

−25

10−20

10−15

10−10

10−5

100


J/J(

1)

Hybrid POD/DEIMStandard POD/DEIM


Figure: Optimization performances of Standard POD/DEIM and HybridPOD/DEIM 4DVAR ADI 2D shallow water equations for 50 DEIM points and -n = 17× 13


0 100 200 300 400 50010

−25

10−20

10−15

10−10

10−5

100


J/J(

1)

k=20k=30k=50k=70k=90full

(a) Number of DEIM points m = 50

0 50 100 150 200 250 300 35010

−25

10−20

10−15

10−10

10−5

100


J/J(

1)

m=10m=20m=30m=50m=100full

(b) Number of POD basis dimension k =50

Figure: Performances of hybrid POD/DEIM SWE DA system with various valuesof POD basis dimensions (a) and different number of interpolation points (b).The spatial configuration uses n = 61× 45 and maximum number of functionevaluation per inner iteration is MXFUN= 30

Hybrid POD/DEIM SWE 4D-Var DA systemk = 20 k = 30 k = 50 k = 70 k = 90 Full

E ou 1.03e-4 1.57e-6 4.1e-11 3.75e-11 2.1e-11 2.77e-12

E ov 4.04e-4 6.29e-6 1.72e-10 1.16e-10 8.55e-11 1.27e-11

E oφ 2.36e-6 3.94e-8 1.09e-12 6.78e-13 2.3e-13 5.79e-14

Table: Relative errors of suboptimal hybrid POD/DEIM SWE 4D-Var solutions,optimal solution computed using high-fidelity ADI SWE 4D-Var system andobservations. The number of DEIM interpolation points is hold constant m = 50while k is varied.

m = 10 m = 20 m = 30 m = 50 m = 100 Full

E ou 4.28e-6 4.91e-8 4.15e-10 4.1e-11 3.08e-11 2.77e-12

E ov 6.88e-6 1.889e-7 1.54e-9 1.72e-10 1.25e-10 1.27e-11

E oφ 7.15e-8 1.04e-9 7.79e-12 1.09e-13 7.39e-13 5.79e-14

Table: Relative errors of suboptimal hybrid POD/DEIM SWE 4D-Var solutions,optimal solution computed using high-fidelity ADI SWE 4D-Var system andobservations. Different DEIM interpolation points are tested and k is holdconstant 50.

POD based SWE 4D-Var DA systemsn = 61× 45 space points, number of POD basis modes k = 50,MXFUN = 10 and ε3 = 10−7.

0 10 20 30 40 50 60 70 8010

−15

10−10

10−5

100


J/J(

1)

hybrid POD/DEIM m=50hybrid POD/DEIM m=120standard PODtensorial POD

(a) Iteration performance

0 50 100 150 200 250 300 35010

−15

10−10

10−5

100

Cpu time

J/J(

1)

hybrid POD/DEIM m=50hybrid POD/DEIM m=120standard PODtensorial POD

(b) Time performance

Figure: Number of minimization iterations and CPU time comparisons forthe reduced Order SWE DA systems vs. full SWE DA system. The spatialconfiguration uses n = 61× 45 and maximum number of function evaluationper inner iteration is MXFUN= 10



POD based SWE 4D-Var DA systems

n = 151× 111 space points, number of POD basis modes k = 50,MXFUN = 15 and ε3 = 10−1.

0 5 10 15 20 25 30 3510

−10

10−8

10−6

10−4

10−2

100


J/J(

1)

hybrid POD/DEIM m=30hybrid POD/DEIM m=50standard PODtensorial PODFull configuration

(a) Iteration performance

100

101

102

103

104

10−12

10−10

10−8

10−6

10−4

10−2

100

Cpu time

J/J(

1)

hybrid POD/DEIM m=30hybrid POD/DEIM m=50standard PODtensorial PODFull configuration

(b) Time performance

Figure: Number of iterations and CPU time comparisons for the reducedOrder SWE DA systems vs. full SWE DA system.



POD based SWE 4D-Var DA systemsSpace points 31× 23 61× 45 101× 71 121× 89 151× 111

ε3 ‖J‖ < 1.e − 09 ‖J‖ < 5.e − 04 ‖J‖ < 1.e − 01 ‖J‖ < 5 ‖J‖ < 1e + 03

hybrid DEIM 50 48.771 63.345 199.468 358.17 246.397hybrid DEIM 120 44.367 64.777 210.662 431.460 286.004

standard POD 63.137 131.438 533.052 760.462 560.619

tensorial POD 54.54 67.132 216.29 391.075 303.95

FULL 10.6441 117.02 792.929 1562.3425 3038.24

Table: CPU time for reduced optimization and full 4D-Var. Number of PODmodes is selected 30 and MXFUN = 15.

Space points 31× 23 61× 45 101× 71 121× 89 151× 111

ε3 ‖J‖ < 1.e − 14 ‖J‖ < 1.e − 07 ‖J‖ < 1.e − 04 ‖J‖ < 5e − 03 ‖J‖ < 1e − 01

hybrid DEIM 30 214.78 288.627 593.357 499.676 594.04hybrid DEIM 50 211.509 246.65 529.93 512.721 603.21

standard POD 190.572 402.208 1243.234 1315.573 1375.4

tensorial POD 269.08 311.106 585.509 662.95 685.57

FULL 14.1005 155.674 1057.715 2261.673 5268.7

Table: CPU time for reduced optimization and full 4D-Var. Number of PODmodes is selected 50 and MXFUN = 15.

Process ||J|| < 1.e − 01 Time # Total

Solve full forward model ≈ 80s 26x ≈ 2080sSolve full adjoint model ≈ 76.45s 26x ≈ 1987.7s

Other (Line Search, Hessian approx) ≈ 46.165 26x ≈ 1200.3sTotal full 4D-Var ≈ 5268s

Table: CPU time for full 4D-Var for a number of mesh points of 151× 111.

Process k= 50 Time # Total

Off-line stageSolve full forward model + nonlinear snap. ≈ 80.88s 2x ≈ 161.76sSolve full adjoint model + nonlinear snap. ≈ 76.45s 2x ≈ 152.9s

SVD for state variables ≈ 53.8 2x ≈ 107.6sSVD for nonlinear terms ≈ 11.57 2x ≈ 23.14s

DEIM interpolation points ≈ 0.115 2x ≈ 0.23sPOD/DEIM model coefficients ≈ 1.06 2x ≈ 2.12s

tensorial POD model coefficients ≈ 8.8 2x ≈ 17.6s

On-line stageSolve ROM forward ≈ 2s 33x ≈ 66sSolve ROM adjoint ≈ 1.9s 33x ≈ 62.69s

Total Hybrid POD/DEIM 4D-Var ≈ 594.04s

Table: Cpu time Hybrid POD/DEIM 4D-Var for a number of mesh points of151× 111, POD basis dimension k = 50 and 30 DEIM interpolation points.

Process k=50 Time # Total

Off-line stageSolve full forward model + nonlinear snap. ≈ 80s 2x ≈ 160sSolve full adjoint model + nonlinear snap. ≈ 76.45s 2x ≈ 152.9s

SVD for state variables ≈ 53.8s 2x ≈ 107.6stensorial POD model coefficients ≈ 23.735s 2x ≈ 47.47s

On-line stageSolve ROM forward ≈ 4.9s 32x ≈ 156.8sSolve ROM adjoint ≈ 1.9s 32x ≈ 60.8s

Total Tensorial POD 4D-Var ≈ 685.57s

Table: The calculation times for solving the optimization problem using TensorialPOD 4D-Var for a number of mesh points of 151× 111, and POD basisdimension k = 50.

Process k=50 Time # Total

On-line stageSolve ROM forward ≈ 26.523s 32x ≈ 846.72sSolve ROM adjoint ≈ 1.9 32x ≈ 60.8s

Total Standard 4D-Var ≈ 1375.4s

Table: The calculation times for on-line stage of Standard POD 4D-Var for anumber of mesh points of 151× 111 and POD basis dimension k = 50. Theoff-line stage is identical with the one in Tensorial POD 4D-Var system.

Conclusions and future research

New efficient POD bases selection strategies for POD based reduced4DVar data assimilation systems governed by nonlinear state modelsusing both Petrov-Galerkin and Galerkin projections.

Consistent reduced Karush Kuhn Tucker (KKT) optimality conditions+ accurate reduced POD adjoint model solutions with respect to thefull adjoint model outputs.

Petrov-Galerkin projection - test functions POD bases of the forwardand adjoint models have to match the trial functions POD bases ofthe adjoint and forward models

Galerkin projection - one single POD basis is required and thecorrelation matrix must contain snapshots from both forward andadjoint full models




Every type of reduced optimization involving adjoint models andprojected based reduced order methods including reduced basisapproach benefit from the new strategies.

Standard POD, Tensorial POD and Standard POD/DEIM SWE4D-Var systems

The POD/DEIM approximations of four nonlinear terms involvingheight field out of ten partially lost their accuracy during theoptimization where input data are different than the ones used togenerate the interpolation points - Hybrid tensorial POD/DEIM4D-Var SWE.

For meshes of 151× 111 points or higher the hybrid POD/DEIMreduced data assimilation system is approximately 10 times fasterthen the full space data assimilation system




This rate increases directly proportional with the the mesh size

Hybrid POD/DEIM SWE 4D-Var is at least two times faster thanstandard POD SWE 4D-Var for numbers of space points larger orequal to 101× 71.

Stabilization strategies proposed by Amsallem and Farhat[1], Bui-Thanh et al. [3] must be pursued in order to obtain feasiblePetrov-Galerkin reduced order data assimilation systems

A generalization of DEIM to approximate operators has not been yetdeveloped.

A-priori error estimates extensions for nonlinear-quadratic reducedorder optimal problems

A-posteriori error estimation apparatus - hybrid POD/DEIM SWE4D-Var system - POD basis construction and to efficiently select thenumber of DEIM interpolation points.

Manuscripts related to the present research effort

R. Stefanescu, A. Sandu, I.M. Navon POD/DEIM Strategies forreduced data assimilation systems, Submitted to Journal ofComputational Physics.

R. Stefanescu, A. Sandu, I.M. Navon Comparison of POD reducedorder strategies for the nonlinear 2D Shallow Water Equations,Submitted to International Journal for Numerical Methods in Fluids.

R. Stefanescu, I.M. Navon, POD/DEIM Nonlinear model orderreduction of an ADI implicit shallow water equations model, Journalof Computational Physics, Vol 237 , pp 95–114.

R. Stefanescu, I.M. Navon, Max Marchand, Henry Fuelberg1D+4D-VAR data assimilation of lightning with WRFDA systemusing nonlinear observation operators, Submitted to Monthly WeatherReview.



Figure: Deep down, the founder of modern weather forecasting, Vilhelm Bjerknes,would have preferred to work on theoretical physics.Until he got funding forweather research, that is.

POD/DEIM Strategies for reduced data assimilationsystems



[1] D. Amsallem and C. Farhat. Stabilization of projection-basedreduced-order models. Int. J. Numer. Meth. Engng., 91:358–377, 2012.

[2] P. Benner and T. Breiten. Two-sided moment matching methods fornonlinear model reduction. Technical Report MPIMD/12-12, MaxPlanck Institute Magdeburg Preprint, June 2012.

[3] T. Bui-Thanh, K. Willcox, O. Ghattas, and B. van Bloemen Waanders.Goal-oriented, model-constrained optimization for reduction oflarge-scale systems. J. Comput. Phys., 224(2):880–896, 2007.

[4] S. Chaturantabut. Dimension Reduction for Unsteady NonlinearPartial Differential Equations via Empirical Interpolation Methods.Technical Report TR09-38,CAAM, Rice University, 2008.

[5] S. Chaturantabut and D.C. Sorensen. Nonlinear model reduction viadiscrete empirical interpolation. SIAM Journal on ScientificComputing, 32(5):2737–2764, 2010.

[6] S. Chaturantabut and D.C. Sorensen. A state space error estimate forPOD-DEIM nonlinear model reduction. SIAM Journal on NumericalAnalysis, 50(1):46–63, 2012.



[7] A. Grammeltvedt. A survey of finite difference schemes for theprimitive equations for a barotropic fluid. Monthly Weather Review, 97(5):384–404, 1969.

[8] R. Stefanescu, A. Sandu, and I.M. Navon. Comparisons of PODreduced order strategies for the nonlinear 2D Shallow Water EquationsModel. Technical Report TR 2, Virginia Polytechnic Institute andState University, February 2014, also submitted to InternationalJournal for Numerical Methods in Fluids.



pod/deim strategies for reduced data assimilation...

Documents