preliminary results comparing adjoint- and … recap of fsoi basics preliminary results general...

IntroductionRecap of FSOI Basics

Preliminary ResultsGeneral Comments

Closing Remarks

Preliminary results comparing adjoint- andensemble-based approaches to observation impact

using the GMAO data assimilation system

Ricardo TodlingFabio Diniz∗ and David Groff†

Global Modeling and Assimilation OfficeNASA

15th JCSDA Technical Review and Science Workshop, 17-19 May 2017

∗ Centro de Previsao de Tempo e Estudos Climaticos, Cachoeira Paulista, Brazil† National Center for Environmental Predictions, College Park, MD

Ricardo Todling Fabio Diniz∗ and David Groff† Adjoint- and ensemble-based approaches to observation impact



Closing Remarks

Outline

1 Introduction

2 Recap of FSOI Basics

3 Preliminary Results

4 General Comments

5 Closing Remarks




Closing Remarks

FSOI at GMAO: from Adjoint- to Ensemble-based

The evolution of FSOI in GEOS DAS:

– GMAO has been calculating FSOI in its Forward Processing (FP) system forseveral years.

– Along the years, FP has evolved from 3dVar to Hybrid-3dVar to what ispresently Hybrid-4dEnVar.

– Our FSOI strategy follows the Langland & Baker (2004) approach and relies onthe availability of an adjoint model.

– Along the years the forward GEOS GCM has gone from FV to FV3, and asneeded the adjoint model gone from AD-FV to AD-FV3.

– Linearized physics has gone from simple diffusion and vertical drag to moreelaborate accountability of convection.

Ensemble DA opens the door for bypassing the Adjoint Model:

In a dual-analysis system (Var & Ens) the possibility exists for basing FSOI fullyon the ensemble (e.g., NCEP-approach, Ota et al. – see D. Groff’s poster) - thishas its disadvantages.

Alternatively, Var-DA can be adapted to make use of ensemble in replacing theadjoint model with forecast of ensemble perturbations.

This presentation provides preliminary evaluation of the two possibilities above.




Closing Remarks

Variational- and Ensemble-based FSOIVariational-Ensemble (Adjoint-free) FSOI

Forecast Error and its Reduction due to Analysis

tatb tv

ea(tv)

eb(tv)

Dt

At least two ways of defining forecast error:

State-space: es(tv |t0) =< [xf (tv |t0)− xv (tv )]T T [xf (tv |t0)− xv (tv )] >

Observation-space: eo(tv |t0) =< [h(xf (tv |t0))− yo(tv )]T C [h(xf (tv |t0))− yo(tv )] >




Closing Remarks


State-Space Approach: Forecast Error Reduction due to Analysis

Error change can be studied as either infinitesimal change to the state-space errormeasure(s) e(tv |t0) due to changes in observations (initial conditions) or as adifference between errors from consecutive analysis times:

(∂e

∂yo

)= <

(∂x0

∂yo

)T (∂xf

∂x0

)T∂e

∂xf>

δe = e(tv |t0)− e(tv |t0 − δt) = ea − eb

The infinitesimal formulation allows for projection of the forecast error change to thespace of observations through expressions of the form:

δe ≈(∂e

∂yo

)=< dTKTg0 >

with d and K being the background residual vector and the analysis gain matrix, and gamounting to a forecast sensitivity vector whose approximation leads to all kinds offormula (e.g. Errico 2007, Tellus; Daescu & Todling 2009; MWR).




Closing Remarks


Variational FSOI

Second-order Approximation (Trapezoidalrule; Langland & Baker 2004; Tellus):

gA0 ≡1

2(MT

a Tea + MT

b Teb)

And in a system such as GSI, the calculationof δe can be done as in:

δe ≈< dTR−1Hg >

where g is derived from the GSI-hybridsolver as in:

(I + BHTR−1HB)z = BgA0g = Bz

for B = βcBc + βeBe .

Ensemble FSOI

In Ensemble systems, the gradient is definedwith respect to the ensemble mean

gE0 ≡1

2XfT

a T(ea + eb)

where Xfa ≡ Xf (tv |ta) is a matrix created

from the ensemble perturbation of forecastsissued from ta and valid at tv , and theover-bar represents ensemble average.

And in a system such as the EnSRF,calculation of δe amount to:

δe ≈1

2< dTR−1H (L•XaXfT ) T(ea+eb) >

where Xa ≡ Xa(ta) is a matrix formed fromensemble analysis perturbations (Ota et. al,2013, Tellus; Kalnay et al., 2012, Tellus).An argument has been made to have Labove as an advected form of the L used inthe forward ensemble analysis.




Closing Remarks


Variational-Ensemble FSOI (Adm-free Var-FSOI)

Imagine a Hybrid (Var-) GSI-like system is reduced to an EnVar system, such that B = Be , andthe impact expression becomes

δe ≈< dTR−1Hge >

where ge is derived from the GSI-hybrid solver as in:

(I + BeHTR−1HBe)z = Beg

A0

ge = Bez

with Be formed from the ensemble of background perturbations.Noticing that the ensemble background covariance allows for the following to be written

BegA0 = L • Xb(Xb)T gA0 ≈ L • XbX

Ta M

T (ta, tb)gA0

We can replace gA0 with gE0 (using central forecast errors) in the RHS term BegA0 to get

BegA0 ≈ Beg

E0 ≈

1

2L • XbX

Ta M

T (ta, tb)XfTa T(ea + eb) ← Note: ea(eb) replaces ea(eb)

=1

2L • XbX

fTb T(ea + eb)

which amounts to a simple change to the RHS of the minimization problem solved for calculationof observation impacts in the Var system. The above is somewhat analogous to Pellerin et al.(2016; WMO). One can argue that L should be different from that used in the LHS of the solver.




Closing Remarks

Configuration of Hybrid GEOS DASVariational FSOI: GMAO FP IllustrationFSOI vs EFSOI: Variational ApproachEns-Var vs Ens-Ana FSOI

Components of GEOS Atmospheric Assimilation System

Atmospheric GCM

Fully ESMF-compliant

Hydrostatic FV3 dynamical core

RAS-Bacmeister convective physics

Chou-Suarez radiation scheme

Koster et al. catchment land-surface model

Lock et al. turbulence physics

Interactive ozone

Interactive GOCART aerosols

OSTIA-prescribed SST

Aerosol Analysis

Central DAS: PSAS-based AOD

Ensemble: members use central AOD ana

Met Analysis: GSI

4DEnVar

Nudged 4D-IAU

4D TLNMC balance

JCSDA CRTM

BiCG minimization

Ensemble Met Analysis: SqKF

ESRL-NCEP Square-root KF

Full obs, but level ozone and precip

Remark: Above is referred to as Dual-Analysis Hybrid System.




Closing Remarks


HybridGSI GOESAGCM

EnsembleofGSI-BasedObservers

EnsembleofGEOSAGCM’s

EnsembleAnalysis(EnKF)

OBS

BKG

OBS En-OmB

CentralADAS

EnsembleADAS

En-BKG

En-BKG

BKG

En-IAU

IAU

BCs

ICs

BCs

En-ICs

En-BKG

Remark: Ensemble analyses are not re-centered around central (top) DAS analysis.




Closing Remarks


From GMAO Forward Processing, 24-hr FSOI: 10d vs 30d stats

The preliminary results shown in thispresentation are for 00 UTC analyses of a10-day period from 1-10 Dec 2016.

To illustrate that statistics of observationimpacts are rather robust even over suchshort time interval, the figure on the rightcompares observation impacts for 10 and30 days from the GMAO FP system:Hyb-4dEnVar using an Adjoint-basedcalculation for deriving 24-hr forecastsensitivities.

From these, it is rather clear that:1 differences are small2 ranking of instruments is preserved




Closing Remarks


Variational Approach




Closing Remarks


From Adm-based Hyb-4dEnVar to 4dEnVar to Ens-based 4dEnVar

ADM-based FSOI: Hyb-4dEnVar vs 4dEnVar 4dEnVar FSOI: ADM vs ENS

Lack of advection of localization scales in the RHS of the Var impact expression motivatesfollowing Pellerin et al. (2016; WMO) and evaluating 12-hr instead of 24-hr FSOI.Left: compares FSOI when backward Var changed from Hyb-4dEnVar to 4dEnVar.Right: compares 4dEnVar-FSOI when Ens-perturbations replace ADM in sensitivity calc.

Remark: 32-member ensemble perturbations seem rather reasonable replacement for ADM for12-hr sensitivity calculation in 4dEnVar context.




Closing Remarks


4dEnVar Backward Approach ADM vs Ens-Pert Fcst Sensitivities: Ob Counts

For consistency, we calculateAdjoint-based impacts for a changedAdjoint analysis integration where theclimatological term is shut off, thusconverting the backward analysis into a4dEnVar instead its forward counterpartHybrid-4dEnVar .

The two approaches treat theobservations in exactly the same way, andfully consistent with how the forward(Hyb-4dEnVar) solver treats them.

Replacing the Adjoint Model withEnsemble perturbations to estimateforecast sensitivities leaves the analysissolver untouched wrt each other.

The figure on the right shows observationcounts between the Adjoint and Ensembletechniques, for backward integrations of4dEnVar, covering a 10-day period.




Closing Remarks


4dEnVar Backward ADM vs Ens-Pert Fcst Sensitivities: Ob Impacts

Overall, impacts don’t seem to changemuch and are largely comparable whenADM is replaced with Ensembleperturbations.

Closer look reveals satellite winds(GeoWind), MHS, some IR and nearsurface observations to have larger impactwhen Ens-Perts are used compared towhen ADM is used to estimate forecastsensitivity.

The above seems to be consistent withthe fact that the simply parameterizedadjoint physics is expected tomis-represent water and near surface fieldsas compared to the full GCM.




Closing Remarks


Hyb-4dEnVar vs 4dEnVar Adm and Ens-Perts: Observation Impacts

The figure on the right adds to theprevious figure bars for the default(present FP settings) using the fullAdjoint-based Hyb-4dEnVar backwardanalysis for FSOI.

This illustrates the changes we’d seeif/when replacing the current ADM-basedHyb-4dEnVar-derived FSOI with aEns-Perts-based 4dEnVar-derived FSOI.

As seen earlier, differences are largelyobserved for those observations that arecloud/water affected and near-surfaceobservations.




Closing Remarks


Ens-Var vs Ens-Ana FSOI in a Dual-Analysis Hybrid System:EFSOI from Central DA vs that from En-DA




Closing Remarks


From Central Forecasts From Ensemble Forecasts

Though error reductions are similar between central and ensemble forecasts, they areslightly smaller in ensemble; latter plays rather indirect role in aiding the former.

More importantly, though observation operators in GSI-EnSRF hybrid systems are shared,GSI and EnSRF treat observations rather differently.

Consequence: Observation impacts derived from ensemble are bound to differ from their hybridanalysis counterpart.




Closing Remarks


En-Var vs En-SRF FSOI: Observation Impacts

With exception of satellite winds(GeoWind), the ranking of impactsobtained with En-Var and En-SRF arerather similar.

But EnF and Var systems are different:

Ensemble mean forecasts are unrelated tocentral forecast.

And more importantly, ensemble analyseshandle observations largely differently ascompared to central hybrid analysis.




Closing Remarks


FSOI vs EFSOI: Observation Counts

The difference in treatment of observationbetween ensemble and central analyses isevident in the observation count.

The GSI and EnSRF solvers haveconsiderably different convergence criteria,with the latter being dependent on theassimilated observations due to its serialprocessing.

Even with the ideal DFS-based criterium(chosen here), the EnSRF ignores a verylarge percentage of the observations.




Closing Remarks

Learning from FSOI0-hr Observation Impacts

Criticism to FSOI

Well known:Reliance on norm that has no connections to the DA problemReliance on validation that is expected to be uncorrelated with initial analysisReliance on validity of TL approximation

In addition, when calculated in ensemble DA:need to rely on artificial advection of localization scales

In addition, in dual hybrid DA system:

EFSOI from ensemble analysis is inconsistent with central analysis:– forecasts from ensemble are unrelated to central forecasts– treatment of observation in solver not always one-to-one

We can learn the value of observation simply from background and analysis residuals

DFS gives good measure of how observations contribute to DAAnalogous to DFS is the so-called 0-hr Impact. This is an observation-space measure thatuses R−1 as the norm and amounts to simply evaluation of the cost before and after theanalysis, that is,

δey = < [h(xa)− yo ]T R−1 [h(xa)− yo ] > − < [h(xb)− yo ]T R−1 [h(xb)− yo ] >

= < Ja > − < Jb >

a completely free byproduct of any DA system (see Todling 2013; MWR).




Closing Remarks


AMSU-A Fractional Impacts: 0-hr vs 12-hr Ob Impacts

N-hr FSOI energy-based norm provides

skewed view of impacts, namely:

Overestimation in troposphereUnderestimation in stratosphere

Figure on the right shows 0-hr and 12-hrfractional impact for AMSU-A channels.Unlike 12-hr EFSOI, the 0-hr results showcontribution of all channels - not onlythose in the envelop of the energy norm.




Closing Remarks


0-hr impact corroborate expectation from simple theoretical argument that about 65% ofthe observations contribute to improve the analysis (see Ehrendorfer 2007; Meteor. Zeits.).

There is largely agreement between (E)FSOI and 0-hr impact, though surprising differencesmay arise, and latter is trivial to calculate.




Closing Remarks

Closing Remarks:

In a dual hybrid DA system, when a (low resolution) ensemble analysis filter isused to provide flow dependence to a (high resolution) hybrid analysis, certainconfigurations of the ensemble filter might discourage assessing observationimpact using the EFSOI based on the EnDA part of system.

The above seems to be the case for current GSI-EnSRF-based systems.

It has been shown here that it is possible to use an EnVar-based FSOI approachthat provides reliable assessment of the observations even when the forwardanalysis is hybrid.

It has also been show here that use of an ensemble seems to provide for betterrepresentation of the forecast error reduction than the adjoint model.

It remains to be seen whether EnVar-based FSOI is capable of representingforecast error reduction at long (larger than 12-hr) forecast lead-times; thusproviding reliable assessment of observation impact.

It is of the opinion of the first author here that an assessment of the use ofobservations in DA can be obtained directly from background and analysisresiduals: no need for adjoint nor ensemble backward analysis.


preliminary results comparing adjoint- and … recap of fsoi basics preliminary results general...

Documents