introduction to data assimilation - esa earth observation ... · esa-esrin, frascati, rome, italy 1...

ESA-ESRIN, Frascati, Rome, Italy

18th – 29th August 20031

D A R C

Introduction to Data Assimilation

Alan O’Neill

Data Assimilation Research CentreUniversity of Reading

D A R C

What is data assimilation?

Data assimilation is the technique whereby observational data are combined with output from a numerical model to produce an optimal estimate of the evolving state of the system.

D A R C



D A R C

Why We Need Data Assimilation

• range of observations• range of techniques• different errors• data gaps• quantities not measured• quantities linked

D A R CD A R C



D A R CD A R C

D A R C

Some Uses of Data Assimilation

• Operational weather and ocean forecasting

• Seasonal weather forecasting• Land-surface process• Global climate datasets• Planning satellite measurements• Evaluation of models and

observationsD A R C



D A R C

What We Want To Know

c

s

x

)(

)(

t

t atmos. state vector

surface fluxes

model parameters

)),(),(()( csxX ttt =

D A R C

What We Also Want To Know

Errors in models

Errors in observations

What observations to make



D A R C

Numerical ModelDAS

DATA ASSIMILATION SYSTEM

O

Data Cache

A

A

B

F

model

observations

Error Statistics

D A R C

The Data Assimilation Process

observations forecasts

estimates of state & parameters

compare reject adjust

errors in obs. & forecasts



D A R C

Retrievals and Assimilation

• Problems with retrievals– a priori state and poorly known errors

• Problems with assimilation of radiance– systematic errors and cloud clearing– expensive (multi-channels)

• Need effective interface– Information content with error analysis

D A R C

D A R C

Statistical Approach to Data Assimilation



D A R CD A R C

D A R C

21~ xxx βα +=

Minimum Variance

Combination of Data

Unbiased, Uncorrelated Errors

2

1= σ)Var( 1x 2

2= σ)Var( 2x

1=+ βα



D A R C

Minimum Variance

Combination of Data


2

2

2

1

2 )−+= σασα 21()~Var(x

0)1(22)~Var( 22

21 =−−=

∂

∂σαασ

αx

βα ,⇒

D A R C

111~ 21

−

2

2

2

1

2

2

2

1

+

+=

σσσσ

xxx

),min()~var( 22

21≤ σσx

Minimum Variance

Combination of Data


Best Linear Unbiased Estimate



D A R C

Variational Method

x~

22

21

−+

−=

σσ

22

21 )()(

)(xxxx

xJ

)(xJ

xx~

at )(min xJ

D A R C

Maximum Likelihood Estimate

• Obtain or assume probability distributions for the errors

• The best estimate of the state is chosen to have the greatest probability, or maximum likelihood

• If errors normally distributed,unbiased and uncorrelated, then states estimated by minimum variance and maximum likelihood are the same



D A R C

Multivariate Case

=

nx

x

x

t 2

1

)(x vector state

my

y

y

2

1

vector nobservatio

D A R C

Variance becomes Covariance Matrix

• Errors in xi are often correlated– spatial structure in flow– dynamical or chemical relationships

• Variance for scalar case becomes Covariance Matrix for vector case COV

• Diagonal elements are the variances of xi

• Off-diagonal elements are covariances between xi and xj

• Observation of xi affects estimate of xj



D A R C

Estimating Covariance Matrix for Observations, O

• O usually quite simple: – diagonal or – for nadir-sounding satellites, non-zero

values between points in vertical only

• Calibration against independent measurements

D A R C

Estimating Covariance Matrix for Model, B

• Use simple analytical functions constructed experimentally, e.g.

• Run ensemble of forecasts from slightly different conditions and analyse error growth.

ji xx and between distance

horizontal the is where ij

ijjiij

d

dB )exp(−= σσ



D A R C

Methods of Data Assimilation

• Optimal interpolation (or approx. to it)

• 3D variational method (3DVar)

• 4D variational method (4DVar)

• Kalman filter (with approximations)

D A R C

D A R C

Optimal Interpolation

))(( bba xyKxx H−+=

“analysis”

“background” (forecast)

observation

1OHBHBHK −+= )( TT

• linearity H H

• matrix inverse

• limited area

observation operator



D A R C

∆

∆

∆

∆

∆

∆∆

∆

)( bxy H−=∆ at obs. point

bx

data void

D A R C

X

t

observation

model trajectory



D A R C

Data Assimilation:an analogy

Driving with your eyes closed:

open eyes every 10 seconds and correct trajectory

D A R C

D A R C

Variational Data Assimilation

)(xJ

x

vary to minimise

x

)(xJ

ax



D A R C

))(())((

)()(

)(

1

1

xyOxy

xxBxx

x

HH

J

T

bT

b

−−

+−−

=

−

−

Variational Data Assimilation

nonlinear operator assimilate y directly global analysis

D A R C

Choice of State Variables and Preconditioning

• Free to choose which variables to use to define state vector, x(t)

• We’d like to make B diagonal– may not know covariances very well – want to make the minimization of J more

efficient by “preconditioning”: transforming variables to make surfaces of constant J nearly spherical in state space



D A R C

x2

x1

Cost Function for Correlated Errors

D A R C

x2

x1

Cost Function for

Uncorrelated Errors



D A R C

x2

x1

Cost Function for Uncorrelated Errors

Scaled Variables

D A R C

4D Variational Data Assimilation

given X(to), the forecast is deterministic

vary X(to) for best fit to datato t

obs. & errors



D A R C

4D Variational Data Assimilation

• Advantages– consistent with the governing eqs.

– implicit links between variables

• Disadvantages– very expensive– model is strong constraint

D A R C

Problem

model error covariances



D A R C

Kalman Filter(expensive)

Use model equations to propagate B forward in time.

B B(t) KAL

Analysis step as in OI

D A R C

What are the benefits of data assimilation?

• Quality control

• Combination of data• Errors in data and in model

• Filling in data poor regions

• Designing observational systems• Maintaining consistency

• Estimating unobserved quantities

D A R C



D A R C

Some Applications of Data Assimilation

D A R C

Skill Measures: Observation Increment, (O-F)

• The difference between the forecast from the first guess, F, and the observations, O, also known as observed-minus-background differences or the innovation vector.

• This is probably the best measure of forecast skill.



D A R C

ECMWF

Ozonemonthly-mean analysis increment.

Sept. 1986

_

+

T

D A R C



D A R C

Best observations to make to characterize the chemical system

D A R C

Impact on NWP at the Met Office

Mar 99. 3D-Varand ATOVS

Jul 99. ATOVS over Siberia, sea-ice from SSM/I

Oct 99. ATOVS as radiances, SSM/I winds

May 00. Retune 3D-Var

Feb/Apr 01. 2nd satellites, ATOVS + SSM/I



D A R C

Impact of satellite data in NWP

D A R CD A R C

Future Advanced Infrared Sounders



D A R CD A R C

IASI vs HIRS

D A R CD A R C



D A R CD A R C

D A R CD A R C



D A R CD A R C

MERIS ocean colour

D A R C

Regional Scale: Regional Scale: WWalnut alnut GGulch (Monsoon 90)ulch (Monsoon 90)

Model

Model with 4DDA

Observation

Tombstone, AZ

0% 20%

Houser et al., 1998



D A R C

Land InitializationLand Initialization: : MotivationMotivation

• Knowledge of soil moisture has a greater impact on the predictability of summertime precipitation over land at mid-latitudes than Sea Surface Temperature (SST).

D A R C

Conclusions

• Data assimilation should be essential part of “ground-segment” of all satellite missions.

• Aim should be to provide all data in “near real time”.

• Need to find optimal ways to assimilate data – efficient interfaces.

• Challenges: errors, resolution, data.

D A R C

introduction to data assimilation - esa earth observation ... · esa-esrin, frascati, rome, italy 1...

Documents