Slide 1

Model-data fusion for the coupled carbon-water system Cathy Trudinger, Michael Raupach, Peter Briggs CSIRO Marine and Atmospheric Research, Australia and Peter Rayner LSCE, France Email: cathy.trudinger@csiro.au

Slide 2

Outline Model-data fusion (= data assimilation + parameter estimation) Model-data fusion (= data assimilation + parameter estimation) Parameter estimation with the Kalman filter Parameter estimation with the Kalman filter Australian Water Availability Project Australian Water Availability Project OptIC project Optimisation Intercomparison OptIC project Optimisation Intercomparison

Slide 3

Model-data fusion Model: - Process representation - Subjective, incomplete - Capable of interpolation & forecast Observations: - Real world representation - Incomplete, patchy - No forecast capability Fusion: Optimal combination (involves model-obs mismatch & strategy to minimise) Analysis: - Best of both worlds - Identify model weaknesses - Forecast capability - Confidence limits

Slide 4

Choices in model-data fusion Target variables what model quantities to vary to match observations e.g. initial conditions, model parameters, time-varying model quantities, forcing Target variables what model quantities to vary to match observations e.g. initial conditions, model parameters, time-varying model quantities, forcing Cost function measure of misfit between observations and corresponding model quantities e.g. J(targets) = (H(targets) - obs) 2 + (targets - priors) 2 Cost function measure of misfit between observations and corresponding model quantities e.g. J(targets) = (H(targets) - obs) 2 + (targets - priors) 2 Fusion method - search strategy Fusion method - search strategy Batch (non-sequential) e.g. down-gradient, global search Batch (non-sequential) e.g. down-gradient, global search Sequential e.g. Kalman filter Sequential e.g. Kalman filter Approach and issues will differ to some extent between disciplines e.g. numerical weather prediction vs terrestrial carbon cycle

Slide 5

The Ensemble Kalman filter Ensemble Kalman filter (EnKF) sequential method that uses Monte Carlo techniques; error statistics are represented using an ensemble of model states. Ensemble Kalman filter (EnKF) sequential method that uses Monte Carlo techniques; error statistics are represented using an ensemble of model states. Initial ensemble Update using measurement t0t0 t1t1 t2t2 Time: Two steps: Two steps: 1. Model used to predict from one time to next 2. Update using observation Model predicts

Slide 6

Augmented state vector to be estimated contains Augmented state vector to be estimated contains Time-dependent model variables Time-dependent model variables Time-independent model parameters Time-independent model parameters State vector estimate at any time is due to observations up to that time State vector estimate at any time is due to observations up to that time Parameter estimation with the Ensemble Kalman filter

Slide 7

Our component of Australian Water Availability project: develop a Hydrological and Terrestrial Biosphere Data Assimilation System for Australia OBSERVATIONS NDVI NDVI Monthly river flows Monthly river flows Weather: rainfall, solar radiation, temperature Weather: rainfall, solar radiation, temperatureMODEL Soil moisture Soil moisture Leaf carbon Leaf carbon Water fluxes Water fluxes Carbon fluxes Carbon fluxes MODEL-DATA FUSION Ensemble Kalman Filter Ensemble Kalman Filter Down-gradient method (LM) Down-gradient method (LM) Analysis of past, present and future water and carbon budgets Analysis of past, present and future water and carbon budgets Maps of soil moisture, vegetation growth Maps of soil moisture, vegetation growth Process understanding Process understanding Drought assessments, national water balance Drought assessments, national water balance PRIOR INFORMATION Initial parameter estimates Initial parameter estimates Soil, vegetation types Soil, vegetation types

Slide 8

AWAP- Dynamic Model and Observation Model State variables (x) and dynamic model State variables (x) and dynamic model Dynamic model is of general form dx/dt = F (x, u, p) Dynamic model is of general form dx/dt = F (x, u, p) All fluxes (F) are functions F (x, u, p) = F (state vector, met forcing, params) All fluxes (F) are functions F (x, u, p) = F (state vector, met forcing, params) Governing equations for state vector x = (W, C L ): Governing equations for state vector x = (W, C L ): Soil water W: Leaf carbon C L : Observations (z) and observation model Observations (z) and observation model NDVI = func(C L ) NDVI = func(C L ) Catchment discharge = average of F WR + F WD [- extraction - river loss] Catchment discharge = average of F WR + F WD [- extraction - river loss] State vector in EnKF: x = [W, C L, NDVI, Dis, params] State vector in EnKF: x = [W, C L, NDVI, Dis, params] Timestep = 1 day Spatial resolution = 5x5 km

Slide 9

Southern Murray Darling Basin, Australia: "unimpaired" gauged catchments

Slide 10

JFMAMJJASOND 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 00 01 02 03 04 05 Murrumbidgee Relative Soil Moisture (0 to 1) (Forward run with priors, no assimilation)

Slide 11

Predicted and observed discharge 11 unimpaired catchments in Murrumbidgee basin 25-year time series: Jan 1981 to December 2005 (Forward run with priors, no assimilation)

Slide 12

Model-data synthesis approach: - State and parameter estimation with the EnKF - Assimilate NDVI and monthly catchment discharge Why Kalman filter? - Can account for model error (stochastic component) - Consistent statistics (uncertainty analysis) - Forecast capability (with uncertainty) Issues: - Time-averaged observations in EnKF (e.g. monthly catchment discharge) - Specifying statistical model (model and observation errors) - KF (sequential) vs batch parameter estimation methods? (using Levenberg-Marquardt method; also OptIC project)

Slide 13

Estimated parameters Monthly mean discharge/runoff Preliminary results: Adelong Creek Blue = Ensemble Kalman filter (sequential) Red = Levenberg-Marquardt (PEST) (batch)

Slide 14

OptIC project Optimisation method intercomparison International intercomparison of parameter estimation methods in biogeochemistry International intercomparison of parameter estimation methods in biogeochemistry Simple test model, noisy pseudo-data Simple test model, noisy pseudo-data 9 participants submitted results 9 participants submitted results Methods used: Methods used: Down-gradient (Levenberg-Marquardt, adjoint), Down-gradient (Levenberg-Marquardt, adjoint), Sequential (extended Kalman filter, ensemble Kalman filter) Sequential (extended Kalman filter, ensemble Kalman filter) Global search (Metropolis, Metropolis MCMC, Metropolis- Hastings MCMC). Global search (Metropolis, Metropolis MCMC, Metropolis- Hastings MCMC).

Slide 15

where F(t) forcing (log-Markovian i.e. log of forcing is Markovian) x 1 fast store x 2 slow store p 1, p 2 scales for effect of x 1 and x 2 limitation of production k 1, k 2 decay rates for pools s 0 seed production (constant value to prevent collapse) OptIC model Estimate parameters p 1, p 2, k 1, k 2

Slide 16

Noisy pseudo-observations T1: Gaussian (G) T4: Gaussian but noise in x 2 correlated with noise in x 1 (GC) T2: Log-normal (L) T6: Gaussian with 99% of x 2 data missing (GM) T3: Gaussian + temporally correlated (Markov) (GT) T5: Gaussian + drifts (GD)

Slide 17

Estimates divided by true parameters p1 p2 k1 k2

Slide 18

Cost function Some participants used cost functions with weights, w i (t), that depended on each noisy observation z i (t)

Slide 19

CodeMethodWeights LM1 Monte Carlo then Levenberg-Marquardt f(z i (t)) LM1Rob As LM1, but ignore 2% highest summands in cost fn f(z i (t)) LM2Levenberg-Marquardt0.01 LM3Levenberg-Marquardt Adj1 Down-gradient search using model adjoint 1.0 Adj2 sd(x) EKF Extended Kalman filter (with parameters in state vector) sd(resids) EnKF Ensemble Kalman filter (with parameters in state vector) sd(resids) MetMetropolissd(resids) MetRob As Met but absolute deviations not least squares sd(resids) MetMCMC Metropolis Markov Chain Monte Carlo 1.0 MetMCMCq As MetMCMC but quadratic weights f(z i 2 (t)) MH_MCMC Metropolis-Hastings Markov Chain Monte Carlo 1.0 Down-gradient Global-search KF w i (t) = f(z i (t)) less successful than constant weights

Slide 20

Choice of cost function Evans (2003) review of parameter estimation in biogeochemical models - it was hard to find two groups of workers who made the same choice for the form of the misfit function, with most of the differences being in the form of the weights. Evans (2003) review of parameter estimation in biogeochemical models - it was hard to find two groups of workers who made the same choice for the form of the misfit function, with most of the differences being in the form of the weights. Evans (2003) and the OptIC project emphasise that the choice of cost function matters, and should be made deliberately not by accident or default. Evans (2003) and the OptIC project emphasise that the choice of cost function matters, and should be made deliberately not by accident or default. (Evans 2003, J. Marine Systems)

Slide 21

Optic project results Choice of cost function had large impact on results Choice of cost function had large impact on results Most troublesome noise types:- temporally correlated noise Most troublesome noise types:- temporally correlated noise The Kalman filter did as well as the batch methods The Kalman filter did as well as the batch methods For more information on OptIc: http://www.globalcarbonproject.org/ACTIVITIES/OptIC.htm For more information on OptIc: http://www.globalcarbonproject.org/ACTIVITIES/OptIC.htm

Slide 22

Thank you!