r. reichle 1 * and c. draper 1,2 with contributions from:

R. Reichle1* and C. Draper1,2

With contributions from: G. De Lannoy, R. de Jeu, Q. Liu, V. Naeimi, R. Parinussa, W. Wagner

AMSR-E Technical Interchange MeetingOxnard, CA

Sep 4-5, 2013

Soil moisture retrievals from AMSR-E and ASCAT:

Error estimates and data assimilation

1NASA/GSFC, Greenbelt, MD, USA2Universities Space Research Association, Columbia, MD, USA

*Email: Rolf.Reichle@nasa.gov

Root Mean Square Error (RMSE) needed • to validate against mission target accuracies, and • to specify observation error for data assimilation.

Difficulty: No recognized soil moisture truth at large scales.

Motivation

• Point-scale (e.g., SCAN): Representativity errors when comparing against satellite estimates.

• Footprint-scale (e.g., USDA ARS): Limited spatial coverage.

RMSE traditionally from validation against in situ observations:

Error estimates for soil moisture retrievalsTriple collocationError propagation

Soil moisture data assimilation

Outline

How can we estimate RMSE in satellite soil moisture retrievals?

Approach

Two methods:• Triple collocation• Error propagation through remote sensing retrieval algorithms

Two remotely sensed soil moisture datasets:• AMSR-E (X-band) Land Parameter Retrieval Model (de Jeu and Owe, 2003)• ASCAT C-band TUWien empirical change detection (Wagner et al., 1999)

Experiment details:• Jan 2007 - Oct 2011• North American domain

Triple Collocation (TC) is a method to estimate RMS errors.

TC requires three independent estimates.

NOTE:1.) TC cannot provide absolute RMS errors because it does not

estimate the bias.2.) For soil moisture, we found TC to work only for “anomalies”

from the seasonal cycle.3.) TC is sensitive to the climatology of the chosen reference

data set.

Triple Collocation

Triple Collocation

Surface soil moisture datasets: (converted to anomalies from the mean seasonal cycle)• θ(A): ASCAT• θ(L): AMSR-E LPRM• θ(C): Catchment model

Additive random error model (θ = truth, < ϵ(X) >= 0)θ(A) = α( θ + ϵ(A))θ(L) = λ(θ + ϵ(L))θ(C) = γ(θ + ϵ(C))

α, λ, γ are calibration constants (rescale to account for systematic differences in variability between datasets)

Stoffelen (1998), Scipal et al. (2008), Miralles et al. (2010)

Triple Collocation

Set θ(A) as the reference dataset (α=1) subscript “A” for estimated calibration constants and estimated errors.

Solve for calibration:• λA ≈ < θ(L) θ(C) > / < θ(A) θ(C) >

• γA ≈ < θ(L) θ(C) > / < θ(A) θ(L) >

Solve for ϵ2 = (RMSETC)2:• ϵA

2(A) ≈ < (θ(A) – θ(L)/λA) · (θ(A) – θ(C)/γA) >

• ϵA2(L) ≈ < (θ(L)/λA – θ(A) ) · (θ(L)/λA – θ(C)/γA) >

• ϵA2(C) ≈ < (θ(C)/γA – θ(A) ) · (θ(C)/γA – θ(L)/λA) >

Assumption: Errors are not correlated with each other, nor with the true state.

RMSE(X)2 = var(X) + var(true) – 2 R sqrt( var(X) var(true)) + bias2

Interpretation of large-scale soil moisture RMSE

• “Because vartrue differs from place to place […] a single global RMSE target may not be appropriate.” (Entekhabi et al., 2010).

• At the global scale, we do not know vartrue (or even the mean):

Information on agreement is in time series correlation coefficient R.

Comparing soil moisture data sets using TC requires rescaling them to match the climatology of a reference data set.

RMSETC estimates will depend on the climatology of the chosen reference data set.

Variability (anomalies)

ASCAT

AMSR-E/LPRM

GEOS-5

RMSE of ASCAT (anom)

[Ref dataset = ASCAT]

[Ref dataset = AMSR-E]

[Ref dataset = GEOS-5]

Triple Collocation

RMSE and Fractional RMSE (fRMSE)

Spatial differences in variability introduce nonlinearities: the ratio (and ranking!) of the domain-average RMSETC in m3m-3 depends on the selected reference:

Solution: Report errors as fractional RMSE:

fRMSE = RMSEX(X) / var(X)

Variability (anomalies)

ASCAT

AMSR-E/LPRM

GEOS-5

RMSE of ASCAT (anom)

[Ref dataset = ASCAT]

[Ref dataset = AMSR-E]

[Ref dataset = GEOS-5]

Fractional RMSE

ASCAT

AMSR-E/LPRM

Draper et al. 2013, RSE, in press

Estimated RMSE reflects variability of reference product.Use fractional RMSE instead.

Triple Collocation

Uncertainty estimates indicate where fRMSETC is reliable.

ASCAT fRMSETC AMSR-E fRMSETC

Width of 90% confidence interval for ASCAT fRMSETC

Width of 90% confidence interval for AMSR-E fRMSETC

Triple Collocation

Draper et al. 2013, RSE, in press

fRMSETC estimates can provide insights into relative skill.Consistent with expectations.

Triple Collocation

Draper et al. 2013, RSE, in press

• fRMSETC is reasonably insensitive to choice of data triplet.• Residual differences between fRMSETC estimates are related

to error correlations between data products.

Triple Collocation Assumptions

Draper et al. 2013, RSE, in press

In situ “errors” are not small! (scale mismatch, sensor issues,...)

Assumption: Errors not correlated across datasets and with the truth.Check by adding a 4th dataset (in situ).

Previous finding holds only if using anomalies from the mean seasonal cycle!

Triple Collocation Assumptions

Error estimates for soil moisture retrievalsTriple collocationError propagation

Soil moisture data assimilation

Outline

Error propagationPropagate expected errors in input observations and parameters through the soil moisture retrieval model:• ASCAT (Naeimi et al. 2007)• LPRM (Parinussa et al. 2011)

Used average of the error time series at each location. Assumed to represent errors in the soil moisture anomalies.

TC and EP yield reasonably consistent fRMSE patterns.

ASCAT fRMSETC AMSR-E fRMSETC

Triple Collocation (TC) and Error Propagation (EP)

Draper et al. 2013, RSE, in press

ASCAT fRMSEEP AMSR-E fRMSEEP

Triple Collocation

fRMSETC is consistent with:

• fRMSE from error propagation

• expectations (vegetation classes)

Draper et al. 2013, RSE, in press

Error estimates for soil moisture retrievalsTriple collocationError propagation

Soil moisture data assimilation

Outline

Improvements from data assimilation

Validated with in situ data

Anomalies ≡ mean seasonal cycle removed

Skill increases significantly through data assimilation. Similar improvements from AMSR-E/LPRM and ASCAT.

Metric: Anom. time series corr. coeff.

Draper et al. (2012), GRL, doi:10.1029/2011GL050655.

Improvements from data assimilation

Skill increases significantly through data assimilation. Similar improvements from AMSR-E/LPRM and ASCAT.

Draper et al. (2012), GRL, doi:10.1029/2011GL050655.

Consistent with similar skill levels for the AMSR-E and ASCAT retrievals themselves….…except for terrain with high topographic complexity (TC>10%).

Root zone soil moisture skill improvement from assimilation over model (ΔR)

As long as (obs skill − model skill) > −0.2, assimilation improved the model skill.

Draper et al. (2012), GRL, doi:10.1029/2011GL050655.

Reichle et al. (2008), GRL, doi:10.1029/2007GL031986.

Improvements from data assimilation

Synthetic experiment

Both triple collocation and error propagation can estimate spatial patterns in the fRMSE.• Good agreement with each other, and with expectations (based on

vegetation, known problems)• Triple collocation is believed to provide correct magnitude, and method is

robust (dependent on careful selection of data sets, use of anomalies from the seasonal mean)

Conclusions (1/4)

How should we evaluate remotely sensed soil moisture globally?

• Substantial spatial variation in RMSE and fRMSE across North America• Evaluation based on handful of in situ sites may not represent global

accuracy• Triple collocation or error propagation could provide a useful complement

• It is unclear how current RMSE target accuracies can be interpreted globally• Selection of reference determines magnitude of RMSE (and domain-

average RMSE has non-linear dependence on reference)• Uniform RMSE target over large domain not sensible• Need alternative metrics (fRMSE, or see Crow et al. (2010), Entekhabi et

al. (2010))

Conclusions (2/4)

Data assimilation:• Assimilation of soil moisture retrievals improves model estimates of surface and root zone soil moisture.

• AMSR-E and ASCAT retrievals have comparable skill and yield similar improvements after data assimilation.

• Improvements can be realized even when the retrievals are (somewhat) less skillful than the model estimates.

Conclusions (3/4)

How should we specify soil moisture observation error variances for data assimilation?• Usually specified as a

constant in the observation climatology.

• Uniform error variance not sensible, and resulting spatial patterns do not resemble other estimates.

• Use a uniform fRMSE, or spatial maps from triple collocation or error propagation (latter gives temporal variability).

Conclusions (4/4)

Thank you for your attention.

Questions?

References:Draper, C. S., R. H. Reichle, R. de Jeu, V. Naeimi, R. Parinussa, and W. Wagner (2013), Estimating root mean square errors in remotely sensed soil moisture over continental scale domains, Remote Sensing of Environment, in press.

Draper, C. S., R. H. Reichle, G. J. M. De Lannoy, and Q. Liu (2012), Assimilation of passive and active microwave soil moisture retrievals, Geophysical Research Letters, 39, L04401, doi:10.1029/2011GL050655.

Back-Up Slides