1 progress on the optimal spectral sampling (oss) method jean-luc moncet aer, inc

1

Progress on the Optimal Spectral Sampling (OSS) Method

Jean-Luc Moncet

AER, Inc.

2Atmospheric and Environmental Research, Inc.

Topics

IntroductionOSS S/W Transition to JCSDAGeneralized training

Clear/cloudy trainingInversion issue

Treatment of multiple scatteringValidation against CHARTSApplication to AIRS

Trace gasesSummary/future work


Overview of the OSS approach

OSS method (Moncet et al. 2003, 2001) models the channel radiance as

Wavenumber ni (nodes) and weights wi are determined by fitting “exact” calculations (from line-by-line model) for globally representative set of atmospheres (training set)Radiance training is fast and provides mechanism for directly including slowly varying functions (e.g. Planck, surface emissivity) in the selection process

i

N

iii νRwdRR ;

1


m

ll m

u

p p d e

mi l lk P ,T

iw

Relationship between OSS and ESFT/correlated-k methods

Extension to multiple absorbers along inhomogeneous path (e.g. Armbruster and Fisher, 1996)

ESFT (Wiscombe and Evans, 1977) for single layer, single absorber case:

OSS solution:

Extension of ESFT to inhomogeneous atmospheres with multiple absorbers reduces the problem to a single (wavenumber) dimension and ensures that the solution is physical

(i,k) = (2,1) (2,2)

i = 2

i = 1

i = 3

(2,3) (2,4) (2,5)

Se

lec

ted

no

de

s

ikiw

k u uu e d e

,νi

iw

m

l l ll m

k P T u

p e

(2)22

21

Kk

k

w

3

1ii

i

R w R


Optimal Spectral Sampling (OSS) method

OSS absorption parameterization leads to fast and numerically accurate RT modeling:

OSS-based RT model can approach line-by-line calculations arbitrarily closely

Adjustable numerical accuracy: • Possibility of trade off between accuracy and speed

Unsupervised trainingNo empirical adjustment: cuts significantly on cost of testing approximations and validating model

Provides flexible handling of (variable) trace molecular speciesDesigned to handle large number of variable trace species w/o any change to model – low impact on computational costSelection of variable trace gases at run time

Memory requirements do not change whether we are dealing with one or more instruments

Execution speed primarily driven by total spectral coverage and maximum spectral resolution (not by number of instruments)

Leads to accurate handling of multiple scattering (cloudy radiance assimilation)

6

OSS S/W Transition to JCSDA


OSS delivery status

OPTRAN/OSS intercomparisonAIRS, GOES, HIRS, AMSU, SSMISResults presented last year indicated significant speed/accuracy advantage of OSSLarger memory requirements (single instrument)

Supported transition of OSS molecular absorption parameterization into CRTM

Dynamic array allocation implementedRevised/adapted interface parametersDevelop pre-processing S/W to merge multiple instruments into single file and remove redundant nodes and reformat OSS coefficient filesOSS files delivered for AMSU, GOES, HIRS, SSMIS and AIRS

First CRTM/OSS version integrated at NOAA


OSS (IR) Parameter Generation S/W diagram


OSS Parameter Generation Delivery Status

Revised/adapted and delivered OSS generation S/W

Microwave (full code) with Rosenkranz spectroscopyInfrared:

Part 2 – localized training Part 1 (partial delivery)

Other items supplied: LBLRTM radiances and optical depth files for 0.01 cm-1 boxcars covering the spectral range encompassing current operational instruments channels for use with Selection Part 2

Delivered generation S/W successfully ran at NOAA

10

Generalized Training for High Spectral Resolution IR Sounders


Generalized training

Goal: Speed up RT calculations/radiance inversion for high resolution IR sounders (e.g. AIRS, IASI, CrIS)

Case considered: Complete channel set (no information loss)


Localized versus non-localized trainingLocalized training (current) operates on individual channels, one at a time – node redundancy due to overlapping ILS

AIRS (2378 channels):Average # nodes per channel: ~9 nodes/channelTotal # of nodes/# of channel (i.e. no redundancy) = 1.9

Non-localized training operates on groups of N channels (up to full channel set)

Exploits node-to-node correlation to minimize total number of nodes across a spectral domain (regression!!!)

Results in significant increase in number of points in any given channel increases

Speed gain varies with surface complexity and differ for clear (or cloud cleared) and cloudy radiances

Gain higher for ocean/highly emissive land surfaces (~flat emissivity spectrum)Minimum gain ~2 for AIRS when considering no correlation on scales larger than 20 cm-1

Nominal accuracy = 0.05K


Performance example (AIRS)

Localized training (0.05K accuracy):

~2nodes /channel~5000 monochromatic calculations for full AIRS channel set

Generalized training:

~0.1 node/channelReduces number of monochromatic calculations to ~250

Speed gain in RT computation ~ 20 compared to localized training for AIRS (ocean/clear)


Examples of error spatial distribution


Non-localized cloudy training

Must include slowly varying cloud/aerosol optical properties in training

Over wide bands: training can be done by using a database of cloud/aerosol optical propertiesMore general training obtained by breaking spectrum in intervals of the order of 10 cm-1 in width (impact of variations in cloud/aerosol properties on radiances is quasi-linear) and by performing independent training for each interval

lower computational gain but increased robustness

Direct cloudy radiance training not recommended !Clouds tend to mask molecular structure which makes training easierIf “recipe” for mixture of clear and cloudy atmospheres in direct training not right: clear-sky performance degrades


Robust, physical approach for including slowly varying functions (e.g. cloud optical properties, surface emissivity) into OSS formalism

Cloudy training preserves clear-sky solution

k = 1 2 3 4 5

1 2 1 21 iki ik i ik i i i i i i

i k ii

R w a R a R w R w w R

Alternate two-step training preserves clear-sky solution

First step: normal clear-sky (transmittance/radiance) training to model molecular absorptionSecond step: duplicate + redistribute nodes across spectral domain and recompute weights to incorporate slowly varying functions into the model

Single/multi-channel cloudy training over wide spectral domains

1 21cld cld cldi k ik i ik iR a R a R

2cldiR

1cldiR


Alternatives (application dependent): A. PC:

Generalized training operates on channel radiances or PC’s Use of PC’s reduces first dimension of A-matrix Further speed gain in inversion

B. Operate directly in node space

More robust• No risk of filtering out important spectral features• Channels are inspected individually after training and local nodes are added where

needed to improve performanceAvoids Jacobians transformation all together and reduce K-matrix size (inversion speed up)

• for AIRS: 2378 channels -> 250 nodes

InversionVariational retrieval methods:

Average channel uses ~150 nodes (instead of ~9 for localized training)Mapping Jacobians from node to channel space partially offsets speed gain

ˆm m m my = Ay y = Hy

1 - ,

where,

and

T -1 -1 T -1n ε n x n ε nK S K + S K S + K

y = Ay

K = AK

mn n nx y y x


Clear/ocean testExample of retrieval performance (AIRS) – constant noise

Channel space retrieval

Node space retrieval

Little impact on temperature and water vapor retrieval performanceOverall speed gain (RT model + inversion) ~15Need strategy for handling input dependent noise (application dependent)

SW band noise most sensitive to scene temperatureCrude classification (or no classification) may be sufficient in the clear skyCloud clearing noise amplification (?)

19

Application to Scattering Atmospheres


CHARTS (Moncet and Clough, 1997): Fast adding-doubling scheme for use with LBLRTM

Uses tables of layer reflection/transmittance as a function of total absorption computed at run time

Validation against measurements from Rotating Shadowband Spectroradiometer (RSS) spectra at the ARM/SGP site

OSS/CHARTS Comparison


OSSSCAT: Single wavelength version of CHARTS (no spectral interpolation)

Cloudy validation:Molecular absorption from 740-900 cm-1 domain1cm-1 boxcars, thermal onlyCloud extinction OD range: 0-100

Example:780-860 cm-1

Low cloud case (925-825 mb)

OSS/CHARTS Comparison (2)


Same as previousHigh cloud case (300-200 mb)

Clear sky training adequate in thermal regime

Refinement in training needed for thick clouds (OD > 50) when SSA approaches 1 and high scan angles

OSS/CHARTS Comparison (3)


MADA** optical properties (tropical cirrus)

**References: (Mitchell, 2000, 2002)


Application to AIRS

Single FOV 1DVAR retrieval

Atmosphere/SST from NCEP/GDASAdjusted parameters:

Cloud top/thicknessIce particles effective diameter (Deff)IWPEffective temperature

MODIS 1st guessAER/SERCAA cloud algorithms

RTM:OSSSCAT (100 layers)4-streams

GOES imagery


MODIS retrieved cloud product


Calculated vs. measured cloudy AIRS spectra

AIRS (896 cm-1) brightness temperature

27

Trace Gases


Trace gases

RT model designed to handle any number of variable trace speciesAdding a new variable species requires no change in OSS parameterization

No change in RTM requiredOnly need to include variability in training (number of nodes may increase as a result)

# of variable trace gases and molecule type specified on node-by-node basis (set by the user at run time)

Average number of trace gases absorbing at any given frequency << total number of absorbing species in the atmosphereComputationally efficient and minimizes memory requirementsInexpensive Jacobian computation:

0,

mlm

l abs l

y yk

u


Example of trace gas profiles

CO2, CH4, N2O and CO profiles

from combined

troposphere/stratosphere

model from NASA Global

Modeling Initiative (GMI)**

Geographically/seasonally

matched to existing

temperature water vapor

training set

** Data provided by Susan Strahan, Jose Rodriguez, and Bryan Duncan


Future Projections

Account for the secular increase in CO2,

CH4, and N2O, in training (avoids having

to periodically retrain the model) by running model into future years

Alternative approach:

scale the current profiles by a constant factor equal to take into account projected increase in surface concentration

May require height-dependent scaling based on the age of air (small effect)

CO projections are highly model-dependent (since CO is more affected by chemistry) and harder to quantify (e.g., biomass burning). Current variability is probably representative of the near future.

Projections of future surface concentrations from IPCC (2001)


Summary/future workTop priorities:

Continue supporting OSS integration in CRTM frameworkProvide support for efficiency improvement

Handling of “slow functions”Handling of variable input pressure gridAdapt RT model structure

Complete coefficient generation (localized training) S/W delivery and provide required support

Generalized (non-localized) trainingProvided brackets for potential gain in speed/memory requirements

Gain depends of surface type and presence of clouds/aerosolsOngoing testing in AER 1D-VAR system with AIRSStarted testing with NPOESS/CrIS

Cloudy conditionsCompleting preliminary validation of OSS in cloudy environment (infrared)

Applied to AIRS cloud property retrievalImprove speed (timing dominated by RT solution):

Generalized trainingFast parameterization of scattering effects (already available for EO imaging instruments)

Started validation effort in microwave (NPOESS/CMIS)

1 progress on the optimal spectral sampling (oss) method jean-luc moncet aer, inc

Documents

oss solution

oss approach oss method

environmental research

ossbased rt model

jcsda slide

noaa slide

multiple instruments

selection process slide