Grand Unified Algorithms

How to retrieve radiatively consistent profiles of clouds, precipitation and

aerosol from radar, lidar and radiometersRobin Hogan, University of ReadingThanks to Julien Delanoe, Nicola Pounder, Nicky

Chalmers, Howard Barker

…and evaluating models

Spaceborne radar, lidar and Spaceborne radar, lidar and radiometersradiometers

The A-Train– NASA– 700-km orbit– CloudSat 94-GHz radar (launch 2006)– Calipso 532/1064-nm depol. lidar– MODIS multi-wavelength radiometer– CERES broad-band radiometer– AMSR-E microwave radiometer

EarthCARE: launch 2012– ESA+JAXA– 400-km orbit: more

sensitive– 94-GHz Doppler radar– 355-nm HSRL/depol. lidar– Multispectral imager– Broad-band radiometer– Heart-warming name

EarthCare

2013201420152016201720182019

What do CloudSat and Calipso What do CloudSat and Calipso see?see?

Cloudsat radar

CALIPSO lidar

Target classificationInsectsAerosolRainSupercooled liquid cloudWarm liquid cloudIce and supercooled liquidIceClearNo ice/rain but possibly liquidGround

Delanoe and Hogan (2008, 2010)

• Radar: ~D6, detects whole profile, surface echo provides integral constraint

• Lidar: ~D2, more sensitive to thin cirrus and liquid but attenuated

• Radar-lidar ratio provides size D

How do we evaluate models?How do we evaluate models?• Traditional approach

– Compare retrieved cloud products to model variables

• New approach– Forward-model observations

and evaluate in obs. space– IceSat lidar: Chepfer et al.

(2007), Wilkinson et al. (2008)

– CloudSat: Bodas et al. (2008)– CloudSat & Calipso:

IPCC/COSP– Much easier!– Avoids contamination by a-

priori information– Avoids competition between

retrievals

– But these are the variables we want to know!

– Simple forward modeling can get right answer for wrong reasons, e.g. right Z with wrong IWC and re

– Can extract “hidden” info, e.g. re from radar-lidar synergy

– Can provide radiative verification of each retrieved profile

– But certainly more difficult!

OverviewOverview• Justification for and design of unified algorithm• Ice clouds

– Evaluation against CERES– Evaluation of ECMWF and Met Office models

• Liquid clouds– Fast multiple scattering model for exploitation of lidar signal– Extinction profile from multiple field-of-view lidar– Can we estimate liquid cloud base from the Calipso lidar?

• First results from unified algorithm applied to A-Train– Simultaneous ice, liquid and rain retrievals

• Outlook– 3D radiatively consistent scene retrieval

““Grand Unified Algorithm”Grand Unified Algorithm”• Combine all measurements available (radar, lidar, radiometers)

– Forms the observation vector y• Retrieve cloud, precipitation and aerosol properties simultaneously

– Ensures integral measurements can be used when affected by more than one species (e.g. radiances affected by ice and liquid clouds)

– Forms the state vector x• Variational approach

– This is the proper way to do it!– Use forward model H(x) to predict observations from state vector– Report solution error covariance matrix & averaging kernel

• Completely flexible– Applicable to ground-based, airborne and space-borne platforms

• Behaviour should tend towards existing two-instrument synergy algos– Radar+lidar for ice clouds: Donovan et al. (2001), Tinel et al. (2005)– CloudSat+MODIS for liquid clouds: Austin & Stephens (2001)– Calipso+MODIS for aerosol: Kaufman et al. (2003)– CloudSat surface return for rainfall: L’Ecuyer & Stephens (2002)

The cost functionThe cost function• The essence of the method is to find the state vector x that minimizes

a cost function:

TxxδxBδxδyRδy

x

T1T1T

1

2

211

12

2

12

2

2

1

2

1

2

1

22

1

2

1)(

2

1

n

iiiii

n

i bi

iim

i yi

ii xxxbxHy

J

Each observation yi is weighted by the inverse of

its error variance

The forward model H(x) predicts the observations from the state vector x

Some elements of x are constrained by a

prior estimate

This term can be used to penalize curvature in the retrieved profile

Unified Unified retrievalretrieval

Ingredients developedWork in progress

1. New ray of data: define state vector x

Use classification to specify variables describing each species at each gateIce: extinction coefficient , N0’, lidar extinction-to-backscatter ratio

Liquid: extinction coefficient and number concentrationRain: rain rate, drop diameter and melting iceAerosol: extinction coefficient, particle size and lidar ratio

2a. Radar model

Including surface return and multiple scattering

2b. Lidar model

Including HSRL channels and multiple scattering

2c. Radiance model

Solar and IR channels

3. Compare to observations

Check for convergence

4. Iteration method

Derive a new state vectorAdjoint of full forward modelQuasi-Newton or Gauss-Newton scheme

2. Forward model

Not converged

Converged

Proceed to next ray of data5. Calculate retrieval error

Error covariances and averaging kernel

Unified retrieval: Forward Unified retrieval: Forward modelmodel

• From state vector x to forward modelled observations H(x)...

Ice & snow Liquid cloud Rain Aerosol

Ice/radar

Liquid/radar

Rain/radar

Ice/lidar

Liquid/lidar

Rain/lidar

Aerosol/lidar

Ice/radiometer

Liquid/radiometer

Rain/radiometer

Aerosol/radiometer

Radar scattering profile

Lidar scattering profile

Radiometer scattering profile

Lookup tables to obtain profiles of extinction, scattering & backscatter coefficients, asymmetry factor

Sum the contributions from each constituent

x

Radar forward modelled obs

Lidar forward modelled obs

Radiometer fwd modelled obs

H(x)Radiative transfer models

Adjoint of radar model (vector)

Adjoint of lidar model (vector)

Adjoint of radiometer model

Gradient of cost function (vector)

xJ=HTR-1[y–H(x)]

Vector-matrix multiplications: around the same cost as the original forward

operations

Adjoint of radiative transfer models

yJ=R-1[y–H(x)]

Ice cloud: non-variational Ice cloud: non-variational retrievalretrieval

• Donovan et al. (2000) algorithm can only be applied where both lidar and radar have signal

Observations

State variables

Derived variables

Retrieval is accurate but not perfectly stable where lidar loses signal

Aircraft-simulated profiles with noise (from Hogan et al. 2006)

Delanoe and Hogan (2008)

Variational radar/lidar Variational radar/lidar retrievalretrieval

• Noise in lidar backscatter feeds through to retrieved extinction

Observations

State variables

Derived variables

Lidar noise matched by retrieval

Noise feeds through to other variables

Delanoe and Hogan (2008)

……add smoothness constraintadd smoothness constraint

• Smoothness constraint: add a term to cost function to penalize curvature in the solution (J’ = id2i/dz2)

Observations

State variables

Derived variables

Retrieval reverts to a-priori N0

Extinction and IWC too low in radar-only region

Delanoe and Hogan (2008)

……add a-priori error add a-priori error correlationcorrelation

• Use B (the a priori error covariance matrix) to smooth the N0 information in the vertical

Observations

State variables

Derived variables

Vertical correlation of error in N0

Extinction and IWC now more accurate

Delanoe and Hogan (2008)

Lidar observations

Radar observations

Visible extinction

Ice water content

Effective radius

Lidar forward model

Radar forward model

Example ice Example ice cloud cloud

retrievalsretrievalsDelanoe and Hogan (2010)

Evaluation using CERES TOA Evaluation using CERES TOA fluxesfluxes• Radar-lidar retrieved profiles containing only ice used with Edwards-

Slingo radiation code to predict CERES fluxes– Note: MODIS IR radiances can be used but aren’t used here

• Small biases but large random shortwave error: 3D effects?

Chalmers (2011)

ShortwaveBias 4 W m-2, RMSE 71 W m-2

LongwaveBias 0.3 W m-2, RMSE 14 W m-2

CERES versus a radar-only CERES versus a radar-only retrievalretrieval

• How does this compare with radar-only empirical IWC(Z, T) retrieval of Hogan et al. (2006) using effective radius parameterization from Kristjansson et al. (1999)?

Bias 10 W m-2

RMS 47 W m-2

ShortwaveBias 48 W m-2, RMSE 110 W m-2

LongwaveBias –10 W m-2, RMSE 47 W m-2

Chalmers (2011)

How important is lidar?How important is lidar?• Remove lidar-only pixels from radar-lidar retrieval• Change to fluxes is only ~5 W m-2 but lidar still acts to improve

retrieval in radar-lidar region of the cloud

ShortwaveBias –5 W m-2, RMSE 17 W m-2

LongwaveBias 4 W m-2, RMSE 9 W m-2

Chalmers (2011)

TOA fluxes don’t tell the TOA fluxes don’t tell the whole story...whole story...

• In terms of net atmospheric heating rate in the tropics, cloud radiative effect is underestimated by a factor of two above 12 km if clouds detected by lidar alone are ignored

A-Train A-Train versus versus

modelsmodels• Ice water

content• 14 July 2006• Half an orbit• 150°

longitude at equator

Delanoe et al. (2011)

• Both models lack high thin cirrus• ECMWF lacks high IWC values; using this work, ECMWF have

developed a new prognostic snow scheme that performs better• Met Office has too narrow a distribution of in-cloud IWC

Evaluation of gridbox-mean ice water Evaluation of gridbox-mean ice water contentcontent

In-cloud mean ice water In-cloud mean ice water contentcontent

Why you must compare the Why you must compare the full PDFfull PDF

• Consider Cloudnet evaluation of IWC in models– DWD model drastically underestimates mean IWC

3-7 km

– But PDF is within observational range in all but the highest bin!

– 10% of cloud volume contains 75% of the ice in observations

– But all parts of PDF can be radiatively significant– Moreover, top 10% of IWC retrievals are the least reliable

Illingworth et al. (2007)

Liquid cloudsLiquid clouds• Stratocumulus clouds are tricky!– Lidar beam is rapidly attenuated– Radar return usually dominated by drizzle (Fox & Illingworth 1997)

• Can we piece together their properties by forward modeling the following?– Multiply scattered lidar signal: information on optical depth and

extinction profile– Surface radar return: path integrated attenuation proportional to

liquid water path (Hawkness-Smith 2010)– Solar radiances: optical depth and droplet size / number

concentration• We need:

– Fast lidar forward model incorporating multiple scattering– Ability to use additional constraints, such as tendency for liquid

water content profile to be adiabatic, particularly near cloud base

CALIPSO CloudSat

Examples of multiple scattering

LITE lidar (<r, footprint~1 km)

CloudSat radar (>r)

StratocumulusStratocumulus

Intense thunderstorm

Surface echoApparent echo from below the surface!

Time-dependent 2-stream Time-dependent 2-stream approx.approx.• Describe diffuse flux in terms of outgoing stream I+ and incoming stream I–, and

numerically integrate the following coupled PDEs:

• These can be discretized quite simply in time and space (no implicit methods or matrix inversion required)

SII

r

I

t

I

c 211

1

SII

r

I

t

I

c 211

1

Time derivative Remove this and we have the time-independent two-stream approximation

Spatial derivative Transport of radiation from upstream

Loss by absorption or scatteringSome of lost radiation will enter the other stream

Gain by scattering Radiation scattered from the other stream

Source

Scattering from the quasi-direct beam into each of the streams

Hogan and Battaglia (2008)

Fast multiple scattering forward Fast multiple scattering forward modelmodel

CloudSat-like example

• New method uses the time-dependent two-stream approximation

• Agrees with Monte Carlo but ~107 times faster (~3 ms)

Hogan and Battaglia (2008)

CALIPSO-like example

Multiple field-of-view lidar Multiple field-of-view lidar retrievalretrieval

• To test multiple scattering model in a retrieval, and its adjoint, consider a multiple field-of-view lidar observing a liquid cloud

• Wide fields of view provide information deeper into the cloud

• The NASA airborne “THOR” lidar is an example with 8 fields of view

• Simple retrieval implemented with state vector consisting of profile of extinction coefficient

lidar

Cloud top

600 m100 m

10 m

Results for a sine profileResults for a sine profile• Simulated test

with 200-m sinusoidal structure in extinction

• With 1FOV, only retrieve first 2 optical depths

• With 3FOVs, retrieve structure down to 6 optical depths

• Beyond that the information is smeared out

Pounder et al. (2011)

• Averaging kernel area: what fraction of retrieval from obs rather than prior?• Averaging kernel width: how much has true profile been smeared out?

Calipso?Calipso?• Can we use this approach

with a lidar with only one field of view?

• Calipso has 90-m footprint• Simulations indicate that

there is measurable difference in apparent backscatter profile up to 20-30 optical depths (would be more for a larger field-of-view)

• Perhaps can’t retrieve full extinction profile, but at least the optical depth and the cloud boundaries?

Simulated profileSimulated profile• Adiabatic cloud retrieved by lidar

using smoothness constraint• Optical depth around 20• Because lidar ratio is well

constrained in liquid clouds, backscatter provides quite accurate extinction (and hence LWC) at cloud top

• Wide-angle multiple scattering provides some optical depth information

• But retrieved shape is wrong• Cloud base too low

One-sided gradient constraintOne-sided gradient constraint

• We have a good constraint on the gradient of LWC with height in stratocumulus: adiabatic profile, particularly near cloud base

• Add an extra term to the cost function to penalize deviations from gradient c:

• This term is only used when the LWC gradient is greater than c, so sub-adiabatic clouds can be retrieved

• Test with simulated lidar-only retrieval of liquid water cloud using unified algorithm, and including simulated instrument noise

Slingo et al. (1982)

2LWC

igrad c

dz

dJ

Gradient constraintGradient constraint• With one-sided gradient

constraint observed by backscatter-only lidar, much better retrieved shape

• Cloud base about right

Clipped profileClipped profile• Multiply scattered signal plus

gradient constraint enables more structured profiles to still be retrieved reasonably well

• Still have the problem of multiple liquid layers if the lower ones are undetected by the lidar

Unphysical profileUnphysical profile• Gradient constraint ensures no

super-adiabatic profiles are retrieved.

Optical depth from multiple Optical depth from multiple scattering lidarscattering lidar

• Total optical depth can be retrieved to ~30 optical depths with 3 fields of view

• Limit is closer to 3 for one narrow field-of-view lidar

• Useful optical depth information from one 100-m-footprint lidar (e.g. Calipso)!

• Why not launch multiple FOV lidar in space?

Pounder et al. (2011)

Unified algorithm: progressUnified algorithm: progress• Bringing the aspects of this talk together…• Done:

– Functioning algorithm framework exists– C++: object orientation allows code to be completely flexible:

observations can be added and removed without needing to keep track of indices to matrices, so same code can be applied to different observing systems

– Preliminary retrieval of ice, liquid, rain and aerosol– Adjoint of radar and lidar forward models with multiple scattering and

HSRL/Raman support– Interface to L-BFGS quasi-Newton algorithm in GNU Scientific Library

• In progress / future work:– Estimate and report error in solution and averaging kernel – Interface to radiance models– Test on a range of ground-based, airborne and spaceborne

instruments– Will produce the standard EarthCARE cloud & precip synergy products

Observations vs forward Observations vs forward modelsmodels

– Radar and lidar backscatter are successfully forward modelled (at final iteration) in most situations

– Can also forward model Doppler velocity (what EarthCARE would see)

• Radar reflectivity factor• Lidar backscatter

Three retrieved componentsThree retrieved components• Liquid water content

• Ice extinction coefficient

• Rain rate

Extension to three Extension to three dimensionsdimensions

• Synergistic retrievals under radar and lidar can be extended laterally using imager, then evaluated radiatively using broadband fluxes

• Part of proposed product chain for EarthCARE satelliteBarker et al. (2011)

A: aerosol not included

B: surface temperature error

OutlookOutlook• Evaluation of climate models in model space has distinct advantages

over comparisons in observation space– Radiatively validated and consistent estimates of atmospheric state– Can say not only in what way model clouds are wrong but what the

radiative consequence is– Forward-model errors affect both approaches

• A “Grand Unified Algorithm” enables all measurements to be combined to provide the optimum estimate of the atmospheric state– Difficult and plenty remains to be done (e.g. precipitation – to be

done with Pavlos Kolias)– Important to report errors (including those due to forward model

errors) and averaging kernel information– Hope to have a fully flexible and freely available code that can be

applied to many different platforms and accommodate new observations

Three years of CloudSat and Calipso ice retrievals: http://www.icare.univ-lille1.fr/projects/dardar/ (Google “dardar icare”)

Clouds in climate modelsClouds in climate models

14 global models (AMIP)

90N 80 60 40 20 0 -20 -40 -60 -80 90S

0.05

0.10

0.15

0.20

0.25

Latitude

Ver

tical

ly in

tegr

ated

clo

ud w

ater

(kg

m-2) But all

models tuned to give about the same top-of-atmosphere radiation

The properties of ice clouds

are particularly uncertain

• Via their interaction with solar and terrestrial radiation, clouds are one of the greatest sources of uncertainty in climate forecasts

• But cloud water content in models varies by a factor of 10• Need instrument with high vertical resolution…

Stephens et al. (2002)

Vertical structure of liquid water Vertical structure of liquid water contentcontent

• Cloudnet: several years of retrievals from 3 European ground-based sites• Observations in grey (with range indicating uncertainty)

• How do these models perform globally?

– ECMWF has far too great an occurrence of low LWC values

0-3 km

– Supercooled liquid water content from seven forecast models spans a factor of 20

Illingworth, Hogan et al. (2007)

CloudSat and Calipso CloudSat and Calipso sensitivitysensitivity

• In July 2006, cloud occurrence in the subzero troposphere was 13.3%

• The fraction observed by radar was 65.9%

• The fraction observed by lidar was 65.0%

• The fraction observed by both was 31.0%

Gauss-Newton method

• Requires the curvature 2J/x2

– A matrix– More expensive to calculate

• Faster convergence– Assume J is quadratic and

jump to the minimum• Limited to smaller retrieval

problems

J

x

x1

J/x2J/x2

Minimization methods - in 1DMinimization methods - in 1DQuasi-Newton method (e.g. L-BFGS)

• Rolling a ball down a hill– Intelligent choice of direction

in multi-dimensions helps convergence

• Requires the gradient J/x– A vector (efficient to store)– Efficient to calculate using

adjoint method• Used in data assimilation

J

x

x2x3x4x5x6x7x8

x1

J/x

x2x3

x4x5

and 2nd derivative (the Hessian matrix):

Gradient Descent methods

– Fast adjoint method to calculate xJ means don’t need to calculate Jacobian

– Disadvantage: more iterations needed since we don’t know curvature of J(x)

– Quasi-Newton method to get the search direction (e.g. L-BFGS used by ECMWF): builds up an approximate inverse Hessian A for improved convergence

– Scales well for large x– Poorer estimate of the error at the

end

Minimizing the cost functionMinimizing the cost function

Gradient of cost function (a vector)

Gauss-Newton method

– Rapid convergence (instant for linear problems)

– Get solution error covariance “for free” at the end

– Levenberg-Marquardt is a small modification to ensure convergence

– Need the Jacobian matrix H of every forward model: can be expensive for larger problems as forward model may need to be rerun with each element of the state vector perturbed

112 BHRHxTJ

axBaxxyRxy 11

2

1)()(

2

1 TT HHJ

axBxyRHx 11 )(HJ T

JJii xxxx

12

1 Jii xAxx 1

EarthCAREEarthCARE• The ESA/JAXA “EarthCARE”

satellite is designed with synergy in mind

• We are currently developing synergy algorithms for its instrument specification

EarthCARE lidarEarthCARE lidar• High Spectral Resolution

capability enables direct retrieval of extinction profile

• First part of a forward model is the scattering and fall-speed model– Same methods typically used for all radiometer and lidar channels– Radar and Doppler model uses another set of methods

Scattering modelsScattering models

Particle type Radar (3.2 mm) Radar Doppler Thermal IR, Solar, UVAerosol Aerosol not

detected by radarAerosol not detected by radar

Mie theory, Highwood refractive index

Liquid droplets Mie theory Beard (1976) Mie theoryRain drops T-matrix: Brandes

et al. (2002) shapes

Beard (1976) Mie theory

Ice cloud particles

T-matrix (Hogan et al. 2010)

Westbrook & Heymsfield

Baran (2004)

Graupel and hail Mie theory TBD Mie theoryMelting ice Wu & Wang

(1991)TBD Mie theory

Unified algorithm: state Unified algorithm: state variablesvariables

State variable Representation with height / constraint A-priori

Ice clouds and snow

Visible extinction coefficient One variable per pixel with smoothness constraint None

Number conc. parameter Cubic spline basis functions with vertical correlation Temperature dependent

Lidar extinction-to-backscatter ratio Cubic spline basis functions 20 sr

Riming factor Likely a single value per profile 1

Liquid clouds

Liquid water content One variable per pixel but with gradient constraint None

Droplet number concentration One value per liquid layer Temperature dependent

Rain

Rain rate Cubic spline basis functions with flatness constraint None

Normalized number conc. Nw One value per profile Dependent on whether from melting ice or coallescence

Melting-layer thickness scaling factor One value per profile 1

Aerosols

Extinction coefficient One variable per pixel with smoothness constraint None

Lidar extinction-to-backscatter ratio One value per aerosol layer identified Climatological type depending on region

Ice clouds follows Delanoe & Hogan (2008); Snow & riming in convective clouds needs to be added

Liquid clouds currently being tackled

Basic rain to be added shortly; Full representation later

Basic aerosols to be added shortly; Full representation via collaboration?

• Proposed list of retrieved variables held in the state vector x

Radiative transfer forward Radiative transfer forward modelsmodels

• Infrared radiances– Delanoe and Hogan (2008) model– Currently testing RTTOV (widely used, can do microwave, has

adjoint)• Solar radiances

– Currently testing LIDORT• Radar and lidar

– Simplest model is single scattering with attenuation: ’= exp(-2)– Problem from space is multiple scattering: contains extra

information on cloud properties (particularly optical depth) but no-one has previously been able to rigorously make use of data subject to pulse stretching

– Use combination of fast “Photon Variance-Covariance” method and “Time-Dependent Two-Stream” methods

– Adjoints for these models recently coded– Forward model for lidar depolarization is in progress

• Computational cost can scale with number of points describing vertical profile N; we can cope with an N2 dependence but not N3

Radiative transfer forward Radiative transfer forward modelsmodels

Radar/lidar model Applications Speed Jacobian Adjoint

Single scattering: ’= exp(-2) Radar & lidar, no multiple scattering N N2 N

Platt’s approximation ’= exp(-2) Lidar, ice only, crude multiple scattering N N2 N

Photon Variance-Covariance (PVC) method (Hogan 2006, 2008)

Lidar, ice only, small-angle multiple scattering

N or N2 N2 N

Time-Dependent Two-Stream (TDTS) method (Hogan and Battaglia 2008)

Lidar & radar, wide-angle multiple scattering

N2 N3 N2

Depolarization capability for TDTS Lidar & radar depol with multiple scattering N2 N2

Radiometer model Applications Speed Jacobian Adjoint

RTTOV (used at ECMWF & Met Office) Infrared and microwave radiances N N

Two-stream source function technique (e.g. Delanoe & Hogan 2008)

Infrared radiances N N2

LIDORT Solar radiances N N2 N

• Infrared will probably use RTTOV, solar radiances will use LIDORT• Both currently being tested by Julien Delanoe

• Lidar uses PVC+TDTS (N2), radar uses single-scattering+TDTS (N2)• Jacobian of TDTS is too expensive: N3

• We have recently coded adjoint of multiple scattering models• Future work: depolarization forward model with multiple scattering

• Regime 0: No attenuation– Optical depth << 1

• Regime 1: Single scattering– Apparent backscatter ’ is easy to

calculate from at range r : ’(r) = (r) exp[-2(r)]

Scattering Scattering regimesregimes

Footprint x

Mean free path l

• Regime 2: Small-angle multiple scattering

– Occurs when l ~ x– Only for wavelength much less than particle size, e.g. lidar & ice clouds

– No pulse stretching

• Regime 3: Wide-angle multiple scattering (pulse stretching)

– Occurs when l ~ x

THOTHOR R

lidarlidar

Comparison of convergence Comparison of convergence ratesrates

• Solution is identical• Gauss-Newton method converges in < 10 iterations• L-BFGS Gradient Descent method converges in < 100 iterations• Conjugate Gradient method converges a little slower than L-BFGS• Each L-BFGS iteration >> 10x faster than each Gauss-Newton one!• Gauss-Newton method requires the Jacobian matrix, which must be

calculated by rerunning multiple scattering model multiple times

Unified algorithm: first results for Unified algorithm: first results for ice+liquidice+liquid

Ob

serv

ati

on

s

Retr

ieval

But lidar noise degrades retrieval

TruthRetrieval

First guessIterations

ObservationsForward modelled retrievalForward modelled first guess

Convergence!

Add smoothness constraintAdd smoothness constraintO

bserv

ati

on

s

Retr

ieval

TruthRetrieval

First guessIterations

ObservationsForward modelled retrievalForward modelled first guess

Smoother retrieval but slower convergence

Comparison with MODISComparison with MODIS• VarCloud-OA “Oblate ice”

– Our preferred ice model– Poor agreement with MODIS

optical depth and effective radius• VarCloud-BR “Bullet rosettes”

– Similar assumption to MODIS– Better agreement

• So why does “OA” IWP agree better?– IWP ~ opt. depth*effective radius– MODIS effective radius is from top

few optical depths of the cloud and assumed constant through cloud

– In reality (according to radar-lidar), effective radius increases with depth

• MODIS underestimates IWP for a given optical depth and effective radius

Optical depth

Effective radius

Ice water path

Satellite observations: IceSATSatellite observations: IceSAT• Cloud observations from IceSAT 0.5-micron lidar

(first data Feb 2004)• Global coverage but lidar attenuated by thick

clouds: direct model comparison difficult

Optically thick liquid cloud obscures view of any clouds beneath

Solution: forward-model the measurements (including attenuation) using the ECMWF variables

Lidar apparent backscatter coefficient (m-1 sr-1)

Latitude

Simulate lidar backscatter:– Create subcolumns with max-rand

overlap– Forward-model lidar backscatter from

ECMWF water content & particle size– Remove signals below lidar sensitivity

ECMWF raw cloud fraction

ECMWF cloud fraction after processing

IceSAT cloud fraction

Global cloud fraction Global cloud fraction comparison comparison

ECMWF raw cloud fraction ECMWF processed cloud fraction

IceSAT cloud fraction

Wilkinson, Hogan, Illingworth and Benedetti (MWR 2008)

• Results for October 2003– Tropical convection peaks too

high– Too much polar cloud– Elsewhere agreement is good

• Results can be ambiguous– An apparent low cloud

underestimate could be a real error, or could be due to high cloud above being too thick

Testing the model Testing the model climatologyclimatology

Reduction in model due to lidar attenuation

Error due to uncertain extinction-to-backscatter ratio