fast forward modelling of cloudy atmospheric statesearth.esa.int/workshops/envisatsymposium/... ·...
TRANSCRIPT
S. Mackie
(1), C. Merchant
(2), P. Francis
(3)
(1) IAES, University of Edinburgh ,West Mains Rd, Edinburgh EH9 3JN, U.K., Email:[email protected]
(2) IAES, University of Edinburgh ,West Mains Rd, Edinburgh EH9 3JN, U.K., Email:[email protected]
(3) NWP Division, Met. Office, Fitzroy Road, Exeter EX1 3PB, U.K., Email: [email protected]
ABSTRACT
Cloud detection always relies on some knowledge of
how clear and cloudy observations will differ. In a full
Bayesian determination of the probability that an
infrared image pixel contains cloud, an estimate of the
brightness temperature distribution for clear and cloudy
cases is required. A method for estimating this
distribution for cloudy atmospheric states through
exploitation of the knowledge already held about an
imaged scene is presented here. Relationships are found
between cloud properties and the brightness-
temperature predictions of a fast radiative transfer
model, run with atmospheric information specific to the
imaged scene. This means that the number of model
runs can be limited, without limiting the number of
clouds represented in the distribution. The technique is
demonstrated here in a case study, the results of which
suggest that clear areas of an image can be identified
with more certainty.
1. INTRODUCTION
As a global source of information on the Earth’s
atmospheric state, satellite data require accurate
interpretation. Interpretation of visible and infrared
imagery usually relies on detection of clouds within an
imaged scene. Standard operational detection is
performed on imagery by threshold testing, which
produces a mask of clear and cloudy pixels, e.g. [1-4],
following the approach of [5]. The threshold tests are
typically set according to the observations of an expert
through inspection of a number of imaged scenes, which
has led the approach to be criticised as ‘non-
transparent’, and dependent on the retaining of expertise
which may be lost. In addition to these criticisms, [6, 7]
point out that the binary mask produced by threshold
testing methods leaves no flexibility for different
tolerances to cloud contamination – more severe
thresholds that detect more cloud are likely to detect
fewer clear pixels, and vice-versa, although the severity
of the tests used in creating the end-product is often
unknown. It is further pointed out that a climatology
may be used to set the thresholds, but numerical weather
prediction (NWP) fields contain more temporally- and
spatially-specific information which could be exploited
to aid the detection [6, 7].
A semi-probabilistic, physically based alternative to the
threshold-method has been developed [7]. NWP fields
are used to calculate a probability density function
(PDF) of observations corresponding to a clear scene.
The PDF-element corresponding to a recorded pixel
observation is the prior probability of that pixel being
clear, according to the NWP fields. This is combined
with the prior probability of imaging a clear scene at the
pixel location (taken from cloud statistics generated by,
e.g. [8]) to calculate the posterior probability of clear for
the pixel. To make the technique fully-probabilistic and
physically robust, an NWP-conditional PDF for
observations of a cloudy scene is also needed, see Eq. 1.
( )
( )
1
1
,cb xo y P c P
c,b
xo
y P c P b
,xo
yc P
−
����
�
�
����
�
�
��
�
�
��
�
�
+=��
�
� (1)
( )cP , ( )cP are the prior probabilities of clear, c, and
not clear, c ; ��
���
�
�� c , x y, P ,c x yP
bobo are the
PDFs probabilities of the observations yo, given the
NWP fields xb, and clear or cloudy conditions.
The clear PDF is generated by running a radiative
transfer model (RTM) on the NWP-profile information,
and accounting for sensor noise and profile uncertainties
using Gaussian error assumptions. At present, [7] uses
an empirical distribution of observations recorded for
cloudy scenes in place of an NWP-conditional PDF for
cloudy scenes because of the difficulty in carrying out
such forward-modelling for cloudy scenes.
Rather than forward-modelling one set of atmospheric
conditions, as for the clear-sky case, the PDF must
represent observations for a range of atmospheric states,
with clouds at different altitudes, with different optical
depths and filling different fractions of the pixel. To be
useful operationally, the PDF must be generated
quickly, making it impractical to cover the range of
possible cloudy conditions with individual RTM runs.
The following sections present a method of generating a
PDF for cloudy atmospheric states.
FAST FORWARD MODELLING OF CLOUDY ATMOSPHERIC STATES
_____________________________________________________
Proc. ‘Envisat Symposium 2007’, Montreux, Switzerland 23–27 April 2007 (ESA SP-636, July 2007)
2. METHOD
The NWP profile is forward modelled using RTTOV-7
with 60 altitude levels. Single phase clouds are added to
each of the modelled altitudes in separate model-runs,
with cloud pixel-coverage varying from 10% to 100%,
and cloud ice- or liquid- water path, cwp, varying from
0 to 0.1 kg m-2
. The forward-modelled brightness
temperatures, BTs, are plotted against cwp for each
modelled altitude and cloud fraction and an exponential
curve fitted, e.g. Fig, 1.
Figure 1. Forward-modelled BTs plotted against cwp.
Dia: forward-modelled BTs, line: fitted curve.
The equation for the curve has the form BT = a + b*(1 –
ecwp/c
) , where a, b and c are fitting parameters. With the
curve defined, BTs can be read for clouds with cwp
values other than those modelled, so reducing the
number of RTM necessary runs.
The steepness and minimum BT value of the curve
changes with altitude and pixel coverage as lower pixel
coverage requires a thicker cloud for an optically
saturated observation, and higher clouds will optically
saturate at lower BTs. The parameters are interpolated
to give the fitting parameters for BT-cwp curves at
altitudes and pixel coverages between those that are
forward-modelled. In this way, BTs for clouds with tops
at 10m intervals through the atmosphere, and with pixel
coverages varying from 10%-100% in 1% increments,
can be found without being explicitly modelled.
2.1. Weighting the Clouds
The clouds that contribute to the PDF must be realistic
given the NWP profile. The dataset from [9] was used
to ensure clouds represented in the PDF have cwp of
less than or equal to the maximum cwp recorded at that
altitude. The contribution of each cloud to the final PDF
needs to be weighted by its relative likelihood. A weight
is given to optically thick clouds using the ratio of
clouds in this dataset with cwp greater than or equal to
the maximum modelled cwp, which was chosen to be
beyond the optical saturation point.
An empirical dataset of AATSSR-acquired imagery,
consisting of measurements from January, April, July
and October, was used to calculate a latitude- and
season-specific ratio of cloud-filled pixels to cloud-edge
pixels, e.g. Fig. 2. The AATSR cloud mask was used to
define a pixel as cloud-edge if it was classed as cloud
but had � 1 neighbours classed as clear, or if it was clear
bur had � 1 neighbours classed as cloud. The 8
immediate neighbours of a pixel were used for this, and
cloud pixels for which all 8 neighbours were also
classed as cloud, were defined as cloud-filled. This ratio
was used to weight clouds with 100% pixel coverage
more heavily in the PDF.
Figure 2. Ratio of cloud-filled to cloud-edge pixels for
autumn AATSR dataset
Ice phase clouds are only represented in the PDF at
altitudes for which the temperature (interpolated from
the NWP profile temperature) is below 273.15K. Liquid
phase clouds at temperatures below 273.15K are
represented with a linearly decreasing weight, which
reaches 0 at 233.15K to account for super-cooled water
clouds.
The aim of the work is to enhance the technique
developed by [7] to aid sea surface temperature (SST)
retrieval, and so only clouds which introduce a bias of
0.2K or greater to the retrieved SST are represented in
the PDF.
The distribution of BTs predicted for a cloudy
atmosphere is convolved with Gaussian error
assumptions in the same way as for the clear sky, and
so the PDF is created.
3. USE OF THE CLOUDY-SKY PDF
To assess the performance of the PDF, it can be
included in the full cloud detection code [7] as a direct
replacement for the global distribution currently used.
That is, the PDF generated from one NWP-profile can
be applied to observations from an imaged scene, which
spans more than one NWP grid cell. Some indication of
the profile-dependence of the PDF is given by its
varying performance in areas close to, and far from,
those described by the NWP-profile.
4. RESULTS
The PDF generated for cloudy atmospheric states using
an NWP-profile for an area off the coast of Korea in
May 2005 is shown in Fig.3. This can be compared to
the global distribution of cloudy atmosphere
observations currently used by [7] in Fig. 5. The PDF in
Fig. 3 was generated for a profile centred to the area
marked in yellow on the visible imagery, Fig. 4a. The
spectral probability of clear calculated for this image
using the global distribution is shown in Fig. 4b. The
PDF in Fig. 3, and a PDF from a profile centred on the
cyan box, were each applied to the whole image,
creating the spectral-probability-of-clear plots shown in
Fig. 4c,d.
Figure 3. PDF plotted in 2-dimensions;,filled contours
at 20 equally spaced intervals, spanning range of
distribution (0 –0.0126874). Black contours plotted on a
logarithmic scale, filled contours equally spaced.
Profile taken from yellow (northern) box in Fig. 4a
a b
c d
Probability of Clear
0 0.5 1
Figure 4 a. 1.6m image with marked regions centred on
location of NWP profiles used to generate PDFs; b. Spectral probability of clear calculated for whole image
using global distribution in place of a PDF; c. using
PDF from profile centred on yellow (northern) box; d.
using PDF from profile centred on cyan (southern) box.
Figure 5. The global distribution of cloudy-atmosphere
observations used to create Fig. 4b (black contours on
logarithmic scale).
Some quantitative comparisons were made between the
results based on the global distribution, and those based
on the PDFs from the 2 profiles. A region of 100-pixels2
centred on the profile-centre was considered for the
comparison, shown in table 1 and Fig. 6. Pixels with a
calculated probability of clear greater than 50% were
deemed clear for the comparison, otherwise they were
considered cloud-pixels. Of each of the two classes,
clear and cloud, the number of pixels that the two
technique classed with higher certainty are compared.
Table 1. Comparison of the calculated probability of
clear for pixels in region centred on NWP-profile
location, using the global distribution and using the
NWP-conditional PDF.
profile 1 region (Fig. 4a yellow box)
global
distribution
NWP-conditional-PDF
cld pix 3906 3945
clr pix 36495 36456
% of clr px > 85% 26.9% 56.3%
% of clr px > 90% 0% 37.5%
% cld px < 15% 70.5% 71.9%
% cld px < 10% 66.6% 68.0%
profile 2 region (Fig. 4a cyan box)
global
distribution
NWP-conditional-PDF
cld pix 7476 7295
clr pix 32925 33106
% of clr px > 85% 20.7% 60.4%
% of clr px > 90% 1.5% 44.8%
% cld px < 15% 12.9% 12.1%
% cld px < 10% 11.9% 11.2%
Figure 6 .Histogram of probability values calculated for
pixels using global distribution (dashed line) and
conditional PDF (solid line). Top: region around
profile 1 (yellow box in Fig. 4a); Bottom: region around
profile 2 (cyan box in Fig. 4a)
5. DISCUSSION
The PDF appears to perform a more polarized
classification in the region from which the profiles were
taken. It is expected that this will lead to a reduction in
the false alarm rate.
It is not expected that the PDF out-perform the global
distribution of BTs for cloudy atmospheric states in
regions away from the profile location, and so these
preliminary results show the location-specific nature of
an NWP-profile-dependent PDF. It is intended that the
cloudy PDF eventually be included in the cloud
detection code [7], in the same way as the clear-sky
PDF is at present. That is, it will be calculated for every
available profile within an imaged scene and the results
interpolated between profile-centre locations. The plots
show the results using scene-specific PDFs to be more
polarized in the region where they apply. Classification
of pixels into ‘clear’ and ‘cloudy’ classes can therefore
be done with more certainty.
The PDF could be made more conditional on the NWP
profile, for example by limiting the clouds represented
to those realistic for the profile. It is intended to
investigate this, but it is also anticipated that such
conditions may lead to problems when atmospheric
conditions vary within a profile grid cell, e.g. in the case
of ocean fronts.
These preliminary results are encouraging, showing the
benefits of exploiting scene-specific information to form
a PDF for cloudy pixels in a probabilistic cloud
classification scheme.
6. REFERENCES
1. Stowe, L.L., P.A. Davis, and E.P. McClain,
Scientific basis and initial evaluation of the CLAVR-
1 Global Clear/Cloud Classification Algorithm for
the Advanced Very High Resolution Radiometer.
Journal of Atmospheric and Oceanic Technology,
1999. 16: p. 656-681.
2. Kriebel, K.T., et al., The cloud analysis tool
APOLLO: Improvements and validations.
International Journal of Remote Sensing, 2003.
24(12): p. 2389-2408.
3. Hutchison, K.D., et al., Automated cloud detection
and classification of data collected by the Visible
Infra-red Imager Radiometer Suite (VIIRS).
International Journal of Remote Sensing, 2005.
26(21): p. 4681-4706.
4. Derrien, M. and H. Le Gléau, MSG/SEVIRI cloud
mask and type from SAFNWC. International Journal
of Remote Sensing, 2005. 26(21): p. 4707-4732.
5. Saunders, R. and K. Kriebel, An improved method
for detecting clear sky and cloudy radiances from
AVHRR data. International Journal of Remote
Sensing, 1988. 9(1): p. 123-150.
6. Merchant, C.J., et al., Probabilistic physically-based
cloud-screening of satellite infra-red imagery for
operational sea surface temperature retrieval.
Quarterly Journal - Royal Meteorological Society,
2005. 131(611): p. 2735-2756.
7. Old, C., et al., Generalised Bayesian cloud detection
for thermal and reflectance imagery: software
development and validation. IEEE Transactions on
Geoscience and Remote Sensing, 2007.
8. Rossow, W.B. and R.A. Schiffer, ISCCP cloud data
products. Bulletin, American Meteorological
Society, 1991. 72(1): p. 2-20.
9. Chevallier, F., Sampled databases of 60-level
atmospheric profiles from the ECMWF analyses, in
SAF Programme, E. ECMWF, Editor. 2001. p.
Research Report no. 4.