multicolour image analysis

7/29/2019 Multicolour image analysis

1/16

Multicolour analysis and local image correlation inconfocal microscopy*

D . D EMAND O LX & J. D AV O U ST

Centre dImmunologie CNRS-INSERM de Marseille-Luminy, Case 906, 13288 MarseilleCedex 9, France

Key words. Co-localization, confocal microscopy, fluorescence microscopy,

fluorogram, image analysis, image correlation, multicolour analysis, pixel

histogram.

Summary

Multiparameter fluorescence microscopy is often used to

identify cell types and subcellular organelles according to

their differential labelling. For thick objects, the quantitative

comparison of different multiply labelled specimens requiresthe three-dimensional (3-D) sampling capacity of confocal

laser scanning microscopy, which can be used to generate

pseudocolour images. To analyse such 3-D data sets, we

have created pixel fluorogram representations, which are

estimates of the joint probability densities linking multiple

fluorescence distributions. Such pixel fluorograms also

provide a powerful means of analysing image acquisition

noise, fluorescence cross-talk, fluorescence photobleaching

and cell movements. To identify true fluorescence co-

localization, we have developed a novel approach based on

local image correlation maps. These maps discriminate the

coincident fluorescence distributions from the superimposi-tion of noncorrelated fluorescence profiles on a local basis,

by correcting for contrast and local variations in back-

ground intensity in each fluorescence channel. We believe

that the pixel fluorograms are best suited to the quality

control of multifluorescence image acquisition. The local

image correlation methods are more appropriate for

identifying co-localized structures at the cellular or

subcellular level. The thresholding of these correlation

maps can further be used to recognize and classify biological

structures according to multifluorescence attributes.

Introduction

In fluorescence microscopy, the simultaneous or sequential

detection of several channels is of great importance in

biological applications where several markers have to be

superimposed and compared (Brelje et al., 1993). Protein

immunofluorescence cytochemistry and fluorescence in situ

hybridization (FISH) of nucleic acids now permit the

detection of a wide variety of macromolecules within cells

and tissues. In conventional epifluorescence microscopy of

thin specimens, the multiple detection of nucleic acids by

FISH can be achieved by combining one or severalfluorophores for each hybridization (Nederlof et al., 1990);

the resulting diversity allows the identification of more than

10 sites with three distinct labels mixed in multiple ratios

(Dauwerse et al., 1992; Nederlof et al., 1992a,b).

For thick specimens, the images acquired with conven-

tional microscopes suffer from the out-of-focus blur of the

fluorescence emissions, impairing the estimation of co-

localization between several markers. Confocal microscopy

has extended the capacity of optical microscopy by allowing

3-D sampling of specimens while excluding out-of-focus

blur (Wijnaendts-van-Resandt et al., 1985; White et al.,

1987; Brakenhoff et al., 1989; Shotton, 1989). The

improved axial resolution allows the 3-D localization of

fluorescence markers (Sheppard, 1989) and 3-D image

reconstruction (Van der Voort et al., 1989, 1993; White,

1995). Moreover, the lateral resolution may, under ideal

conditions, be improved compared with nonconfocal optical

microscopy (Sheppard & Cogswell, 1990; Wilson, 1990).

The confocal out-of-focus fluorescence rejection laid the

foundations for the quantitative comparison of multifluor-

escence signals (Akner et al., 1991; Sandison & Webb,

1994; Sandison et al., 1995). In multifluorescence confocal

microscopy, the specimens are scanned with one or several

excitation laser lines, allowing sequential or simultaneous

detection of several fluorescent markers through multipleacquisition channels (Mossberg et al., 1990; Brelje et al.,

1993). Compared with flow cytometry, cells are analysed at

a slower rate, yielding a confocal image of high spatial

resolution of a limited number of cells. Multifluorescence

confocal microscopy can also yield valuable information for

the identification of multiply labelled structures, or to

monitor the time evolution of intracellular Ca2+, H+ ion or

cyclic AMP concentrations to be followed by fluorescence

Journal of Microscopy, Vol. 185, Pt 1, January 1997, pp. 21 36.

Received 21 June 1996; accepted 16 September 1996

21 1997 The Royal Microscopical Society

Correspondence to: Denis Demandolx. Tel: (+33) 4 91 26 94 36; Fax: (+33) 491

26 94 30; E-mail: [email protected]

* Paper presented at MICRO 96, London, 2 4 July 1996.


2/16

ratio imaging of dedicated probes (Poenie et al., 1986;

Bright et al., 1989; Adams et al., 1991).

Double and triple immunofluorescence labelling permits

one to estimate the co-localization between fluorescent

markers in the same field (Schubert, 1991; Brelje et al.,

1993; Paddock et al., 1993; Paddock, 1995). Several

methods based on the superimposition of colour channels

have been explored to reveal coincident immunofluores-

cence labelling (Arndt-Jovin et al., 1990; Fox et al., 1991;Humbert et al., 1992, 1993; Dutartre et al., 1996) or to

compare FISH vs. immunofluorescence labelling (Leger et

al., 1994). The local subtraction of appropriately scaled

fluorescence channels (Akner et al., 1991) and the double

thresholding of fluorescence values (Mossberg et al., 1990)

have also been used to compare confocal images. Global

correlation coefficients have been explored to estimate the

degree of correspondence between two related fluorescence

micrographs (Manders et al., 1992, 1993). In addition,

several authors have identified fluorescence cross-talk and

image registration defects due to chromatic aberrations or

misalignment of confocal optics as fundamental problems in

multifluorescence acquisitions (Akinyemi et al., 1992;

Sandison et al., 1995; White et al., 1996). A judicious choice

of fluorophores, laser types, objective lenses, fluorescence

filter sets, detection pinhole settings and photon detectors

often reduces these limitations (Mossberg & Ericsson, 1990;

Sheppard et al., 1992; Brelje et al., 1993; Keller, 1995; Tsien

& Waggoner, 1995), which can further be compensated for

by digital image processing (Carlsson & Mossberg, 1992;

Brelje et al., 1993). Multicolour image analysis software is

highly desirable to control the acquisition conditions, to

evaluate the problems of registration and aberrations and to

reveal the co-localized fluorescence distributions.

To check the quality of image acquisition procedures andto study the correspondence between multifluorescence

labelled structures, we define here several levels of image

analysis. We first propose the use of an improved dual-

channel look-up table (LUT), second, we create statistical

representations of multiple fluorescence data, and third we

develop local image correlation methods. The enhanced

dual-channel LUT allows a higher colour discrimination of

double fluorescence labelling, the pixel fluorograms provide

a statistical representation of multifluorescence distributions,

and the local correlation maps reveal co-localized struc-

tures. Use of the improved LUT, pixel fluorogram analysis

and local image correlation have to be combined for the

global and local analysis of fluorescence co-localization.

Materials and methods

Materials, cells and antibodies

All chemicals and proteins were purchased from Sigma Chemi-

cal Co. (St Louis, MO, U.S.A.). Major histocompatibility

complex (MHC) class II IAk and invariant (Ii) chain positive

fibroblast cells were obtained after transfection of IAka, IAkb

and Ii chains using the calcium phosphate precipitation

method as described (Salamero et al., 1990; Humbert et al.,

1993). Anti-MHC class II IAk 10.2.16 monoclonal anti-

bodies and rabbit polyclonal anti-cathepsin D antibodies

were obtained and used as described (Humbert et al., 1993).

Rabbit polyclonal anti-Ii chain antibodies were generously

provided by Nicolas Barois (Centre dImmunologie CNRS-INSERM de Marseille-Luminy, France). Texas-Red-coupled

wheat germ agglutinin (WGA) was obtained from Molecular

Probes Inc. (Eugene, OR, U.S.A.).

Intracellular indirect immunofluorescence

Mouse fibroblast and B lymphoma cells were cultured for

48 h on glass coverslips. After washing three times with

PBS, cells were fixed for 15 min with a 4% paraformalde-

hyde solution in phosphate-buffered saline (PBS) and

permeabilized with 0.1% Triton X100 solution. Cells were

subsequently washed and blocked with PBS containing

0.2% gelatin. Antibodies were diluted to a 10mg mL1

concentration in PBSgelatin. To perform indirect immuno-

fluorescence labelling, cells were incubated for 20 min with

primary antibodies at room temperature, and then washed

three times, before incubation with either FITC- or Texas-

Red-coupled anti-IgG antiserum (1/200 dilution in PBS

gelatin). The labelled cells adherent to the glass coverslips

were mounted on glass slides with Mowiol embedding

medium (Humbert et al., 1993). To stain the surface of

living cells, we incubated cells grown on coverslips with

1mg mL1 of Texas-Red-coupled WGA for 20 min at 20 C.

The cells were mounted in optical chambers containing

400mL of medium as described (Pollerberg et al., 1986).

Configuration of the confocal laser scanning microscope

Confocal laser scanning microscopy was carried out using a

Leica TCS 4D instrument (Leica Lasertechnik, Heidelberg,

Germany), based on a Leitz DMRBE microscope interfaced

with an argonkrypton laser specified to have a maximum

power of 25 mW in each line (488, 568 and 647 nm). The

total laser output power was set to 25 mW. Selecting the

488- and 568-nm lines simultaneously, we have 8 mW for

each line. Owing to an initial attenuation with filters, to

optical fibre coupling, to the confocal excitation aperture

and to an over-illumination of the 1.4 numerical aperture

(NA) of the microscope 100 PL APO objective, the input

power is reduced 25-fold. The total illumination power

thus lies around 0.3 mW for each laser line on the specimen

under these conditions. To minimize the noise and to keep

the photobleaching rate below 1% for FITC, we selected an

acquisition time of 1 s per scan and averaged 16 scans to

produce each 1024 1024-pixel image, as a standard

22 D. DEMANDOLX AND J. DAVOUST

1997 The Royal Microscopical Society, Journal of Microscopy, 185, 2136


3/16

procedure for all the applications. (For the application of

Fig. 2, where we wanted to accentuate the effects of

photobleaching, we tripled the laser power.) For most of the

applications, we selected a field of interest of 512 512

pixels from the raw micrographs. The Nyquists criterion

indicates that to digitize a signal whose cut-off frequency is

fc, it is necessary to sample at a frequency of at least 2fc. This

principle can be applied in confocal microscopy to select the

appropriate sampling rate of the detected signal (Young,1988). With the 100 PL APO objective, the lateral optical

resolution is about 0.18mm and the axial resolution along

the z-axis is 0.5mm. We used a sampling step of 0.095mm in

the plane of section and 0.25mm in the axial direction.

In multicolour analysis, the image registration between

channels is of great importance. We checked that

wavelength-dependent distortions were small compared

with the optical resolution for objects close to the coverslip.

The possible chromatic aberrations (Akinyemi et al., 1992)

are considerably reduced by using plan apochromatic

objective lenses, provided that for an oil-immersion lens

imaging an aqueous specimen, the plane of focus is adjacent

to the coverslip (Keller, 1995). Actual measurements

provided by Leica Lasertechnik indicate that the relative

axial shifts between 488 and 568 nm lines are never greater

than 50 nm using 100 1.4-NA PL APO oil-immersion

objective. Experimental verifications using the detection of

double-labelled subcellular endosomal structures present in

the cells close to the coverslip/medium interface showed no

visible shift between green and red fluorescence emissions

even at the periphery of the field. The use of a single argon

krypton laser feeding a monomode optical fibre guarantees

the alignment of laser lines. Focal series of four horizontal

planes of section spaced at 0.25mm have been monitored

simultaneously for FITC and Texas Red. As we recordedseries of optical sections close to the coverslips, we have

minimized the optical aberrations due to differences of

refractive index inside the specimen. Moreover, the Mowiol

mounting medium has a refraction index close to the glass

coverslip (n = 1.52). For standard acquisitions, we used a

double dichroic mirror for the excitation beam, a band pass

520560 nm barrier filter for FITC, a long pass barrier filter

above 580 nm for Texas Red detection and two photo-

multiplier tubes (PMT). Depending on the relative amount

of FITC and Texas Red, and the gain settings of the two

detectors, simultaneous dual-channel acquisition can gen-

erate a small amount of fluorescence cross-talk between

FITC and Texas Red channels (Mossberg & Ericsson, 1990;

Carlsson & Mossberg, 1992; Brelje et al., 1993).

Implementation of image processing programs

The individual 8-bit-encoded 1024 1024- or 512 512-

pixel images, one from each channel for each plane of section,

were combined and visualized with a 16-million-colour

display screen and printed using a Codonics NP-1600 photo-

graphic network colour printer utilizing dye-sublimation

technique (Codonics Inc., Middleburg Heights, OH, U.S.A.).

Image processing, false-colour mapping, pixel fluorogram

and local image correlation algorithms were implemented

in C programming language using Microwares C Language

Compiler System (Microware, Des Moines, IA, U.S.A.) and

then linked with Leicas image processing library. Our

current programs run on a Motorola 68040 microprocessorsystem (Motorola, U.S.A.) with Microwares OS-9 operating

system. To display dual-colour fluorescence images, we have

used both classical and extended LUT as described in the

Results. Pixel fluorogram and local image correlation

techniques are extensively described in the Results and the

Appendix.

Results

Improved look-up tables

Through all this work, we have used double fluorescence

confocal micrographs obtained from fibroblast tissue culture

cells expressing the IAkab MHC class II molecules and the

associated Ii chain (Humbert et al., 1993). Figure 1a shows

a typical plane of section recorded 1mm above the solid

coverglass support used for cell culture and simultaneously

observed with FITC and Texas Red acquisition channels.

MHC class II and Ii chain are represented in green and red,

respectively. This type of double fluorescence micrograph

can be visualized with a standard redgreen LUT producing

a limited range of shades from green for FITC to red for

Texas Red. Note that the plasma membrane is only labelled

by indirect FITC immunofluorescence revealing the surface

expression of MHC class II molecules. Internal vesicles arelabelled with both fluorophores but with different ratios,

indicating a partial segregation of MHC class II molecules

and Ii chains (Humbert et al., 1993).

As shown above, multifluorescence confocal micrographs

are usually represented by the superimposition of their red,

green and blue colour-coded channels. However, this direct

representation leads to interpretation problems linked with

a differential sensitivity of our perception of red, green and

blue. To circumvent these difficulties, we have extended this

dual-channel LUT from cyan for FITC to magenta for Texas

Red through green, yellow and red for halfway combina-

tions, improving the discrimination of the fluorescence

ratios between FITC and Texas Red (Fig. 1b). The concept

behind this extended dual-channel LUT is inspired by

consideration of the classical huesaturationluminance

(HSL) colour space. Here, a specific hue value is defined for

each ratio between the pure FITC and Texas Red

fluorescence components x and y (see LUT bar in Fig. 1a).

The corresponding luminance is here calculated by the or

fuzzy logical operator x + y xy/255 varying from 0 to 255


M U LT I C O L O U R A N A L Y SI S I N C O N F O C A L M I C R O S C O P Y 23


4/16




5/16

for initial 8-bit-encoded fluorescence values. Other lumi-

nance operators are conceivable, such as the maximum

value ofx and y. The final choice and the usefulness of such

colour correction depends on accepted conventions. If the

hues of the extended LUT are too far from conventional red

green LUT, their interpretation can require an adaptation

time.

Statistical representations of the fluorescence micrographs

To provide a statistical representation of the image data sets

for each channel, we can calculate the pixel histogram

distribution, showing the number n(k) of each pixel value

within the image:

nk cardinalfxi; j kg; 1

where n(k) denotes the number of pixels whose fluorescence

value x(i,j) equals k for all possible coordinates (i,j).

A large contribution of weakly fluorescent areas leads to a

sharp peak at the origin of the graphs for each channel

(Fig. 1c,d). To classify the occurrences of double-labelled

pixels in the micrographs, we decided to use second-order

histograms to estimate the joint probability density linking

two fluorescence distributions (Pratt, 1991). Second-order

histograms show the number n(k,l) of pixels for which the

values of each channel x(i,j) and y(i,j) equal, respectively k

and l:

nk; l cardinalfxi; j k; yi; j lg: 2

As an application, second-order semilogarithmic histograms

have been calculated for each cell circled on the micrograph

of Fig. 1b (Fig. 1eh). By analogy with the well-known

cytofluorogram representation from flow cytometry, we are

proposing to call this representation a pixel fluorogram.Pixel values of FITC and Texas Red components (k,l) are

placed, respectively, along the x- and y-axes. These graphs

can also be represented by means of contour lines

(Demandolx & Davoust, 1995). The sharp peak at the

origin of these pixel fluorograms corresponds to the low

luminance levels of the double fluorescence labelling. It

should be noticed that these pixel fluorograms are composed

of a multitude of radial streaks showing the local correla-

tions between fluorescence distributions. Each co-localized

structure corresponds to a radial streak. On the other hand,

horizontal or vertical clouds of dots lying along the x- and y-

axes are related to single labelled structures in the

micrographs. Usually, each combination of pixel values

(k,l) corresponds to a specific colour on the dual-channel

micrographs. It is therefore possible to derive colour-coded

pixel fluorograms from the actual semilogarithmic repre-sentation, as shown in Fig. 1i,k. Dots on such pixel

fluorograms are represented with the same colour as in

the raw dual-colour image. A straightforward visualization

of double-labelled pixels in the micrograph can be achieved

by selecting the pixels above a threshold for each

fluorescence channel in the pixel fluorogram (Fig. 1j,l).

This is equivalent to a double thresholding using the image

amplitudes as attributes (Mossberg et al., 1990; Pratt, 1991;

Pal & Pal, 1993). We preferred to limit ourselves to each

elliptical area of interest because it is often difficult to use a

global threshold for a whole field containing several cells.

Provided that the nonspecific fluorescence is uniformly

distributed within the object, this method allows us to

distinguish double positive, single positive and background

areas according to the selected portion of the pixel

fluorogram.

Application of the pixel fluorograms

The pixel fluorograms can be used to identify double-

labelled pixels of interest and are helpful in the analysis of

small variations between closely related images. Signal-to-

noise ratio (SNR) and basic movements can be analysed in

this way. As an example, a confocal fluorescence micro-

graph of B lymphoma cells is shown in Fig. 2a. Cells arelabelled indirectly with primary anti-IAk MHC class II

molecules and FITC-coupled secondary antibodies. We have

represented the pixel fluorograms obtained between two

sequentially single scanned images (Fig. 2b), four-time-

averaged scanned images (Fig. 2c) and 16-time-averaged

scanned images (Fig. 2d). The diagonal cloud of dots on the

pixel fluorogram gets thinner as the fluorescence signal and


Fig. 1. Double-labelling analysis with pixel fluorograms. (a) Double immunofluorescence labelling of fibroblast transfectants monitored by

confocal microscopy. IAkab MHC class II molecules are represented in green after FITC labelling and the Ii chain is represented in red

after Texas Red labelling. The additive superimposition of both channels gives a range of yellow shades as shown in the LUT bar in the bottom

right corner. Field of view 50 50 mm. (b) Same field as in (a) but the combination between FITC and Texas Red is visualized using anextended LUT with more shades for different FITC and Texas Red ratios. Four elliptical areas of interest 14 have been defined around

the cells to compute the pixel fluorograms. (c,d) Semilogarithmic pixel histograms of the FITC and Texas Red components, respectively.

(eh) Second-order pixel fluorograms from the four elliptical areas of interest 14 displayed in (b). The grey level is inversely proportional

to the logarithm of pixel numbers. To distinguish isolated dots, we have used a black background. As a standard procedure, FITC and Texas

Red components are represented along the x- and y-axes, respectively. (i) Colour-coded pixel fluorograms of the same elliptical areas of interest

1 from panel (b). Pairs of pixel values present in the area of interest are represented by a dot of the same colour in the pixel fluorogram and in

the image. (j) Double-labelled pixels selected and displayed in white in the colour-coded pixel fluorogram (i) are visualized on the micrograph

(j). (k,l) This is the same as (i,j) but for the area of interest 3 of (b).



6/16

the number of scans increase, indicating an improved SNR

(Fig. 2bd). The width of the cloud-of-dots is proportional to

the standard deviation of the acquisition noise and

decreases as 1/

n

p

where n denotes the number of scans(Mossberg et al., 1990). In confocal fluorescence micro-

scopy, the noise originates mostly from the photon number

which follows Poisson statistics (Young, 1996) character-

ized by a standard deviation proportional to the mean of the

corresponding signal. This explains the parabolic distribu-

tion of the cloud-of-dots of Fig. 2. The photobleaching of

FITC between two acquisitions is also detected as a tilt of the

diagonal as shown in Fig. 2d (Brakenhoff et al., 1994). If

parts of the specimen undergo limited movement during

time lapse recording of a living specimen, off-diagonal dots

will appear on the pixel fluorogram. Such an example is

shown in the second part of Fig. 2, where we have presented

the superimposition of the surface labelling of living cells

recorded at time 0 and time 1 min (Fig. 2e) and at time 0

and time 15 min (Fig. 2g). The corresponding colour-coded

pixel fluorograms become more spread as a function of time

(Fig. 2f,h, respectively). For a clear detection of the

movement, good SNR and limited fluorescence photobleach-

ing are obviously required. More sophisticated methods,

optimized to track the cell movements, can also be

considered. However, the pixel fluorograms developed here

are very helpful in the analysis of SNR, photobleaching and

can reveal partial displacements in the specimens.

Application of intensity-weighted pixel fluorograms

To further quantify the fluorescence values for each class of

pixels identified on the pixel fluorograms, we have defined a

fluorescence intensity-weighted pixel fluorogram from the

double confocal immunofluorescence micrograph of Fig. 3a.

In such a pixel fluorogram, the number of pixels n(k,l) is

multiplied by the FITC or the Texas Red fluorescence

intensities k and l leading to the integrated FITC and Texas

Red fluorescence values k.n(k,l) and l.n(k,l). Colour-coded

intensity-weighted pixel fluorograms are then obtained by

merging FITC and Texas Red fluorescence integrated values

using dual-colour LUT (see Fig. 3be). As illustrated below,

fluorescence intensity-weighted pixel fluorograms can be

used to analyse fluorescence cross-talk and background

fluorescence.

The fluorescence cross-talk occurs when part of a

fluorophore emission is detected in the wrong fluorescence

channel. In particular, FITC and Texas Red fluorophores can

emit photons of longer wavelength which are detected and

Fig. 2. Pixel fluorograms for noise and movement analysis. (a) Confocal immunofluorescence micrograph of IAk MHC class II molecules in B

lymphoma cells indirectly labelled with FITC. Field of view 70 70 mm. (b-d) Pixel fluorograms of sequential acquisitions from the section (a)

for 1-, 4- and 16-scan averaged images, respectively. Components from the first and second acquisition are represented along the x- and y-

axis, respectively. The tilt of the cloud-of-dots (d) is due to the photobleaching of FITC after 16 scans with triple laser power. (e) Time-lapse

confocal image of living fibroblasts labelled on their surface with Texas Red-coupled WGA. Two time lapse acquisition with a 1-min interval

are superimposed. Field of view 50 50 mm. (f) Colour-coded pixel fluorogram of the two superimposed components of (e). (g,h) Same as (e,f),

but with a time interval of 15 min.




7/16

added in another fluorescence channel (Carlsson & Moss-

berg, 1992). FITC emission can leak into the Texas Red

acquisition channel and Texas Red can leak into the

Cyanine 5 channel. Fluorescence cross-talk results in verycharacteristic pixel fluorograms. Everything happens as if

the fluorophore x,y-base was no longer orthogonal. Since

FITC-stained regions of the specimen emitting green

photons are also emitting a smaller number of red photons

detected in the Texas Red channel, the FITC x-axis makes an

acute angle as shown in Fig. 3b. Pixel fluorogram

representations are thus important to evaluate the cross-

talk between fluorescence channels and to perform the

appropriate corrections. Different approaches have been

proposed for the compensation of colour images (Carlsson et

al., 1994). A practical approach implies the specification of

a matrix operator that will remove cross-talk contributions

as described by Brelje et al. (1993). Matrix coefficients can

be determined by estimating the cross-talk percentage from

the slant in the contours of the intensity-weighted pixel

fluorogram. The tangent of the angle made with the FITC

axis defines the ratio of the FITC intensity detected in the

Texas Red channel to the FITC intensity detected in the FITC

channel (Fig. 3b) and consequently the matrix coefficients

for its correction (Fig. 3ce).

Most indirect immunofluorescence micrographs contain

nonspecific fluorescence levels referred to as background

fluorescence. This background fluorescence may be due to

autofluorescence of the specimen, or to nonspecific absorp-tion of fluorescence coupled secondary antibodies usually

present all over cellular specimens. This gives a significant

contribution to the integrated fluorescence, as shown in the

intensity-weighted pixel fluorogram close to the origin of the

graph. Therefore, we recalculated the intensity-weighted

pixel fluorograms after appropriate fluorescence offsetting

(Fig. 3d,e). In these representations, dots on the pixel

fluorograms are weighted with offset-corrected fluorescence

intensities. Mossberg et al. (1990) proposed a method for the

choice offsets from each channel histogram. Ideally the

offsets should correspond to the averaged FITC and Texas

Red background fluorescence recorded under the same

optical conditions for a negative control specimen. By

calculating intensity-weighted pixel fluorograms using off-

set corrected fluorescence values, we can now resolve

differently the bins of integrated fluorescence. The joint

distribution of fluorescence integrated values now reveals

the contribution of specific fluorescence measurements in a

better way. To minimize the local dispersion of the pixel

fluorogram distributions, linear low-pass or nonlinear


Fig. 3. Application of intensity-weighted pixel fluorograms. (a) Double confocal immunofluorescence micrograph of fibroblasts. FITC-labelled

antibodies directed against the lysosomal enzyme cathepsin D labels intracellular vesicles whereas Texas Red-coupled WGA binds mostly at

the cell surface. Field of view 50 50 mm. (b) Intensity-weighted pixel fluorogram of (a). The white line corresponds to the slant induced by

the 12% cross-talk of FITC to the Texas Red channel. (c) Same as (b) but with a cross-talk compensation. (d) Same as (c) but FITC and Texas

Red fluorescence have been offset to correct for background fluorescence before the computation of the fluorogram. Offset values of 16 for

FITC and 16 for Texas Red are shown by vertical and horizontal lines, respectively. (e) Same as (d) but using a preprocessing 13 13-pixel

low-pass filter.



8/16

median filtering is usually advantageous. To remove from

the pixel fluorograms the contribution of the high spatial

frequency components due to the acquisition noise, we have

applied a 3 3-pixel low-pass convolution filter (Pratt,

1991) throughout this paper. Using as a test a more drastic

Gaussian convolution filter whose standard deviation is set

to 3 pixels on a 13 13-pixel window, all clouds-of-dots are

smoothed (Fig. 3e).

The intensity-weighted pixel fluorograms can revealseveral bins of pixels corresponding to statistically signifi-

cant areas of interest, such as different levels of background,

and single or double-labelled areas. However, very large or

complex images contain many overlapping bins of pixels

which leads to confused situations for the interpretation of

pixel fluorograms. Background fluorescence correction and

local averaging are needed to extract more information on

multifluorescence images. The evaluation of specific vs.

background fluorescence is always more accurate on a local

basis.

Local image correlation

In order to detect co-localization of fluorescence distribu-

tions on a local basis, we have to extract more information

from the neighbouring pixels. In the strict sense, a co-

localization means a correspondence between two fluores-

cence distributions varying locally in the same way.

Provided that the co-localized area stretches over relatively

few pixels and that we do not reach fluorescence saturation

level (Visscher et al., 1994), the cloud-of-dots of a very local

pixel fluorogram is clustered along a straight line (Fig. 4a,b).

The slope of such a cloud-of-dots is related to the

fluorescence ratio between the markers used. The shape

and the width of the cloud depend on acquisition noises.The global pixel fluorograms are indeed composed of a

multitude of radial streaks describing each individual double

positive structure. In the following, we have developed a

local approach using a sliding window to characterize the

dispersion of the cloud-of-dots in second-order histograms.

To find out the correspondences between local fluorescence

distributions, we have compared two methods based on

local image correlation. The first is based on the calculation

on the local correlation coefficient of raw images (CCRI) and

the second on the local joint moment of standardized

images (JMSI). This second-order joint moment, also calledthe noncentral covariance or correlation function (Svedlow

et al., 1978; Anderson, 1984), is computed between the

images using a sliding window after a statistical differencing

filter to standardize the two image data sets (Pratt, 1991).

The local CCRI method consists of computing the correla-

tion coefficient for a sliding window between the two raw

images. In the two methods, detailed in the Appendix, we

obtain a complete correlation map expressing a local co-

localization coefficient for each pixel. Both methods are

suited for registered image pairs.

To illustrate the principle of the JMSI method, we have

chosen a small area of interest of about 100 pixels around a

vesicle in which both fluorophores are co-localized, zoomed

and circled in Fig. 4a. If we form a pixel fluorogram

representation, the corresponding binary cloud-of-dots is

elongated, showing a linear dependence between intensities

and thus a significant correlation (Fig. 4b). To perform a

statistical differencing, we have, respectively, subtracted

from each pixel x(i,j) and y(i,j) of the raw images, the

averaged values x(i,j) and y(i,j) of the surrounding pixels

computed with a 9 9-pixel Gaussian convolution filter

whose standard deviation is set to 1.33 pixel (Fig. 4c) (Pratt,

1991). We have next divided the pixel values of x and y

images by their modified standard deviation jx(i,j) and

jy(i,j) calculated over the same window at the coordinate(i,j) as defined in Eqs. (4)(7) in the Appendix. The local

means and standard deviations of the resulting images are

Fig. 4. Standardization of image data sets. (a) A subregion of Fig. 1b is selected to illustrate the principle of the local joint moment of stan-

dardized images (JMSI). The circle identifies a small co-localized vesicle of about 100 pixel area. Field of view 12.5 12.5 mm. (b) Pixel fluor-

ogram from the circled area of interest in the raw micrograph (a). (c) First step of statistical differencing showing the subtraction of the low-

pass component of each raw image. (d) Second step of statistical differencing showing the normalization of the cloud-of-dots. By dividing each

central image by their modified local standard deviation, the cloud-of-dots has been straightened along the diagonal of the graph.




9/16

normalized (Fig. 4d). As a result of this standardization, the

cloud-of-dots is now straightened along the bisecting line of

the pixel fluorogram. The elongated shape along the

diagonal shows the linear dependence between correlated

intensities in the source image of Fig. 4a. Noncorrelated

structures give rise to cloud-of-dots dispersed in all

directions. To visualize the resulting joint moment, we

have multiplied and locally averaged the two standardized

images. Considering each pixel neighbourhood, this locally

averaged joint moment is sensitive to the shape of the


Fig. 5. Application of local image correlation methods. (a) Correlation maps derived from the local correlation coefficient of raw images

(CCRI), applied to the micrograph shown in Fig. 1a, using a black and white LUT between values 0 and 1 of the CCRI result. (b) Same

as (a) but for the correlation map of the local JMSI method displayed with the same LUT. (c,d) Same micrographs as in Fig. 1b, but the con-

tours of the thresholded correlation maps (a) and (b), respectively, are shown as black lines to reveal co-localized vesicles.



10/16

standardized cloud-of-dots. Highly diagonal clouds give rise

to maximum values of the joint moment.

Detection of co-localized areas with local image correlation

Image correlation maps contain positive and negative areas

depicting positive or inverse correlations between local

fluorescence distributions. The local correlation maps of the

CCRI and of the JMSI methods have been calculated fromthe original images of Fig. 1a (Fig. 5a and 5b, respectively).

Here, we have only displayed the positive part of correlation

maps since it is the most significant one for our applications.

Both methods use local and standardized data sets, and both

correlation maps reveal the co-localized structures of

interest in the raw images. As shown in Fig. 5a, the local

CCRI correlates the edges better than the top areas whose

distributions are less stretched out than the edges. In local

JMSI method, the sliding estimation of local statistical

parameters allows a better discrimination of top areas and

an easier determination of the threshold needed for further

segmentation of co-localized structures (Fig. 5b). To com-

pare the local image correlation maps with the correspond-

ing multifluorescence micrographs, we have first segmented

the co-localized areas from each correlation map (Pal & Pal,

1993). Practically, for each correlation map, we have

extracted regions above a global correlation threshold of

0.5. We have then superimposed in black the contours of

these segmented areas on the raw image of Fig. 1b for the

CCRI and JMSI correlation maps (Fig. 5c and 5d,

respectively). For sake of clarity on the images, we do not

show the contours of correlated objects smaller than 9

pixels. This segmentation allows the extraction of highly

correlated structures whatever their fluorescence intensi-

ties. Some cyan and magenta structures are scored as co-localized indicating a linear dependence between uneven

labelling.

Since the JMSI method reveals the co-localized vesicles

better than the CCRI, it facilitates subsequent segmentation.

In our second example, we have applied the JMSI method to

confocal micrographs from other fibroblast cells having a

higher surface expression of MHC class II molecules labelled

indirectly with Texas Red-coupled antibodies. The second

indirect labelling was carried out with antibodies coupled

with FITC and directed against the cathepsin D lysosomal

enzyme. As in Fig. 5d, we have extracted and superimposed

in black the contours of correlated regions on the raw image

(Fig. 6a). Black-surrounded areas, thresholded from the

local JMSI map, reveal the presence of a collection of co-

localized structures located in the cytoplasm of double

positive cells. Finally, to estimate the correspondences

between local correlation methods and pixel fluorograms,

one can either classify pixels from individual segmented

objects as shown in Fig. 4a or pool the segmented pixels

surrounded in Fig. 6a in an intensity-weighted pixel

fluorogram representation (Fig. 6b). On the other hand,

the pixel fluorogram calculated on noncorrelated areas only

is highly dominated by the background fluorescence

(Fig. 6c). As expected from the correlation method, super-

impositions of nonrelated FITC and Texas Red labelling are

excluded from the correlated areas as shown in the

micrograph (Fig. 6a) and in the intensity-weighted pixel

fluorogram (Fig. 6c).

Fig. 6. Pixel fluorograms and correlated areas. (a) Application of

the JMSI method to identify and surround co-localized structures

in a double immunofluorescence labelling of fibroblasts. FITC-labelled antibodies directed against the lysosomal enzyme cath-

epsin D reveal intracellular vesicles and Texas Red antibodies direc-

ted against mutated MHC class II molecules now label the cell

surface and some internal vesicles. The JMSI correlation map

reveals colocalized vesicles but excludes nonrelated fluorescence

superimposition. Field of view 50 50 mm. (b,c) Intensity-weighted

pixel fluorograms of (b) correlated and (c) noncorrelated areas

defined in (a). The local JMSI correlation map extracts both weakly

and highly labelled structures (b). Double positive pixels that have

not been selected in colocalized areas in (a) are displayed in the

intensity-weighted pixel fluorograms (c) with single positive and

background pixels.




11/16

Discussion

Optical microscope users usually face difficulties during the

course of multifluorescence imaging experiments carried

out with confocal or nonconfocal equipment. Among these,

multicolour visualization, quality control of multifluores-

cence acquisition and finally how to evaluate the statistical

significance of fluorescence co-localization are the main

practical problems. For dual-channel imaging purposes, we

have proposed first to enhance the standard redgreen

superimposition with an extended dual-channel LUT,

thereby allowing a higher discrimination between the two

labels. Second, we have defined pixel fluorograms, which

provide a statistical representation of multiple fluorescence

acquisitions, in a similar way to flow cytometric cytofluoro-

grams. Third, we have developed a local approach for the

detection of colocalization, by estimating the linear depen-

dence between the fluorescence distributions on a sliding

window assuming that intensity vs. binding follows the

same characteristics for the two probes.

Visualization of multiple fluorescence micrographs

For multicolour representations, when the emission spec-

trum of each fluorophore produces a distinct colour which

is close to one of redgreenblue (RGB) primary colours, we

generally associate each fluorescence channel to the RGB

colour plane that best matches the fluorophore emission

colour. This straightforward method produces colour

combinations close to the ones observed through the

microscope oculars provided that the emission energy is

similarly distributed among different fluorescence detectors.

In the other cases, an arbitrary RGB colour plane can be

attributed to each channel. For double-labelling images, it is

possible to improve upon the standard redgreen LUT by

using the third primary colour to create an extended dual-

channel LUT, allowing a higher discrimination of fluores-

cence ratios while conserving the luminance information.

Triple-channel acquisition allows less flexibility. In the

extended dual-channel LUT presented here, five hues define

five sectors of fluorescence ratios, and the luminance is a

function of both fluorescence intensities as defined in the

Results section. We think that our sequence from magenta,

red, yellow, green to cyan enhances our perception of colour

combination, as shown in Fig. 1. Other choices of hue and

luminance formulae can be applied to visualize fluorescence

ratio imaging, such as obtained with Ca2+

or H+

probes(Poenie et al., 1986; Bright et al., 1989). Anyhow, the

interpretation of colour superimposition remains linked to

subjective criteria. The extension of multicolour represen-

tation to 3-D fluorescence data sets is not obvious. Still,

some interesting approaches have been proposed (Van der

Voort et al., 1989; Messerli et al., 1993; Messerli & Perriard,

1995).

Statistical analysis of multichannel images

Despite using an improved LUT, it is generally difficult to

estimate the occurrences of dual fluorescence measure-

ments on multicolour micrographs. To visualize the

fluorescence joint distribution, we have created second-

order pixel fluorograms showing the frequency of registered

pairs of pixels coming from both fluorescence channels. By

defining a threshold for each channel on these graphs, we

can now classify pixels according to their intensities in four

categories: the background, the single-labelled areas, and the

double-labelled areas. To provide a user-friendly correspon-

dence between images and pixel fluorograms, we have

created a colour-coded pixel fluorograms. As a complement

to the spatial representation of images, we think that

second-order pixel fluorograms are appropriate to reveal

intrinsic properties of multifluorescence images that would

otherwise be difficult to infer from the micrographs directly.

The first group of applications concerns the analysis of the

time-dependence of single labelling including the SNR,

fluorescence photobleaching and cell movement detection.

A second group of applications deals with the multicolouranalysis of the channel registration, and measurements of

fluorescence cross-talk and background fluorescence. These

two groups of applications require either time-resolved or

multifluorescence images. Photobleaching and fluorescence

cross-talk can be corrected sequentially depending on the

available data sets provided that single labelled structures

are present in the micrographs.

Owing to the low number of detected photons (Pawley,

1995), the photon statistics have been proven to be the

main limiting factor of the SNR (Young, 1996). As shown in

our results, the pixel fluorograms allow one to visualize the

noise distribution for each grey level from two successively

scanned images. The intrinsic dispersion of the cloud-of-dots

due to the acquisition noise limits the precision with which

two images can be compared. Temporal and/or spatial

filtering techniques improve the SNR, as revealed by pixel

fluorograms constructed from time-averaged images. On the

other hand, a too long or too intense laser excitation of

fluorophores induces a fluorescence attenuation due to

photobleaching, which can also be estimated from pixel

fluorograms. The acquisition noise can be detected as the

width of the diagonal cloud-of-dots, and the photodamage

as a tilt of the pixel distribution. Finally, limited movements

of specimens generate random distributions of points on the

pixel fluorograms.In multifluorescence acquisitions, image registration and

fluorescence cross-talk are the main experimental problems

for which a statistical analysis of multicolour pixels is highly

desirable. Relative distortions between multifluorescence

channels originate from chromatic aberrations in the

objective lenses, in the confocal optics, or from the

movement of the specimen between sequential acquisitions.




12/16

Defect in the colour registration is detected by off-diagonal

dots on pixel fluorograms if the specimen contains sharp

and colocalized fluorescence structures.

To reveal the cross-talk between fluorescence channels,

we have designed intensity-weighted pixel fluorograms.

Nevertheless, the conditions of acquisition should first be

optimized to reduce cross-talk during the acquisition by

adequate selection of laser lines, fluorophores and chro-

matic filters (Mossberg & Ericsson, 1990; Brelje et al.,1993). For given acquisition settings, the cross-talk

coefficients can be evaluated by comparing the two

acquisition channels on second-order pixel fluorograms for

each single-labelled specimen (Mossberg et al., 1990). As

carried out in fluorescence flow cytometry, a real time cross-

subtraction method can be implemented at the level of the

signal acquisition electronics. Furthermore, the autofluor-

escence and/or the nonspecific absorption of fluorescent

antibodies generate a non-negligible part of the signal, in

addition to any instrumental acquisition noise. Our

intensity-weighted pixel fluorograms show that the low

luminance levels constitute the main part of the total image

fluorescence. Special intensity-weighted pixel fluorograms

were designed after offsetting the fluorescence values to

remove a large part of the nonspecific background

fluorescence from the integration. However, the statistical

properties of the noise vary within the observed areas, and it

is generally difficult to associate bins for each component of

the noise in wide-field pixel fluorograms. Therefore, we

recommend others to restrict the area of interest from

which the pixel fluorogram is calculated. Within local areas,

colocalized structures are defined by an elongated cloud-of-

dots, the slope of which is related to the ratio of fluorescence

labelling. A single hue of the extended dual-channel LUT is

attributed to each co-localized distribution, provided thatthe background fluorescence is small enough compared

with the specific fluorescence.

Pixel fluorograms can easily be calculated on serial

optical sections, but again it is preferable to limit the volume

of interest in 3-D data sets. If needed, 3-D representation

techniques could be used to visualize third-order pixel

fluorograms associated with triple fluorescence labelling.

Pixel fluorogram analysis beyond the third-order presents

serious problems of representation and interpretation

(Bonnet et al., 1995).

Detection of co-localization

To detect the local correlations of fluorescence distributions,

we had to consider a collection of neighbouring pixels. A

standardization of the fluorescence values was required to

obtain comparable data sets for each channel despite the

different acquisition conditions. Various advanced methods

have been proposed to compare images, either in pattern

recognition to match a template and a portion of image

(Pratt, 1991) or for the spatial registration of multispectral

or multitemporal satellite images (Svedlow et al., 1978).

Among these, the correlation-based methods prove to be

well-suited to compare registered digital images, or to

determine the spatial shift separating two acquisitions.

Manders et al. (1992, 1993) have applied correlation

methods to find out the degree of correspondence on entire

confocal micrographs. However, this approach does not

provide local information on the sites of colocalization. Toenlighten highly correlated structures in the images, we

have developed two image correlation methods adapted on a

local scale for the detection of fluorescence colocalization.

The methods use standardized data sets to estimate the

linear dependence between the fluorescence distributions.

One is based on the local correlation coefficient between raw

images (CCRI) and the second on the local second-order

joint moment of standardized images (JMSI), also called

noncentral covariance or correlation function (Svedlow et

al., 1978; Anderson, 1984).

To compute the correlation measurements locally, we

have chosen a Gaussian sliding window whose standard

deviation appears to be the main parameter of the methods.

The standard deviation of the Gaussian window should be

large enough to estimate correctly the local statistical

parameters such as the mean and the standard deviation of

pixel values with regard to the optical resolution and the

sampling frequency of the micrograph. Too broad a window

lowers the resolution of the correlation map, hampering the

detection of the small objects. The standard deviation

defining the Gaussian window should be chosen according

to the size of objects to be colocalized, the optical resolution

and the sampling rate. We chose a 1.33-pixel standard

deviation on a 9 9-pixel window containing 99.9% of the

Gaussian distribution, which turned out to be large enoughto estimate local statistical parameters while conserving a

high resolution in the correlation map. A compromise

should be found between spatial resolution and validity of

statistical measurements. Using standardized data sets, the

correlation methods can reveal the correlation of very

weakly labelled areas. Indeed, we noted that the autofluor-

escence of the specimen as well as a part of the nonspecific

absorption of antibodies give rise to highly correlated

structures due to the normalization by very small standard

deviations. To prevent these nonsignificant outputs when

the variance of very flat areas is low, we have added an

additional constant to the variance of each channel, which

modifies the denominator of the correlation formulae. In

statistical differencing a constant is added to the denomin-

ator (Pratt, 1991). This additional constant can be also

interpreted as the variance of a noncorrelated noise added

to each channel. This noise constant can be adjusted to

drown any type of weak correlated structures (autofluores-

cence or correlated components of weak labelling). In our

256-grey-level images, we have added a uniform noise




13/16

whose standard deviation was set to four quantization levels

for both channels. The photon statistics and the dark

current give typical noncorrelated noises, but these were

too small to mask nonsignificant structures in the low

fluorescence areas of the micrographs. As expected, the

fluorescence cross-talk increases the correlation measure-

ments and should be compensated for before computing the

correlation maps. In addition to pixel fluorograms, image

correlation maps can be used to adjust acquisition settings,to eliminate fluorescence cross-talk and to optimize image

registration locally. Among the two methods described in

the Results and the Appendix, the JMSI method better

defines the tops of vesicular structures. Since this method

uses an image-orientated way of standardizing the data sets

called statistical differencing (Pratt, 1991), it provides

intermediate images with normalized fluorescence distri-

butions. Except for very low signals, for which local

variance is covered by the noise constant, both methods

detect colocalization independently of the local mean of the

signal. Double positive pixels which do not correspond to a

local covariation of fluorescence distributions are not

revealed by these methods (White et al., 1996), as shown

in Fig. 6. Local image correlation excludes the super-

imposition of flat fluorescent profiles when a high spatial

resolution is required for the analysis. At low spatial

resolution, flat areas can be considered if they are smaller

than the local scanning window. We think that the

identification of colocalized structures is meaningful only

for a specified scale. Accordingly, unknown structures of

any size can be compared with known fluorescence

elements.

Perspectives in image cytometry

Image segmentation is required to carry out fluorescence

measurements on recognized objects, and its quality

depends on the selection of the attributes. The intensity,

together with the variance and sometimes textural attri-

butes, have been used to achieve improved segmentations

(Santisteban & Brugal, 1995; Wu et al., 1995). Our two

concurrent local correlation measurements prove to be

appropriate attributes for multicolour image segmentation

and are more discriminative than the double thresholding of

raw pixel values. The slope of the correlated cloud-of-dots

(Fig. 4a) reveals the ratio between fluorescence distributions

on a local window and characterizes the association of

markers. We propose to use the local correlation measure-

ment as a basic attribute for further classifications and

analyses in multifluorescence image subcellular cytometry.

The identification of segmented objects will allow their

classification according to different parameters such as their

size or fluorescence attributes, as performed at the cellular

level in conventional fluorescence microscopy (Galbraith et

al., 1991).

The extension of local correlation methods to serial

confocal sections requires the use of 3-D Gaussian windows.

The fluorescence attenuation could then automatically be

corrected by the standardization of the data. Independent

methods for the correction of this attenuation have been

proposed (Rigaut & Vassy, 1991; Visser et al., 1991;

Strasters et al., 1994; Liljeborg et al., 1995). To apply

local image correlation methods to triple labelled specimens

in a basic way, we propose to consider the three channelstwo-by-two, and then finally to superimpose the three maps

by associating each of them to a RGB colour plane.

Acknowledgments

This work was supported by the Centre National de la

Recherche Scientifique (CNRS), the Institut National de la

Sante et de la Recherche Medicale (INSERM) and the

Association pour la Recherche contre le Cancer (ARC). D.D.

is the recipient of a predoctoral BDI fellowship from the

CNRS and the Conseil Regional Provence-Alpes-Cotes-

dAzur. We wish to thank Dr Jacques Barbet (Centre

dImmunologie de Marseille-Luminy, France) and especially

Dr David Shotton (University of Oxford, U.K.) for critical

reading of the manuscript. We also thank Dr Graca Raposo

(Institut Curie, Paris, France) and Dr Patrizia Rovere (Centre

dImmunologie de Marseille-Luminy, France) for the pre-

paration of various specimens and especially Mr Nicolas

Barois (Centre dImmunologie de Marseille-Luminy, France)

for providing us with rabbit polyclonal anti-Ii chain

antibodies.

References

Adams, S.R., Harootunian, A.T., Buechler, Y.J., Taylor, S.S. & Tsien,R.Y. (1991) Fluorescence ratio imaging of cyclic AMP in single

cells. Nature, 349, 694697.

Akinyemi, O., Boyde, A., Browne, M.A., Hadravsky, M. & Petran, M.

(1992) Chromatism and confocality in confocal microscopes.

Scanning, 14, 136143.

Akner, G., Mossberg, K., Wikstrom, A.C., Sundqvist, K.G. &

Gustafsson, J.A. (1991) Evidence for colocalization of glucocor-

ticoid receptor with cytoplasmic microtubules in human gingival

fibroblasts, using two different monoclonal anti-GR antibodies,

confocal laser scanning microscopy and image analysis. J. Steroid

Biochem. Mol. Biol., 39, 419432.

Anderson, T.W. (1984) An Introduction to Multivariate Statistical

Analysis. Wiley-Interscience, New York.

Arndt-Jovin, D.J., Robert-Nicoud, M. & Jovin, T.M. (1990) ProbingDNA structure and function with a multi-wavelength fluores-

cence confocal laser microscope. J. Microsc. 157, 6172.

Bonnet, N., Herbin, M. & Vautrot, P. (1995) Extension of the

scatterplot approach to multiple images. Ultramicroscopy, 60,

349355.

Brakenhoff, G.J., Van der Voort, H.T.M., Van Spronsen, E.A. &

Nanninga, N. (1989) Three-dimensional imaging in fluorescence

by confocal scanning microscopy. J. Microsc. 153, 151159.




14/16

Brakenhoff, G.J., Visscher, K. & Gijsbers, E.J. (1994) Fluorescence

bleach rate imaging. J. Microsc. 175, 154161.

Brelje, T.C., Wessendorf, M.W. & Sorenson, R.L. (1993) Multicolor

laser scanning confocal immunofluorescence microscopy:

practical application and limitations. Methods Cell Biol. 38,

97181.

Bright, G.R., Fisher, G.W., Rogowska, J. & Taylor, D.L. (1989)

Fluorescence ratio imaging microscopy. Methods Cell Biol. 30,

157192.

Carlsson, K., Aslund, N., Mossberg, K. & Philip, J. (1994)Simultaneous confocal recording of multiple fluorescent

labels with improved channel separation. J. Microsc. 176,

287299.

Carlsson, K. & Mossberg, K. (1992) Reduction of cross-talk between

fluorescent labels in scanning laser microscopy. J. Microsc. 167,

2337.

Dauwerse, J.G., Wiegant, J., Raap, A.K., Breuning, M.H. & Van

Ommen, G.J.B. (1992) Multiple colors by fluorescence in situ

hybridization using ratio-labelled DNA probes create a molecular

karyotype. Hum. Mol. Genet. 1, 593598.

Demandolx, D. & Davoust, J. (1995) Multicolor analysis in confocal

immunofluorescence microscopy. J. Trace Microprobe Tech. 13/3,

217225.

Dutartre, H., Davoust, J., Gorvel, J.P. & Chavrier, P. (1996)

Cytokinesis arrest and redistribution of actin-cytoskeleton

regulatory components in cells expressing the Rho GTPase

CDC42Hs. J. Cell Sci. 109, 367377.

Fox, M.H., Arndt-Jovin, D.J., Jovin, T.M., Baumann, P.H. &

Robert-Nicoud, M. (1991) Spatial and temporal distribution of

DNA replication sites localized by immunofluorescence and

confocal microscopy in mouse fibroblasts. J. Cell Sci. 99, 247

253.

Galbraith, W., Wagner, M.C.E., Chao, J., Abaza, M., Ernst, L.A.,

Nederlof, M.A., Hartsock, R.J., Taylor, D.L. & Waggoner, A.S.

(1991) Imaging cytometry by multiparameter fluorescence.

Cytometry, 12, 579596.

Humbert, C., Santisteban, M.S., Usson, Y. & Robert-Nicoud, M.(1992) Intranuclear co-location of newly replicated DNA and

PCNA by simultaneous immunofluorescent labelling and con-

focal microscopy in MCF-7 cells. J. Cell Sci. 103, 97103.

Humbert, M., Raposo, G., Cosson, P., Reggio, H., Davoust, J. &

Salamero, J. (1993) The invariant chain induces compact forms

of class II molecules localized in late endosomal compartments.

Eur. J. Immunol. 23, 31583166.

Keller, H.E. (1995) Objective lenses for confocal microscopy.

Handbook of Biological Confocal Microscopy (ed. by J. B. Pawley),

pp. 111126. Plenum Press, New York.

Leger, I., Guillaud, M., Krief, B. & Brugal, G. (1994) Interactive

computer-assisted analysis of chromosome 1 colocalization with

nucleoli. Cytometry, 16, 313323.

Liljeborg, A., Czader, M. & Porwit, A. (1995) A method to

compensate for light attenuation with depth in 3-D DNA image

cytometry using a confocal scanning laser microscope. J.

Microsc. 177, 108114.

Manders, E.M.M., Stap, J., Brakenhoff, G.J., Van Driel, R. & Aten,

J.A. (1992) Dynamics of three-dimensional replication patterns

during the S-phase, analysed by double labelling of DNA and

confocal microscopy. J. Cell Sci. 103, 857862.

Manders, E.M.M., Verbeek, F.J. & Aten, J.A. (1993) Measurement of

co-localization of objects in dual-colour confocal images. J.

Microsc. 169, 375382.

Messerli, J.M. & Perriard, J.C. (1995) Three-dimensional analysis

and visualization of myofibrillogenesis in adult cardiomyo-

cytes by confocal microscopy. Microsc. Res. Technique, 30, 521

530.

Messerli, J.M., Van der Voort, H.T.M., Rungger-Brandle, E. &

Perriard, J.C. (1993) Three-dimensional visualization of multi-

channel volume data: the amSFP algorithm. Cytometry, 14,725735.

Mossberg, K., Arvidsson, U. & Ulfhake, B. (1990) Computerized

quantification of immunofluorescence-labeled axon terminals

and analysis of colocalization of neurochemicals in axon

terminals with a confocal scanning laser microscope. J.

Histochem. Cytochem. 38, 179190.

Mossberg, K. & Ericsson, M. (1990) Detection of doubly stained

fluorescent specimens using confocal microscopy. J. Microsc.

158, 215224.

Nederlof, P.M., Van der Flier, S., Verwoerd, N.P., Vrolijk, J., Raap,

A.K. & Tanke, H.J. (1992a) Quantification of fluorescence in situ

hybridization signals by image cytometry. Cytometry, 13, 846

852.

Nederlof, P.M., Van der Flier, S., Vrolijk, J., Tanke, H.J. & Raap, A.K.

(1992b) Fluorescence ratio measurements of double-labeled

probes for multiple in situ hybridization by digital imaging

microscopy. Cytometry, 13, 839845.

Nederlof, P.M., Van der Flier, S., Wiegant, J., Raap, A.K., Tanke, H.J.,

Ploem, J.S. & Van der Ploeg, M. (1990) Multiple fluorescence in

situ hybridization. Cytometry, 11, 126131.

Paddock, S.W. (1995) Imaging drosophila development with the

laser scanning confocal microscope. Microsc. Anal. 34, 911.

Paddock, S.W., Langeland, J.A., DeVries, P.J. & Carroll, S.B. (1993)

Three-color immunofluorescence imaging of Drosophila embryos

by laser scanning confocal microscopy. BioTechniques , 14, 42

48.

Pal, N.R. & Pal, S.K. (1993) A review on image segmentationtechniques. Pattern Recogn. 26, 12771294.

Pawley, J.B. (1995) Fundamental limits in confocal microscopy.

Handbook of Biological Confocal Microscopy (ed. by J. B. Pawley),

pp. 1937. Plenum Press, New York.

Poenie, M., Alderton, J., Steinhardt, R. & Tsien, R. (1986) Calcium

rises abruptly and briefly throughout the cell at the onset of

anaphase. Science, 233, 886889.

Pollerberg, G.E., Schachner, M. & Davoust, J. (1986) Differentiation

state-dependent surface motilities of two forms of the neural cell

adhesion molecule. Nature, 324, 462465.

Pratt, W.K. (1991) Digital Image Processing, 2nd edn. Wiley-

Interscience, New York.

Rigaut, J.P. & Vassy, J. (1991) High-resolution three-dimensional

images from confocal scanning laser microscopy. Quantitative

study and mathematical correction of the effets from bleaching

and fluorescence attenuation in depth. Anal. Quant. Cytol. Histol.

13, 223232.

Salamero, J., Humbert, M., Cosson, P. & Davoust, J. (1990) Mouse B

lymphocyte specific endocytosis and recycling of MHC class II

molecules. EMBO J. 9, 34893496.

Sandison, D.R. & Webb, W.W. (1994) Background rejection and




15/16

signal-to-noise optimization in confocal and alternative fluores-

cence microscopes. Appl. Opt. 33, 603615.

Sandison, D.R., Williams, R.M., Sam Wells, K., Strickler, J. & Webb,

W.W. (1995) Quantitative fluorescence confocal laser scanning

microscopy (CLSM). Handbook of Biological Confocal Microscopy

(ed. by J. B. Pawley), pp. 3953. Plenum Press, New York.

Santisteban, M.S. & Brugal, G. (1995) Fluorescence image analysis

of the MCF-7 cycle related changes in chromatin texture.

Differences between AT- and GC-rich chromatin. Anal. Cell.

Pathol. 9, 1328.Schubert, W. (1991) Triple immunofluorescence confocal laser

scanning microscopy: spatial correlation of novel cellular

differentiation markers in human muscle biopsies. Eur. J. Cell

Biol. 55, 272285.

Sheppard, C.J.R. (1989) Axial resolution of confocal fluorescence

microscopy. J. Microsc. 154, 237242.

Sheppard, C.J.R. & Cogswell, C.J. (1990) Three-dimensional image-

formation in confocal microscopy. J. Microsc. 159, 179194.

Sheppard, C.J.R., Gu, M. & Roy, M. (1992) Signal-to-noise ratio in

confocal microscope systems. J. Microsc. 168, 209218.

Shotton, D.M. (1989) Confocal scanning optical microscopy and

its applications for biological specimens. J. Cell Sci. 94, 175

206.

Strasters, K.C., Van der Voort, H.T.M., Geusebroek, J.M. &

Smeulders, A.W.M. (1994) Fast attenuation correction in

fluorescence confocal imaging: a recursive approach. Bio-

imaging, 2, 7892.

Svedlow, M., McGillem, C.D. & Anuta, P.E. (1978) Image

registration: Similarity measure and preprocessing method

comparisons. IEEE Trans. Aerospace and Electronic Systems, 14,

141149.

Tsien, R.Y. & Waggoner, A. (1995) Fluorophores for confocal

microscopy. Handbook of Biological Confocal Microscopy (ed. by J.

B. Pawley), pp. 267279. Plenum Press, New York.

Van der Voort, H.T.M., Brakenhoff, G.J. & Baarslag, M.W. (1989)

Three-dimensional visualization methods for confocal

microscopy. J. Microsc. 153, 123132.Van der Voort, H.T.M., Messerli, J.M. & Noordmans, H.J. (1993)

Volume visualisation for interactive microscopical image

analysis. Bioimaging, 1, 2029.

Visscher, K., Brakenhoff, G.J. & Visser, T.D. (1994) Fluorescence

saturation in confocal microscopy. J. Microsc. 175, 162165.

Visser, T.D., Groen, F.C.A. & Brakenhoff, G.J. (1991) Absorption and

scattering correction in fluorescence confocal microscopy. J.

Microsc. 163, 189200.

White, J.G., Amos, W.B. & Fordham, M. (1987) An evaluation of

confocal vs. conventional imaging of biological structures by

fluorescence light microscopy. J. Cell Biol. 105, 4148.

White, N.S. (1995) Visualisation systems for multidimensional

confocal microscopy data. Handbook of Biological Confocal

Microscopy (ed. by J. B. Pawley), pp. 211254. Plenum Press,

New York.

White, N.S., Errington, R.J., Fricker, M.D. & Wood, J.L. (1996)

Aberration control in quantitative imaging of botanical speci-

mens by multidimensional fluorescence microscopy. J. Microsc.

181, 99116.

Wijnaendts-van-Resandt, R.W., Marsman, H.J.B., Kaplan, R.,

Davoust, J., Stelzer, E.H.K. & Stricker, R. (1985) Optical

fluorescence microscopy in three dimensions: microtomoscopy.

J. Microsc. 138, 2934.

Wilson, T. (1990) Confocal microscopy. Confocal Microscopy (ed. by

T. Wilson), pp. 164. Academic Press, London.

Wu, K.H., Gauthier, D. & Levine, M.D. (1995) Live cell image

segmentation. IEEE Trans. Biomed. Eng. 42, 112.

Young, I.T. (1988) Sampling density and quantitative microscopy.

Anal. Quant. Cytol. Histol. 10, 269275.

Young, I.T. (1996) Quantitative microscopy. IEEE Eng. Med. Biol.

Mag. 15, 5966.

Appendix: the local image comparison methods

The correlation coefficient of raw images

Considering a local area of interest in the pair of

fluorescence micrographs, the correlation coefficient esti-

mates the linear dependencies between the two pixel value

distributions independently of background fluorescence

levels. The correlation coefficient of raw images (CCRI),

given in the correlation formula (3), is computed on sliding

windows across the whole image field, i.e. for each pixelneighbourhood, to produce a complete image correlation

map. All means on local windows are computed using a

Gaussian low-pass filter.

CCRI xy xy

jxjy; 3

where x and y denote the digital fluorescence images as

arrays of pixel values for both images. The image arrays j2xand j2y denote the modified local variances as indicated in

Eqs. (4) and (5). All arithmetical operations are calculated

pixel-to-pixel between pairs of images. The bar () indicates

an image operation where each pixel value is replaced bythe averaged value of its neighbourhood. For example, the

image xy is the result of the image product between x and y

convoluted by a Gaussian convolution filter, whereas xy is

the pixel-to-pixel product between the filtered images x and

y. In our example the experimental averages () are all

calculated by smoothing the input images through a 9 9-

pixel Gaussian convolution filter whose standard deviation

is set to 1.33 pixel. The variances j2x and j2

y have been

modified by adding the constants ax and ay (Eqs. 4 and 5) to

prevent the generation of nonsignificant correlation coef-

ficient values due to very flat areas with small variances.

The constants ax and ay can be also interpreted asvariances of noncorrelated noises added to the images. axand ay can be set to zero to obtain the exact variances and

the exact correlation coefficient varying from 1 to 1.

j2x x2 x 2 ax 4

j2y y2 y

2 ay: 5




16/16

The joint moment of standardized images

In an alternative approach, we applied a statistical

differencing (Pratt, 1991) to locally standardize the mean

and the variance in the images. The standardized images

are then compared by the calculation of their local second-

order joint moment (Anderson, 1984). The statistical

differencing of images allows the generation of locally

standardized images successively by subtracting the locally

averaged image x from the raw image x and dividing the

result by the standard deviation image jx defined above. The

overall operator is defined by

xs x x

jx6

ys y y

jy: 7

The notations are the same as for Eqs (3), (4) and (5). All

the means () are computed using the Gaussian convolution

filter defined above.

Having defined two standardized image data sets xs and

ys, we can compute their joint moment between sliding

windows across the whole image field. This second-order

joint moment between standardized images (JMSI) is theaveraged value of the product between images xs and ys:

JMSI xsys: 8

The JMSI values are not necessarily between 1 and 1 but

are usually in this range.



multicolour image analysis

Documents