Real-time digital aperture sampling sensor

Anthony M. Tai and Jack N. Cederquist

In real-time speckle imaging and speckle interferometry, there is often the need to remove a nonuniformincoherent background from the detected image to enhance the embedded coherent speckle image. Previousapproaches generally require the use of a reference beam which makes the system very sensitive to mechanicalinstability. A new technique that operates on uncorrelated speckle images obtained by aperture sampling isproposed. The design of a real-time system for digital speckle image processing is described, and theexperimental results are presented. Possible applications for the system on speckle interferometry arediscussed.

1. Introduction

In many speckle imaging and speckle interferometryapplications, there is a need to separate the coherentspeckle image from a nonuniform incoherent back-ground. The incoherent field may be generated by anincoherent illuminating source, or it may be due to thedecorrelation of a coherent incident field by the re-flecting surface. Previous approaches generally in-volve the use of a coherent reference beam, and, as aresult, they are very sensitive to mechanical vibrationand air turbulence.Y-3 In this paper, an aperture sam-pling technique is introduced for the removal of thenonuniform incoherent background from a detectedimage to enhance the embedded coherent speckle im-age. Since a reference beam is not used, the system issimpler to construct, and it is much more resistant tosystem instabilities. The simplicity of the approachalso makes it easy to fabricate a real-time system thatis capable of processing images at video frame rates.

In the following section, previous processing ap-proaches are described. The proposed technique ispresented in Sec. III, and possible applications for thesystem are discussed in Sec. IV. They include time-averaged speckle interferometry for vibration analysis,imaging through a dynamic scattering medium, andthe generation of surface roughness maps. The imple-

The authors are with Optical Science Laboratory, Advanced Con-cepts Division, Environmental Research Institute of Michigan, P.O.Box 8618, Ann Arbor, Michigan 48107.

Received 20 March 1987.0003-6935/87/235130-06$02.00/0.© 1987 Optical Society of America.

mentation of a real-time system is then described, andexperimental results are shown in Sec. V. A summaryof the results is in Sec. VI.

II. Previous Approaches

As stated earlier, the image processing required inmany real-time speckle interferometry applications isthe removal of the incoherent component of the de-tected light field to enhance the cohereht componentforming the speckle image. This incoherent compo-nent may be introduced externally to the system, or itmay be due to the decorrelation of a coherent objectfield by, for example, surface motion. When the back-ground bias corresponding to the incoherent inputfield is uniform, its removal simply involves the sub-traction of a constant value from the image. If thebackground bias is nonuniform, the intensity distribu-tion must be determined before it can be effectivelyremoved. For the case where the background biasvaries slowly compared to the speckle intensity distri-bution, simple high-pass filtering may be satisfactory.4However, when the spatial frequency distributions ofthe speckle and the incoherent background overlapwith each other, simple filtering cannot effectively re-move all the background from the detected image.

A more effective approach is to take advantage of thedifference in coherence between the coherent and in-coherent components in the detected image. Onecommon method is to interfere the detected imagewith a coherent reference beam and then remove allthe self-interfering terms in the processing to enhancethe mutually interfering cross terms. This is generallyaccomplished by acquiring and digitizing two framesof image data between which a r phase shift is intro-duced in the reference beam. The phase shift has noeffect on the portion of the detected image contributed

5130 APPLIED OPTICS Vol. 26, No. 23 / 1 December 1987

by the incoherent light field, but it inverts the coherentspeckle image. Constructive interference between ar-eas of the coherent speckle image and the referencebeam becomes destructive after the r phase shift.Subtracting the two frames digitally removes the self-interfering terms and thus incoherent backgroundwhich is unchanged between the two frames. Whatremains is the cross terms corresponding only to thecoherent portion of the detected image field. Becausedigital processing is employed, such an approach isreferred to as digital speckle-pattern interferometry(DSPI).1-3 One drawback of using a reference beam isthe system sensitivity to mechanical stability. Precisefringe monitoring and stabilization are often required.The aperture sampling technique that is described inthe next section accomplishes the incoherent back-ground removal without the need for a coherent refer-ence beam and thereby avoids the stability problem.

Ill. Aperture Sampling Technique

We want to determine the conditions for creatingtwo speckle patterns which, on subtracting the intensi-ty of one from the other, produce an image with thehighest possible SNR. First, we note that the sameconditions will also result in the lowest possible imagecontrast when the two speckle images are added to-gether by their intensities. Thus the optimum tech-nique for speckle reduction is also optimum for specklesubtraction to enhance a speckle image embedded in anoncoherent background. From Goodman,5 we knowthat adding two speckle patterns that are completelyuncorrelated by their intensities produces an imagewith the lowest possible contrast. We can, therefore,conclude that the best SNR is obtained by subtractingtwo uncorrelated speckle patterns. Borrowing alsofrom techniques for speckle reduction, one simple wayto generate uncorrelated speckle patterns is to sampledisjoint portions of the input field at the lens aperture.

For an object plane that is sufficiently far from theimaging lens, the light field at the collecting aperturecorresponds to the complex spatial spectrum of thefield scattered by the object scene. By using differentportions of the input field, different bands of the spec-trum are sampled. The light field at the lens apertureis uncorrelated over a spatial distance of XR/D (the sizeof a speckle) where X is the illumination wavelength, Ris the object distance, and D is the diameter of theilluminated patch. The speckle images formed withdifferent portions of input light field which are disjointand separated from each other by more than a specklewidth are uncorrelated. Subtracting the intensities oftwo uncorrelated speckle images produces a bipolaroutput. To display the image, the resulting image canbe rectified (i.e., its absolute value taken). We findthat for the polarized case, the image obtained bysubtracting two uncorrelated speckle images and recti-fying the result has the same intensity statistic as aregular speckle pattern (see Appendix A). The back-ground bias and the detector pattern noise, on theother hand, are unaffected by the shifting of the imag-ing aperture, and they are removed in the subtraction

Probability

1.0

Polarized Speckle Patternand Processed Speckle Pattern

Processed Unpolarized Speckle Pattern

0.5Unpolarized Speckle Pattern

0 0.5 1

Relative Pixel Intensity

Fig. 1. Probability density distributions of processed polarized andunpolarized speckle patterns.

of the two image frames (assuming that the amount ofaperture shift is small compared with target distance,and parallax effects are insignificant).

In the unpolarized case, the probability density dis-tribution of the resulting pattern is different from thatof a regular unpolarized speckle pattern (see AppendixB). However, the mean value of the processed image isstill proportional to the mean of the coherent portionof the original detected image. For both the polarizedand unpolarized cases and for both fully developed andpartially developed speckle, the standard deviation ofa speckle pattern is proportional to the mean. There-fore, the mean of the processed image is proportionalto the standard deviation of the original detected im-age. If the reflectance of the target is uniform (i.e., themean of the image intensity is uniform), the mean ofthe processed image is also proportional to the specklecontrast of the detected image. The probability den-sity distributions of the original and the processedspeckled images for the polarized and unpolarizedcases are plotted in Fig. 1.

IV. Applications

In many speckle imaging and speckle interferometryapplications, the image processing problem can be re-duced to the removal of the incoherent backgroundbias in the detected image to enhance the embeddedcoherent speckle image. One example is the use oftime-averaged speckle interferometry to study themodal characteristics of mechanical vibration.4 Mov-ing surfaces decorrelate the scattered field, and thecorresponding image exhibits little speckle contrast.The surface areas corresponding to the nodes of thevibration modes remain stationary and form high-con-trast speckles in the image. The areas correspondingto the moving parts of the vibrating surface thus be-come black after the processing, and a high-contrastimage showing the vibrational modes is produced.

Another interesting application is the real-time gen-eration of surface roughness maps. It is well knownthat speckle contrast is directly related to the rough-ness (surface height variation) of the scattering surfacefor an object under spectrally wideband illumination.6When the surface height variation within a resolutioncell is larger than the coherence length of the illumi-

1 December 1987 / Vol. 26, No. 23 / APPLIED OPTICS 5131

nating source, the field forming the image becomespartially decorrelated. The degree of coherence asmeasured by speckle contrast is, therefore, a functionof the magnitude of the surface height variation. Theintensity of an image processed by the aperture sam-pling technique is directly proportional to the specklecontrast. Thus, for an object scene with uniform re-flectance, the intensity distribution of the processedimage is a direct representation of the surface rough-ness distribution in the object scene.

Other applications include speckle imaging of theobject under strong incoherent illumination (e.g., un-der direct sunlight) where spectral filtering alone is notsufficient to remove the incoherent background andthe imaging of a coherently illuminated target througha dynamic scattering medium. In both cases, the co-herent image is buried in an incoherent backgroundbias. A system utilizing the aperture sampling tech-nique can be used to extract the embedded coherentspeckle image from the nonuniform background.

V. Experiments

The simplicity of the aperture sampling approachmakes it easy to fabricate a real-time system. Ourimplementation is illustrated in Fig. 2. A FairchildCCD camera was used to detect the input image, and aHeurikon 68000-based microcomputer with ITI videoprocessing boards was used to perform the image ac-quisition, subtraction, and rectification. To changethe position of the imaging aperture between twoframes, the 60-Hz vertical sync pulses of the videocamera were used as the clock input to a two-countcounter producing a synchronous 30-Hz square wave.The square wave was then used to drive a galvanom-eter scanner which moved a rectangular apertureacross the imaging lens.

Alternate frames were subtracted and then rectifiedthrough a look-up table. The processed image wasupdated 15 times a second. To achieve continuousimage display, two video frame buffers were utilized.The system first acquired and stored one image frame,moved the aperture mask, subtracted the next framefrom the stored data, and displayed the image contentwith the proper look-up table to achieve rectification.(If desired, the subtracted and rectified image can alsobe amplified by simply left shifting the image bits.)While the image content of this first video frame bufferwas being displayed, the next two image frames wereacquired and processed and stored in the second videoframe buffer. After 1 /15 s, the display pointer wasswitched to the second video frame buffer to displaythe new processed image, and the first video framebuffer was freed to process the next pair of frames.

To test the effectiveness of the technique in enhanc-ing a speckle image embedded in an incoherent back-ground bias, we illuminated a diffuse 3-D target withlaser light. The light level was adjusted so that theilluminated target was just bright enough to be seen bythe video camera as shown in Fig. 3(a). A strongnonuniform incoherent background was then added byilluminating the detector array with an incandescent

Processed Image Line ImageDisplay Display

Fig. 2. Real-time aperture sampling digital speckle-pattern inter-ferometer.

(a)

(C)

Fig. 3. Enhancement of a coherent speckle image embedded in anonuniform incoherent background: (a) direct speckle image, (b)direct speckle image embedded in nonuniform incoherent back-ground; (c) enhanced speckle image obtained with aperture sam-

pling technique.

light. The brightness of the background light wasadjusted so that the brightest part nearly saturated thedetector. The resulting image is shown in Fig. 3(b).Due to this background light, the contrast of the speck-le image was so low that it was not visible on the CRTdisplay. In Fig. 3(c), we show the processed imageobtained with the aperture sampling technique. Theincoherent background was completely removed, and ahigh-contrast image of the hidden object was dis-played.

5132 APPLIED OPTICS / Vol. 26, No. 23 / 1 December 1987

(a)

(c)

Fig. 4. Generation of a surface roughness map:(a) image of object under incoherent illumination;(b) image of object under partially coherent illumi-nation; (c) image of object under partially coherentillumination with the imaging aperture moved to a

(d) new position; (d) processed image obtained by sub-tracting (b) from (c) and rectifying the result.

The system was then used in speckle interferometryapplications. To demonstrate the use of the systemfor the generation of surface roughness maps, a smoothobject on a rough background was created by pasting acutout from an index card on a piece of foam. Thesurface of the foam had many tiny pores while thesurface of the index card was quite smooth. A dyelaser with no internal cavity or dispersive element wasused as the illuminating source. The spectral band-width of the illuminating beam was 2 nm, which pro-vided a longitudinal coherence length of -150 ,um.Figure 4(a) shows the image of the object under inco-herent illumination. The index card cutout and thefoam were both white, and the contrast of the incoher-ent image was very low. The image under the illumi-nation of the dye laser is shown in Fig. 4(b). Thecoherence length of the laser was much shorter thanthe surface height variation of the foam, and the scat-tered light was decorrelated. The speckle contrast inthe resulting image is very low. The index card, on theother hand, was smooth compared to the coherencelength of the laser, and high-contrast speckles wereformed in the image of the paper cutout. Figure 4(c) isthe image obtained after moving the imaging apertureto a new position. The speckle pattern was changedwhile the decorrelated noncoherent background wasunaffected. Subtracting the two images and rectify-ing the result produced the image in Fig. 4(d). Thepart of the object field with smooth surfaces appeared

bright in the processed image while the areas withrough surfaces appeared dark. To generate a surfaceroughness map, the processed image can be normalizedby the incoherent image to remove the effect of differ-ing surface reflectances. The resulting image bright-ness is then directly proportional to the speckle con-trast, which in turn is determined by the magnitude ofthe surface height variations. We may note that theconventional approach utilizing a reference beam can-not be used in this application or in any situationwhere the depth of the object is longer than the coher-ence length of the laser source. Thus, the aperturesampling technique not only provides higher toleranceto system instability, it can also be applied in somesituations not possible with conventional DSPI.

To illustrate its use in time-average speckle interfer-ometry, a piece of plastic film was mounted on a ringand excited by a loudspeaker. The film was illuminat-ed by an argon laser and image by our aperture sam-pling system. Figure 5(a) shows the image obtainedwith a stationary imaging aperture. The changes inspeckle contrast corresponding to the vibration modesare difficult to see. The image obtained with theaperture sampling technique is given in Fig. 5(b). Theplastic film was not mounted with sufficient rigidity onthe ring, and the vibration modes were not very dis-tinct. Nevertheless, the nodes of the vibration areclearly visible.

1 December 1987 / Vol. 26, No. 23 / APPLIED OPTICS 5133

(b)

VI. Summary

In many speckle imaging and speckle interferometryapplications, image processing is required to remove anonuniform incoherent (decorrelated) backgroundfrom the detected image to enhance the embeddedcoherent speckle image. Simple filtering is ineffectiveif the spatial frequency distributions of the speckleimage and the incoherent background overlap sub-stantially with each other. The previous processingapproach which utilizes a coherent reference beammakes the system overly sensitive to mechanical insta-bility and air turbulence.

We have shown that by subtracting two uncorrelat-ed speckle images of an object scene obtained throughdisjoint imaging apertures and rectifying the result, anew speckle image is formed. The mean of the newspeckle pattern is proportional to the mean and conse-quently to the standard deviation of the original speck-le pattern. For the case where the speckle pattern isembedded in a bias due to the decorrelation of part ofthe object field or the introduction of an incoherentcomponent, the incoherent portion of the image field isremoved in the subtraction of the two image framessince it is unaffected by shifting of the image aperture.Therefore, for an object of uniform reflectance, themean of the processed image is also proportional to thespeckle contrast of the detected image.

In gaining system tolerance to turbulence and me-chanical instability, however, some versatility is lost.The use of synchronous phase modulation of the refer-ence beam,7 8 for example, cannot be utilized with theaperture sampling technique to improve system sensi-tivity. On the other hand, the proposed technique canbe used with large objects whose depths are muchlonger than the coherence length of the laser source aswith the case of surface roughness measurement.Moreover, the inherent simplicity of the proposed sys-tem makes it very attractive in field applications whereportability and ruggedness are imperative. Similar tosome speckle shearing interferometric techniques, ourproposed approach also utilizes multiple apertures.But, unlike conventional speckle shearing interferom-etry, the apertures do not exist at the same time. Theproposed technique is fundamentally different fromother speckle interferometric techniques but the func-tions it can perform are quite similar to that of conven-tional DSPI systems.

Possible uses of the aperture sampling sensor in-clude the generation of surface roughness maps, time-averaged speckle interferometry, and the enhance-ment of speckle images obtained in unfavorableconditions. A real-time implementation of the tech-nique was described, and experimental results demon-strating its applications in speckle interferometry werepresented.

The authors gratefully acknowledge the assistanceof K. Ellis in performing the experiments.

Appendix A

The probability density distribution of the intensity

(a)

(b)

Fig. 5. Displaying vibration modes of a mounted plastic film: (a)direct speckle image; (b) processed image.

of a speckle pattern formed by polarized light is givenby 5

P1(i) = I exp(-i/A),A

(Al)

where A is the mean intensity of the speckle pattern.Let us compute the probability density distribution ofan image obtained by subtracting two uncorrelatedspeckle images with the same statistical properties andrectifying the result; that is, if I and 12 are randomvariables corresponding to the intensities of the twouncorrelated speckle patterns, we are to determine theprobability density distribution of I, where I = 2 - I.The value of I, like I and 12, will take on only positivevalues.

I is defined within a sample space S which can bedecomposed into two subsets or events; E': i2 i andE" i > i2. Since I and 2 are identically distributed,E' and E" are equally likely to occur.

Thus

P1[E'] = P[E"] = /2,

P,[S] = P,[E'n E"] = 1. (A2)

The probability density distribution of Ican be writ-ten

P,(i e S) = P,[E'Jp,(i e E') + P,[E']p,(i e E). (A3)

With I, and I2 being identically distributed, pI(i e E')= pJ(i E"), and Eq. (A4) can be written as

p,(i) = p,(i e E'). (A4)

5134 APPLIED OPTICS / Vol. 26, No. 23 / 1 December 1987

Histogram of Intensity for

Subtrting and rectifying UncorrelatedaL 1.0 Speckle Patterns

a Speckle PatternE

0.5 1.0

Relative Pixel Intensity

Fig.6. Histograms of polarized speckle pattern and image obtainedby subtracting two uncorrelated speckle patterns and rectifying the

results.

First, let us compute the probability that I 2 i giventhat I2 = i2 and i E E'. We obtain

F1(i1I2 = i2, i E E') = -I exp(-i,/A)dil

= 1 - exp[-(i 2 - i)/A]J for i2 > i

= 0 for i2 < i.

The probability that I 2 i is, therefore, equal to

F1(ili e E') =- exp[-(i 2 -i)/Al * exp(-i2/A)di2

+ A exp(-i 2 /A)di 2 = /2 exp(-i/A).

Since

F1(ili E E') = P1 [E'] J P1 (i')di',

we have

(A5)

(A6)

(A7)

sum of two uncorrelated polarized speckle patterns.Its probability density distribution is given by

PIO(io) = 10 exp(-io/A). (Bi)

Using the same mathematical approach as before, theprobability that I 2 i given that I2 = i2 and i eE' isequal to

F1(i1I2 = i, i E E') = 2 I il exp(-il/A)dil

for 2 i

(B2)= 0 fori 2 <i

The probability that I 2 i is then

F1(ili E E') = -exp(i/A) I i2 exp(-2i2 /A)di2

+ 3 Ji i2 exp(-2i 2 /A)di 2

- 2 exp(i/A) f2 exp(-2i 2 /A)di2

+ - Jt i2 exp(-i2 /A)di2

= exp(-i/A) 11+ /2 exp(-i/A)\iA/

= P1 [E'] J p1 (i')di'.

Thus

p1(i) = 2exp(-i/A) + - exp(-i/A).2A2 2A

(B3)

(B4)

p1 (i) = exp(-i/A), (A8)

which is identical to that of a speckle pattern. There-fore, subtracting two uncorrelated speckle patternswith the same statistics and rectifying the result pro-duce another speckle pattern.

A computer simulation was performed to verify thetheory. For a Gaussian distributed random phaseinput, histograms (probability density distributions)of the speckle image and the processed image obtainedusing the same aperture (or bandwidth) are plottedtogether in Fig. 6. We see that they fall on top of eachother. Therefore, the image obtained by subtractingtwo uncorrelated speckle patterns and rectifying theresult is another speckle pattern.

Appendix B

A polarized illuminating field is often depolarizedwhen scattered by a diffuse surface. It is interesting tosee the results when the aperture sampling techniqueis applied to unpolarized speckle images.

The speckle pattern obtained with unpolarized co-herent light can be considered to be the incoherent

References1. K. Creath and G. A. Slettemoen, "Vibration Observation Tech-

nique for Digital Speckle-Pattern Interferometry," J. Opt. Soc.Am. A 2, 1629 (1985).

2. K. Creath, "Digital Speckle Pattern Interferometry (DSPI) Us-ing a 100 X 100 Imaging Array," Proc. Soc. Photo-Opt. Instrum.Eng. 501, 292 (1984).

3. 0. J. Lokberg, J. T. Malmo, and G. A. Slettemoen, "Interfero-metric Measurements of High Temperature Objects by Electron-ic Speckle Pattern Interferometry," Appl. Opt. 24, 3167 (1985).

4. A. E. Ennos, "Speckle Interferometry," in Laser Speckle andRelated Phenomena, J. C. Dainty, Ed. (Springer-Verlag, Berlin,1975).

5. J. W. Goodman, "Statistical Properties of Laser Speckle Pat-terns," in Laser Speckle and Related Phenomena, J. C. Dainty,Ed. (Springer-Verlag, Berlin, 1975).

6. R. A. Sprague, "Surface Roughness Measurement Using WhiteLight Speckle," Appl. Opt. 11, 2811 (1972).

7. 0. J. Lokberg and O. K. Ledang, "Vibration of Flutes Studied byElectronic Speckle Pattern Interferometry," Appl. Opt. 23, 3052(1984).

8. 0. J. Lokberg and G. A. Slettemoen, "Electronic Speckle PatternInterferometry," in Applied Optics and Optical Engineering, R.R. Shannon and J. C. Wyant, Eds. (Academic, New York, to bepublished).

1 December 1987 / Vol. 26, No. 23 / APPLIED OPTICS 5135