, member, ieee, and p. ferraro ieee proofweb...

12
IEEE Proof Web Version JOURNAL OF DISPLAY TECHNOLOGY 1 Automatic Digital Hologram Denoising by Spatiotemporal Analysis of Pixel-Wise Statistics M. Leo, C. Distante, Member, IEEE, M. Paturzo, P. Memmolo, M. Locatelli, E. Pugliese, R. Meucci, Member, IEEE, and P. Ferraro, Senior Member, IEEE Abstract—In this paper, a new technique to reduce the noise in a reconstructed hologram image is proposed. Unlike all the tech- niques in the literature, the proposed approach not only takes into account spatial information but also temporal statistics associated with the pixels. This innovative solution enables, at rst, the au- tomatic detection of the areas of the image containing the objects (foreground). This way, all the pixels not belonging to any objects are directly cleaned up and the contrast between objects and back- ground is consistently increased. The remaining pixels are then processed with a spatio-temporal ltering which cancels out the effects of speckle noise, while preserving the structural details of the objects. The proposed approach has been compared with other common speckle denoising techniques and it is found to give better both visual and quantitative results. Index Terms—Holography, image denoising, statistical learning, adaptive lters, image segmentation. I. INTRODUCTION I N DIGITAL holography (DH), a CCD camera is used to record the holograms without wet chemical processing which is complex and time-consuming [1]. The continuous spa- tial distribution of an optically generated hologram is sampled by the discrete sensitive pixels on a CCD array, whose outputs are converted to the digitized signals and stored in an image processing system for numerical evaluation. Digital Holography uses coherent source and, when the light is diffused by an optically rough surface, speckle noise is gen- erated. It degrades the quality of the rendered images making the accurate viewing of small details very difcult. Moreover, speckle noise is emphasized by the limited CCD pixel size. It follows that it is very important to search for an effective way to reduce the speckle noise in the DH [2] but, unfortunately, its reduction is a challenge because of the badly statistical regu- larity of the noise. A variety of approaches have been proposed to overcome this drawback: some of them are based on image processing techniques [12], others on optical techniques [13]. Optical tech- Manuscript received February 08, 2013; revised May 14, 2013; accepted June 11, 2013. This work was supported by the Programma Operativo Nazionale (PON) project IT@CHA funded by the Italian Ministry of Education, Univer- sity and Research (MIUR). M. Leo and C. Distante are with the CNR—Istituto Nazionale di Ottica, 73010 Arnesano (LE), Italy. M. Paturzo, P. Memmolo and P. Ferraro are with the CNR—Istituto Nazionale di Ottica, Comprensorio “A. Olivetti,” I-80078 Pozzuoli (Naples), Italy. P. Memmolo is with Center for Advanced Biomaterials for Health Care@CRIB, Istituto Italiano di Tecnologia, Napoli 80125, Italy. M. Locatelli, E. Pugliese and R. Meucci are with the CNR—Istituto Nazionale di Ottica, I-50125 Firenze, Italy. Color versions of one or more of the gures are available online at http:// ieeexplore.ieee.org. Digital Object Identier 10.1109/JDT.2013.2268936 niques are based on the denition of specic acquisition setup. Instead, digital image processing techniques do not affect the acquisition setup but they try to reduce the noise by numerical processing of the recorded data. Non-adaptive lters (e.g., mean or median lter [12]) have been largely used for this purpose. To reduce the impact of ltering in image areas containing tex- ture and edges, adaptive lters have also been introduced (e.g., Frost lter [16], Lee lter [15], Discrete Fourier Filtering [3], non-local means ltering [14], etc.). Overall, the techniques in the literature try to identify the con- tribution of the noise in each pixel through the only information of spatial proximity in a single image. Unfortunately, as evi- denced from the literature, this does not always allow to prop- erly separate the noise from the information content of the holo- graphic images. In this paper a new technique to reduce the noise in a re- constructed hologram image is therefore proposed: the key idea is to extract additional useful information, for noise detection and suppression, through the observation of temporal numer- ical values at each pixel location in a sequence of images. The main novelty of the proposed solution is that it combines both spatial information and statistics along the time to detect and suppress the speckle noise as well as to preserve the information content of holograms. This is achieved in two steps: the auto- matic detection of the areas of the image containing the objects (foreground) is rstly performed. This way, all the pixels not be- longing to any objects are directly cleaned up and the contrast between object and background is consistently increased. The remaining pixels are then processed with a spatio-temporal l- tering which cancel out the effects of speckle, while preserving the structural details of the objects. In the full digital holographic process, two kinds of noise have to be handled: the rst one is an additive uncorrelated noise that corrupts the observed irradiance, the other one is the speckle noise [17]. In the proposed approach, the rst computational step, that detects the region in which the object is contained, removes the contribution of additive noise in the background. The second step, performing spatio-temporal ltering, deals instead with the speckle noise reduction. The proposed approach has been tested on an holographic se- quence of the Perseus statue captured at long wavelength in- frared (IR) range (10.6 m) in off-axis Fourier conguration. The statuette is about 33 cm in height. The recording distance, that is the distance between the object and the camera is 80 cm. As a coherent light source, a 110 W—CW CO laser emitting at 10, 6 m, is used. In this case, the speckle size is about 50 microns. In fact, the speckle size along one direction in the ob- servation plane is proportional to the incident wavelength, and 1551-319X/$31.00 © 2013 IEEE

Upload: others

Post on 23-Jul-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: , Member, IEEE, and P. Ferraro IEEE ProofWeb Versionpeople.isasi.cnr.it/~m.leo/pubblica/JDT2268936_PROOF.pdf · IEEE ProofWeb Version JOURNAL OF DISPLAY TECHNOLOGY 1 Automatic Digital

IEEE

Pro

of

Web

Ver

sion

JOURNAL OF DISPLAY TECHNOLOGY 1

Automatic Digital Hologram Denoising bySpatiotemporal Analysis of Pixel-Wise Statistics

M. Leo, C. Distante, Member, IEEE, M. Paturzo, P. Memmolo, M. Locatelli, E. Pugliese,R. Meucci, Member, IEEE, and P. Ferraro, Senior Member, IEEE

Abstract—In this paper, a new technique to reduce the noise ina reconstructed hologram image is proposed. Unlike all the tech-niques in the literature, the proposed approach not only takes intoaccount spatial information but also temporal statistics associatedwith the pixels. This innovative solution enables, at first, the au-tomatic detection of the areas of the image containing the objects(foreground). This way, all the pixels not belonging to any objectsare directly cleaned up and the contrast between objects and back-ground is consistently increased. The remaining pixels are thenprocessed with a spatio-temporal filtering which cancels out theeffects of speckle noise, while preserving the structural details ofthe objects. The proposed approach has been compared with othercommon speckle denoising techniques and it is found to give betterboth visual and quantitative results.

Index Terms—Holography, image denoising, statistical learning,adaptive filters, image segmentation.

I. INTRODUCTION

I N DIGITAL holography (DH), a CCD camera is used torecord the holograms without wet chemical processing

which is complex and time-consuming [1]. The continuous spa-tial distribution of an optically generated hologram is sampledby the discrete sensitive pixels on a CCD array, whose outputsare converted to the digitized signals and stored in an imageprocessing system for numerical evaluation.Digital Holography uses coherent source and, when the light

is diffused by an optically rough surface, speckle noise is gen-erated. It degrades the quality of the rendered images makingthe accurate viewing of small details very difficult. Moreover,speckle noise is emphasized by the limited CCD pixel size. Itfollows that it is very important to search for an effective wayto reduce the speckle noise in the DH [2] but, unfortunately, itsreduction is a challenge because of the badly statistical regu-larity of the noise.A variety of approaches have been proposed to overcome

this drawback: some of them are based on image processingtechniques [12], others on optical techniques [13]. Optical tech-

Manuscript received February 08, 2013; revisedMay 14, 2013; accepted June11, 2013. This work was supported by the Programma Operativo Nazionale(PON) project IT@CHA funded by the Italian Ministry of Education, Univer-sity and Research (MIUR).M. Leo and C. Distante are with the CNR—Istituto Nazionale di Ottica,

73010 Arnesano (LE), Italy.M. Paturzo, P. Memmolo and P. Ferraro are with the CNR—Istituto

Nazionale di Ottica, Comprensorio “A. Olivetti,” I-80078 Pozzuoli (Naples),Italy.P. Memmolo is with Center for Advanced Biomaterials for Health

Care@CRIB, Istituto Italiano di Tecnologia, Napoli 80125, Italy.M. Locatelli, E. Pugliese and R. Meucci are with the CNR—Istituto

Nazionale di Ottica, I-50125 Firenze, Italy.Color versions of one or more of the figures are available online at http://

ieeexplore.ieee.org.Digital Object Identifier 10.1109/JDT.2013.2268936

niques are based on the definition of specific acquisition setup.Instead, digital image processing techniques do not affect theacquisition setup but they try to reduce the noise by numericalprocessing of the recorded data. Non-adaptive filters (e.g., meanor median filter [12]) have been largely used for this purpose.To reduce the impact of filtering in image areas containing tex-ture and edges, adaptive filters have also been introduced (e.g.,Frost filter [16], Lee filter [15], Discrete Fourier Filtering [3],non-local means filtering [14], etc.).Overall, the techniques in the literature try to identify the con-

tribution of the noise in each pixel through the only informationof spatial proximity in a single image. Unfortunately, as evi-denced from the literature, this does not always allow to prop-erly separate the noise from the information content of the holo-graphic images.In this paper a new technique to reduce the noise in a re-

constructed hologram image is therefore proposed: the key ideais to extract additional useful information, for noise detectionand suppression, through the observation of temporal numer-ical values at each pixel location in a sequence of images. Themain novelty of the proposed solution is that it combines bothspatial information and statistics along the time to detect andsuppress the speckle noise as well as to preserve the informationcontent of holograms. This is achieved in two steps: the auto-matic detection of the areas of the image containing the objects(foreground) is firstly performed. This way, all the pixels not be-longing to any objects are directly cleaned up and the contrastbetween object and background is consistently increased. Theremaining pixels are then processed with a spatio-temporal fil-tering which cancel out the effects of speckle, while preservingthe structural details of the objects.In the full digital holographic process, two kinds of noise have

to be handled: the first one is an additive uncorrelated noise thatcorrupts the observed irradiance, the other one is the specklenoise [17].In the proposed approach, the first computational step, that

detects the region in which the object is contained, removes thecontribution of additive noise in the background. The secondstep, performing spatio-temporal filtering, deals instead with thespeckle noise reduction.The proposed approach has been tested on an holographic se-

quence of the Perseus statue captured at long wavelength in-frared (IR) range (10.6 m) in off-axis Fourier configuration.The statuette is about 33 cm in height. The recording distance,that is the distance between the object and the camera is 80 cm.As a coherent light source, a 110 W—CW CO laser emittingat 10, 6 m, is used. In this case, the speckle size is about 50microns. In fact, the speckle size along one direction in the ob-servation plane is proportional to the incident wavelength, and

1551-319X/$31.00 © 2013 IEEE

Page 2: , Member, IEEE, and P. Ferraro IEEE ProofWeb Versionpeople.isasi.cnr.it/~m.leo/pubblica/JDT2268936_PROOF.pdf · IEEE ProofWeb Version JOURNAL OF DISPLAY TECHNOLOGY 1 Automatic Digital

IEEE

Pro

of

Web

Ver

sion

2 JOURNAL OF DISPLAY TECHNOLOGY

the distance between the object and the observation plane, is in-versely proportional to the maximal illuminated dimension ofthe object parallel to that direction. Qualitative and quantitativecomparisons with the leading approaches of the state of the artdemonstrated the effectiveness of the proposed approach.

II. THE PROPOSED APPROACH

The theoretical basis of the proposed approach is the adap-tive mixture models theory proposed in [4] for visual trackingpurpose.The mixture models theory can be summarized as follows: at

any time what is known about a particular pixel is itshistory:

(1)

where is the image sequence.The temporal occurrences of the values of each pixel can ob-

viously be statistically modeled: the simplest way to do thisis through a Gaussian distribution centered at the mean pixelvalue. In order to take into account also noise, a more sophis-ticated model must be introduced. In other words, a mixture ofGaussian distributions models the recent history of each pixel.In this way the probability of observing the current pixel valuein the position is

(2)

where is the number of distributions, is an estimate ofthe weight of the Gaussian in the mixture at time is themean value of the Gaussian in the mixture at time is thecovariance matrix of the Gaussian at time and, is a Gaussianprobability density function:

(3)

In this way a mixture of Gaussians (MOG) characterizes thedistribution of observed values of each pixel, especially whenthe pixel is subjected to visit several states as shown below.Every new pixel value is then used to update the model.

In particular, if among the Gaussians there is one having thenew pixel value within 2.5 standard deviation of the distribution(matching model), then its parameters are updated as follows:

(4)

(5)

where

(6)

The and parameters for the other distributions that do notsatisfy the matching condition remain the same.The prior weights of the distributions at time , are

adjusted as follows:

(7)

where is the learning rate and is 1 for the model whichmatch and 0 for the remaining models.If none of the distributions matches the value of the current

pixel, the least probable distribution is replaced with a distribu-tion having the current value as its mean value, an initially fixedhigh variance, and low prior weight.In this paper the above mixture models theory has been

adapted for the denoising purpose in the holographic recon-structed image.In particular, the proposed approach consists of a preliminary

phase involving the selection of a small number of pixels notbelonging to any object placed within the depth of field of theholographic acquisition setup. Using the terminology of the re-search area identified as machine learning, these pixels can beconsidered the reference set that allowthe system to learn the temporal behaviors of non-object pixels.During the acquisition phase, at the time instant , a statisticalmodel is built and updated according to (2)–(7) for each pixelof the holographic image.In addition, the likelihood of the set T is computed for each

pixel according to the probability computation in case of com-patible events as follows

(8)

where states for the all possible permutations of the ele-ments in taken at a time. In this way a likelihood image isgenerated and an adaptive threshold can be used to generate abinary image where pixels most probably belonging to the ob-ject under consideration are set to 1 and all the remaining pixelsare set to 0.The resulting binary image is then processed in order to de-

tected connected regions and to discard isolated pixels [5]. Theoutcome of the region connectivity analysis is a binary mask. This mask is then superimposed to the original holographic

image and, in this way, a new intermediate holographic imageis obtained:

(9)

The image is finally spatial-temporally further filtered inorder to smooth the speckle effects on the region belonging tothe objects under observation. The final image outcome ofthe proposed denoising approach is obtained in this way. Thisfiltering is done by replacing at time the pixel value inwith the value of its temporal statistics averaged with those ofthe 4-distance neighboring pixels. This is carried out for all non-zero pixels in whereas zero pixels remain unchanged. This canbe formalized as follows:

if

if

(10)

Page 3: , Member, IEEE, and P. Ferraro IEEE ProofWeb Versionpeople.isasi.cnr.it/~m.leo/pubblica/JDT2268936_PROOF.pdf · IEEE ProofWeb Version JOURNAL OF DISPLAY TECHNOLOGY 1 Automatic Digital

IEEE

Pro

of

Web

Ver

sion

LEO et al.: AUTOMATIC DH DENOISING BY SPATIOTEMPORAL ANALYSIS OF PIXEL-WISE STATISTICS 3

Fig. 1. Time distribution of the intensity values of a single pixel affected bynoise.

Fig. 2. (a) A photo of the Perseus statue. (b) The recorded and reconstructedholographic image.

III. EXPERIMENTAL RESULTS

The proposed approach has been tested on a holographicimage sequence of the Perseus statue [9], [10]. The hologramswere captured at long wavelength infrared (IR) range (10.6 m)in off-axis Fourier configuration [10]. A Miricle Thermoteknix307k, 640 480 detector resolution and 25 m pitch was used.At first, analyzing over time the statistical distribution of a set

of pixels in the image affected by the speckle noise has allowedthe choice of a suitable Gaussian mixture model. Fig. 1 showsthe distribution of a noisy pixel belonging to the foreground,where it is possible to notice a multimodal behavior of the in-tensity values for a time window of 100 temporally contiguousframes.This preliminary study has demonstrated that at least two

Gaussians are needed to describe the dynamics of pixel values.However, in order to correctly model also very infrequentvalues occurrences (that are usually associated to the speckle) a3-Gaussian model has been used in the following experiments.

Fig. 3. (a) The non-object likelihood image. (b) The segmented image.

Fig. 4. (a) The binary mask obtained after topological analysis of the seg-mented image. (b) The new holographic image obtained superimposing themask to the original holographic image.

Conversely, a greater number of Gaussians has been experi-mentally proven to be useless since the models generated onthe reference points experienced very low values of coefficientsassociated with the additional Gaussians.In Fig. 2(a), a photo of the Perseus statue is shown whereas,

Fig. 2(b) reports the 100th image of the recorded holographicsequence. Speckle noise is evident and strongly affects the holo-graphic image quality.In Fig. 3(a) the likelihood image relative to the 100th image

and obtained by using a set of 20 reference pixels not be-longing to the statue area is shown. In this figure, likelihoodvalues are computed as described in (8) and then rescaled inthe range for display purposes: the lightest pixels arerelated to areas of the background (highest non-object likeli-hood values) and the darkest pixels are instead related to areasbelonging to the statue (lowest non-object likelihood values).Fig. 3(b) reports the corresponding segmented binary image ex-tracted using a threshold on the likelihood values.

Page 4: , Member, IEEE, and P. Ferraro IEEE ProofWeb Versionpeople.isasi.cnr.it/~m.leo/pubblica/JDT2268936_PROOF.pdf · IEEE ProofWeb Version JOURNAL OF DISPLAY TECHNOLOGY 1 Automatic Digital

IEEE

Pro

of

Web

Ver

sion

4 JOURNAL OF DISPLAY TECHNOLOGY

Fig. 5. Different outcomes obtained by: (a) proposed approach; (b) Amedianfilter; (c) a discrete Fourier Filter; (d) a non-local mean filtering; (e) a Frost filter;and (f) a Lee filter.

In Fig. 4(a) the binary mask obtained after topological anal-ysis of the segmented image is shown. Fig. 4(b) presents thenew holographic image obtained as described in equation (9).In Fig. 5(a) the image obtained after the spatiotemporal fil-

tering detailed in (10) is shown.The outcomes of the proposed approach have then been quali-

tatively compared with those obtained using latest state of the artspatial speckle denoising techniques. In particular Figs. 5(b)–(f)show the hologram denoised by amedian filter (MF) [17], a Dis-crete Fourier Filter (DFF) [3], a nonlocal means filter (NLM)[14], a Frost filter [16] and a Lee Filter [15] respectively.

Fig. 6. [Note: Print is B/W only. ]Region used for qualitymetrics computation. Green rectangle includes a (quasi) uniform region usedfor speckle index computation. Red rectangle includes the region consideredfor Local Contrast evaluation.

All in all, from Fig. 5, the superiority of the proposed de-noising approach is evident. Its outcome is in fact much clearerthan those obtained by using other denoising strategies: on theone hand the noise is completely suppressed in the areas outsidethe object and strongly attenuated on the regions of the objectand, on the other hand, the details of the object appear preservedallowing a high visual quality.For a quantitative comparison of the various filters, three dif-

ferent metrics have been considered:1) Speckle Index [8], [9] on (quasi) homogenous areas of theobject under consideration;

2) Contrast of the whole image (Global Contrast) [6];3) Contrast [7] of a small region containing some details ofthe object (Local Contrast) [7].

Speckle index is a measure of the level of noise in the image:it is computed on a uniform region of the object in order to besure that every observed deviation from the expected referencevalue in the pixels is due to noise. That said, it is obvious thatthe lower the index, the less noisy is the image and in particular,in an ideally uncorrupted image the speckle index value is 0.Global Contrast indicates how much the object is enhanced

with respect to the whole content of the image, whereas, LocalContrast, is a measure of how much small details in the objectare preserved in the holographic reconstruction.Both the global and the local contrast should be kept as high

as possible: high values are in fact the numerical evidence thatthere is variability in the representation of the structures consti-tuting the object under observation.From the computation point of view, the Speckle index value

is computed as

(11)

where and are the standard deviation and the mean com-puted in the (quasi) uniform considered region included in thegreen rectangle in Fig. 6.Contrast measurements are instead computed as:

(12)

Page 5: , Member, IEEE, and P. Ferraro IEEE ProofWeb Versionpeople.isasi.cnr.it/~m.leo/pubblica/JDT2268936_PROOF.pdf · IEEE ProofWeb Version JOURNAL OF DISPLAY TECHNOLOGY 1 Automatic Digital

IEEE

Pro

of

Web

Ver

sion

LEO et al.: AUTOMATIC DH DENOISING BY SPATIOTEMPORAL ANALYSIS OF PIXEL-WISE STATISTICS 5

TABLE IVALUES OF QUALITY METRICS

where is a positive number (more precisely in thefollowing), is the standard deviation and is the kurtosismeasure defined by

(13)

where is the fourth moment about the mean and is thevariance.As mentioned above, the numerator and denominator in (12)

are computed on the whole image for Global Contrast measureand on a small selected region for Local Contrast respectively.The region chosen for Local Contrast computation is includedin the red rectangle in Fig. 6: this choice has been made con-sidering that this region contains some details of the face thatshould be preserved.A comparison between the six filters in terms of the above

metrics is shown in Table I, which provides the numerical ev-idence that the proposed approach achieved the best results interms of all the considered metrics.Going into detail, the analysis of the table shows that the pro-

posed approach is able, on the one hand, to best reduce the noise(lowest speckle index), but at the same time it also enhances therelevance of the object with respect to the background (highestGlobal contrast) and finally it preserves the structural details onthe object (highest Local Contrast).In particular, it is possible to argue that this encouraging re-

sult is due to the combined actions of two operational stepsinvolved and described in Section II: on the one hand the ini-tial temporal segmentation process increases the global imagecontrast (second last row), whereas, on the other hand, the fol-lowing spatiotemporal filtering reduces speckle on the objectregions without heavily affecting object structures (last row).This favorable effect could be exploited in contexts where ei-ther higher-contrast or lower speckle index is required.This section concludes with some details about the processing

time of the proposed technique and a quantitative comparisonwith conventional techniques that only use spatial information.All techniques have been implemented in Matlab®, version

R2012a, running on a commercial Laptop equippedwith an IntelPentium P6200 (with a clock frequency of 2.13 GHz) and 4 GBof RAM. Regarding the proposed technique, the temporal anal-ysis, defined by (8)–(9), takes, for the computation of the likeli-

hood in each pixel, on average of 3.8 10 sec. The spatialanalysis, defined by (10), takes instead, for the computation ofthe new value in each pixel of the holographic image, an averageof 1.4 10 sec. The final estimate of the time required forthe overall processing of each pixel in the image is in the orderof 10 sec.This result should be placed in relation with the average

time per pixel observed for the application of conventionaltechniques. In particular the median filtering [17] takes onaverage of 3.9 10 s for each pixel, the Discrete FourierFiltering [3] 9.44 10 s, the NonLocal Mean filtering1.1 10 s, the Frost filter 1.12 10 and finally the

Lee filter 1.49 10 . Although these data demonstratethe comparability of the computational time of the proposedtechnique with most of the conventional ones, it should beemphasized that the code used for the proposed techniquecould be further computationally enhanced (for example, usingbuilt-in Matlab functions for multidimensional data handling).Furthermore the computation required for the proposed tech-

nique is largely parallelizable (in particular (8) and (9) are em-barrassing parallel since they may be parallel performed foreach pixel).

IV. CONCLUSION

In this paper a new and innovative technique to reduce thenoise in a reconstructed hologram image has been proposed.In order to overcome the limitation of the techniques in the

literature, both spatial information and temporal statistics asso-ciated with individual pixels are used to detect and suppress thespeckle noise as well as to enhance the information content.Experimental results on an holographic image sequence cap-

tured at long wavelength infrared demonstrated that the pro-posed approach gives better results in both visual quality of thecleaned images and quantitative analysis of numerical outputsin terms of three different metrics.The analysis and comparison of processing time has also been

conducted demonstrating the comparability with the state-of-the-art techniques.Future works will be addressed to further improve re-

constructed hologram image by introducing iterative regiongrowing strategies that will allow overcoming segmentationerrors that occurred at the boundaries of the object regions.Moreover a refinement of the analysis of the pixel statistics

will be experimented in order to better preserve details on thereconstructed object.

REFERENCES[1] L. Rong, W. Xiao, F. Pan, S. Liu, and R. Li, “Speckle noise reduction

in digital holography by use of multiple polarization holograms,”Chin.Opt. Lett., vol. 8, no. 7, pp. 653–655, 2010.

[2] X.-O. Cai, “Reduction of speckle noise in the reconstructed image ofdigital holography,” Optik—Int. J. for Light & Electron Opt., vol. 121,no. 4, pp. 394–399, Feb. 2010, ISSN 0030-4026.

[3] J. Maycock, B. M. Hennelly, J. B. Mcdonald, Y. Frauel, A. Castro,B. Javidi, and T. J. Naughton, “Reduction of speckle in digital holog-raphy by discrete Fourier filtering,” J. Opt. Soc. Amer. A, vol. 24, pp.1617–1622, 2007.

[4] C. Stauffer and W. Grimson, “Adaptive background mixture modelsfor real-time tracking,” in Proc. IEEE Conf. on Comp. Vision and Pat-tern Recogn. (CVPR 1999), 1999, pp. 246–252.

[5] L. Shapiro and G. Stockman, Computer Vision. : Prentice Hall, 2002,pp. 69–73.

Page 6: , Member, IEEE, and P. Ferraro IEEE ProofWeb Versionpeople.isasi.cnr.it/~m.leo/pubblica/JDT2268936_PROOF.pdf · IEEE ProofWeb Version JOURNAL OF DISPLAY TECHNOLOGY 1 Automatic Digital

IEEE

Pro

of

Web

Ver

sion

6 JOURNAL OF DISPLAY TECHNOLOGY

[6] H. Tamura, S.Mori, and T. Yamawaki, “Texture features correspondingto visual perception,” IEEE Trans. Syst., Man, Cybern., vol. SMC-8,no. 6, pp. 460–473, Nov. 1978.

[7] P. Memmolo, C. Distante, M. Paturzo, A. Finizio, P. Ferraro, and B.Javidi, “Automatic focusing in digital holography and its applicationto stretched holograms,” Opt. Lett., vol. 36, pp. 1945–1947, 2011.

[8] L. S. Lim, “Techniques for speckle noise removal,” Opt. Eng., vol. 20,pp. 670–678, 1981.

[9] T. J. Crimmins, “Geometric filter for speckle reduction,” Appl. Opt.,vol. 24, pp. 1438–1443, 1985.

[10] M. Paturzo, A. Pelagotti, A. Finizio, L. Miccio, M. Locatelli, A.Gertrude, P. Poggi, R. Meucci, and P. Ferraro, Opt. Lett. vol. 35, p.2112, 2010.

[11] E. Stoykova, F. Yaraş, H. Kang, L. Onural, A. Geltrude, M. Lo-catelli, M. Paturzo, A. Pelagotti, R. Meucci, and P. Ferraro, “Visiblereconstruction by a circular holographic display from digital holo-grams recorded under infrared illumination,” Opt. Lett., vol. 37, pp.3120–3122, 2012.

[12] J. Garcia-Sucerquia, J. Alexis, H. Ramı́rez, and D. V. Prieto, “Reduc-tion of speckle noise in digital holography by using digital image pro-cessing,” Optik, vol. 116, no. 1, pp. 44–48, 2005.

[13] Q. Tian, H. Wang, and Z. Ren, “Reduction of speckle noise in digitalholography,” Appl. Laser, vol. 26, no. 6, pp. 429–442, 2006.

[14] A. Uzan, Y. Rivenson, and A. Stern, “Speckle denoising in digitalholography by nonlocal means filtering,” Appl. Opt., vol. 52, pp.A195–A200, 2013.

[15] J. S. Lee, “Digital image enhancement and noise filtering by use oflocal statistics,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-2,no. , pp. 165–168, 1980.

[16] V. S. Frost, J. A. Stiles, K. S. Shanmugan, and J. C. Holtzman, “Amodel for radar images and its application to adaptive digital filteringof multiplicative noise,” IEEE Trans. Pattern Anal. Mach. Intell., vol.PAMI-4, no. , pp. 157–166, 1982.

[17] P. Memmolo, I. Esnaola, A. Finizio, M. Paturzo, P. Ferraro, and A.M. Tulino, “SPADEDH: A sparsity-based denoising method of digitalholograms without knowing the noise statistics,”Opt. Express, vol. 20,pp. 17250–17257, 2012.

Marco Leo received the Honors degree in computer science engineering fromthe University of Salento, Lecce, Italy, in 2001.FromMay 2001 to August 2012, he was a Researcher with the Institute of In-

telligent Systems for Automation (ISSIA-CNR) in Bari, Italy. Since September2012 he has been a Researcher with the Institute of Optics (INO-CNR) in Lecce,Italy. His main research interests are in the fields of image processing, imageanalysis, computer vision, pattern recognition, digital signal processing, neuralnetworks, graphical models, wavelet transform, and independent componentanalysis. He participated in a number of national and international researchprojects facing automatic video surveillance of indoor and outdoor environ-ments, human attention monitoring, real-time event detection in sport contexts,and nondestructive inspection of aircraft components. He is the author of morethan 100 papers in national and international journals and conference proceed-ings. He is also a coauthor of three international patents on visual systems forevent detection in sport contexts.

Cosimo Distante received the degree in computer science from the Universityof Bari, Bari, Italy, in 1997, and the Ph.D. in engineering in 2001 by working inthe field of artificial intelligent systems.Dr. Distante has been Visiting researcher at the Computer Science Depart-

ment of the University of Massachusetts (Umass at Amherst, MA) where hasworked in the context of robot learning. In 1998, he served as a teaching assistantof the Artificial Intelligence course of the Master in Science of the Universityof Massachusetts. In 2000, he received the Ph.D. in engineering from the Uni-versity of Salento Thesis: “Intelligent Sensors and Systems”. Since 2003, he isContract Professor for the courses of Pattern Recognition and Image Processingin Computer Engineering at the University of the Salento. He is expert memberfor the panel in Innovation Technology of the Ministry of the Economic De-velopment. Dr. Distante is currently with the National Institute of Optics of theCNR where he is responsible for the following research theme: “optical devicesand ICT methods for industrial applications”. His main research interests arein the field of computer vision and pattern recognition applied to the followingsectors: video surveillance, robust estimation, medical imaging and manufac-turing.

Melania Paturzo is research scientist at INO-CNR. Her research interest is inthe field of Optics. Her research activities concerns the investigation of materialproperties by means of interferometric techniques, fabrication of periodicallypoled lithium niobate samples by electric field poling process, development ofoptical techniques to get super-resolution in digital holographic microscopy,holographic display, quantitative phase-contrast microscopy for cells investi-gation, optofluidics. She is author and/or coauthor of more than 40 papers ininternational peer reviewed journals and of more than 50 conference papers.

Pasquale Memmolo was born in Avellino, Italy, in 1982. In 2005, he receivedthe first level degree in computer engineering. In 2008, he received the secondlevel degree in communication engineering, with final mark 110/110, at the“Università degli Studi di Napoli Federico II” (Napoli, Italy), working on nu-merical and optical multiplexing and de-multiplexing methods in digital holog-raphy. In 2011, he completed the Ph.D. in electronic and communication en-gineering, with a thesis entitled “Compressed Sensing: a new framework forsignals recovery and its application in Digital Holography”. Now he worksas a Post Doctoral Junior at the Center for Advanced Biomaterials for HealthCare@CRIB, Istituto Italiano di Tecnologia, Napoli, Italy, and he is a collabo-rator at the CNR—Istituto Nazionale di Ottica, Pozzuoli, Italy. He is also co-au-thor of more of 30 international journal’s publications and proceedings papers.

Massimiliano Locatelli graduated in physics at Florence University, in 2009,with a Thesis on ‘Digital HolographyApplications’, developed at INO-CNR.Hegot a Ph.D. at LENS (European Laboratory for Non-linear Spectroscopy)-Uni-versity of Florence, in 2013, with a thesis on ‘Mid Infrared digital holographyand Terahertz imaging’. He is actually a researcher at LENS and INO-CNR ofFlorence. His research activity is conducted in the field of Optics (in particularInfrared and Terahertz imaging and digital holography), Biophysics and Com-plex System Physics.

Eugenio Pugliese received the Physics degree from the University of Florence,Italy, with a thesis on Mossbauer Spectroscopy, and is currently a Ph.D. studentin nonlinear dynamics and complex systems at the same university. He carriesout its research activities in the laboratories of INO-CNR in Florence. His re-search interests are in the field of chaotic dynamics of classical and quantumsystems, digital holography both in visible and IR range and optical investiga-tion of complex systems.

Riccardo Meucci graduated in physics at the University of Florence in 1982and received the Ph.D. degree in optics at the same University in 1987.From 1984 to 1987, he served as Researcher in Optics at Istituto di Ciber-

netica of the Italian National Research Council (CNR) Arco Felice (Naples),thereafter at Istituto Nazionale di Ottica (INO). Actually he is Research Di-rector at INO-CNR. His scientific activity mainly includes: chaotic instabilitiesin single mode lasers, laser transients, dynamical models for the CO laser, con-trol of chaos, spatio-temporal instabilities and delayed systems, polarization in-stabilities and digital holography in the mid-infrared region.

Pietro Ferraro (M’02–SM’07) is currently Chief Research Scientist at Isti-tuto Nazionale di Ottica del CNR, Pozzuoli, NA, Italy. Previously he workedas Principal Investigator with Alenia Aeronautics. He has published 12 bookchapters, 170 papers in journals, more than 150 papers at International Con-ferences. He Edited two books with Springer. He holds 14 patents. Among hiscurrent scientific interests are: holography, interferometry, microscopy, fabrica-tion of nanostructures, ferroelectric crystals, optical fiber sensors, fiber bragggratings, nano-microfluidics, optofluidics. Dr. Ferraro has chaired many Inter-national Conferences He is in the Editorial Board of Optics and Lasers in En-gineering (Elsevier), is Topical Editor of Optics Letters.Dr. Ferraro is Fellow of SPIE and Fellow of OSA.

Page 7: , Member, IEEE, and P. Ferraro IEEE ProofWeb Versionpeople.isasi.cnr.it/~m.leo/pubblica/JDT2268936_PROOF.pdf · IEEE ProofWeb Version JOURNAL OF DISPLAY TECHNOLOGY 1 Automatic Digital

IEEE

Pro

of

Prin

t Ver

sion

JOURNAL OF DISPLAY TECHNOLOGY 1

Automatic Digital Hologram Denoising bySpatiotemporal Analysis of Pixel-Wise Statistics

M. Leo, C. Distante, Member, IEEE, M. Paturzo, P. Memmolo, M. Locatelli, E. Pugliese,R. Meucci, Member, IEEE, and P. Ferraro, Senior Member, IEEE

Abstract—In this paper, a new technique to reduce the noise ina reconstructed hologram image is proposed. Unlike all the tech-niques in the literature, the proposed approach not only takes intoaccount spatial information but also temporal statistics associatedwith the pixels. This innovative solution enables, at first, the au-tomatic detection of the areas of the image containing the objects(foreground). This way, all the pixels not belonging to any objectsare directly cleaned up and the contrast between objects and back-ground is consistently increased. The remaining pixels are thenprocessed with a spatio-temporal filtering which cancels out theeffects of speckle noise, while preserving the structural details ofthe objects. The proposed approach has been compared with othercommon speckle denoising techniques and it is found to give betterboth visual and quantitative results.

Index Terms—Holography, image denoising, statistical learning,adaptive filters, image segmentation.

I. INTRODUCTION

I N DIGITAL holography (DH), a CCD camera is used torecord the holograms without wet chemical processing

which is complex and time-consuming [1]. The continuous spa-tial distribution of an optically generated hologram is sampledby the discrete sensitive pixels on a CCD array, whose outputsare converted to the digitized signals and stored in an imageprocessing system for numerical evaluation.Digital Holography uses coherent source and, when the light

is diffused by an optically rough surface, speckle noise is gen-erated. It degrades the quality of the rendered images makingthe accurate viewing of small details very difficult. Moreover,speckle noise is emphasized by the limited CCD pixel size. Itfollows that it is very important to search for an effective wayto reduce the speckle noise in the DH [2] but, unfortunately, itsreduction is a challenge because of the badly statistical regu-larity of the noise.A variety of approaches have been proposed to overcome

this drawback: some of them are based on image processingtechniques [12], others on optical techniques [13]. Optical tech-

Manuscript received February 08, 2013; revised May 14, 2013; accepted June11, 2013. This work was supported by the Programma Operativo Nazionale(PON) project IT@CHA funded by the Italian Ministry of Education, Univer-sity and Research (MIUR).M. Leo and C. Distante are with the CNR—Istituto Nazionale di Ottica,

73010 Arnesano (LE), Italy.M. Paturzo, P. Memmolo and P. Ferraro are with the CNR—Istituto

Nazionale di Ottica, Comprensorio “A. Olivetti,” I-80078 Pozzuoli (Naples),Italy.P. Memmolo is with Center for Advanced Biomaterials for Health

Care@CRIB, Istituto Italiano di Tecnologia, Napoli 80125, Italy.M. Locatelli, E. Pugliese and R. Meucci are with the CNR—Istituto

Nazionale di Ottica, I-50125 Firenze, Italy.Color versions of one or more of the figures are available online at http://

ieeexplore.ieee.org.Digital Object Identifier 10.1109/JDT.2013.2268936

niques are based on the definition of specific acquisition setup.Instead, digital image processing techniques do not affect theacquisition setup but they try to reduce the noise by numericalprocessing of the recorded data. Non-adaptive filters (e.g., meanor median filter [12]) have been largely used for this purpose.To reduce the impact of filtering in image areas containing tex-ture and edges, adaptive filters have also been introduced (e.g.,Frost filter [16], Lee filter [15], Discrete Fourier Filtering [3],non-local means filtering [14], etc.).Overall, the techniques in the literature try to identify the con-

tribution of the noise in each pixel through the only informationof spatial proximity in a single image. Unfortunately, as evi-denced from the literature, this does not always allow to prop-erly separate the noise from the information content of the holo-graphic images.In this paper a new technique to reduce the noise in a re-

constructed hologram image is therefore proposed: the key ideais to extract additional useful information, for noise detectionand suppression, through the observation of temporal numer-ical values at each pixel location in a sequence of images. Themain novelty of the proposed solution is that it combines bothspatial information and statistics along the time to detect andsuppress the speckle noise as well as to preserve the informationcontent of holograms. This is achieved in two steps: the auto-matic detection of the areas of the image containing the objects(foreground) is firstly performed. This way, all the pixels not be-longing to any objects are directly cleaned up and the contrastbetween object and background is consistently increased. Theremaining pixels are then processed with a spatio-temporal fil-tering which cancel out the effects of speckle, while preservingthe structural details of the objects.In the full digital holographic process, two kinds of noise have

to be handled: the first one is an additive uncorrelated noise thatcorrupts the observed irradiance, the other one is the specklenoise [17].In the proposed approach, the first computational step, that

detects the region in which the object is contained, removes thecontribution of additive noise in the background. The secondstep, performing spatio-temporal filtering, deals instead with thespeckle noise reduction.The proposed approach has been tested on an holographic se-

quence of the Perseus statue captured at long wavelength in-frared (IR) range (10.6 m) in off-axis Fourier configuration.The statuette is about 33 cm in height. The recording distance,that is the distance between the object and the camera is 80 cm.As a coherent light source, a 110 W—CW CO laser emittingat 10, 6 m, is used. In this case, the speckle size is about 50microns. In fact, the speckle size along one direction in the ob-servation plane is proportional to the incident wavelength, and

1551-319X/$31.00 © 2013 IEEE

Page 8: , Member, IEEE, and P. Ferraro IEEE ProofWeb Versionpeople.isasi.cnr.it/~m.leo/pubblica/JDT2268936_PROOF.pdf · IEEE ProofWeb Version JOURNAL OF DISPLAY TECHNOLOGY 1 Automatic Digital

IEEE

Pro

of

Prin

t Ver

sion

2 JOURNAL OF DISPLAY TECHNOLOGY

the distance between the object and the observation plane, is in-versely proportional to the maximal illuminated dimension ofthe object parallel to that direction. Qualitative and quantitativecomparisons with the leading approaches of the state of the artdemonstrated the effectiveness of the proposed approach.

II. THE PROPOSED APPROACH

The theoretical basis of the proposed approach is the adap-tive mixture models theory proposed in [4] for visual trackingpurpose.The mixture models theory can be summarized as follows: at

any time what is known about a particular pixel is itshistory:

(1)

where is the image sequence.The temporal occurrences of the values of each pixel can ob-

viously be statistically modeled: the simplest way to do thisis through a Gaussian distribution centered at the mean pixelvalue. In order to take into account also noise, a more sophis-ticated model must be introduced. In other words, a mixture ofGaussian distributions models the recent history of each pixel.In this way the probability of observing the current pixel valuein the position is

(2)

where is the number of distributions, is an estimate ofthe weight of the Gaussian in the mixture at time is themean value of the Gaussian in the mixture at time is thecovariance matrix of the Gaussian at time and, is a Gaussianprobability density function:

(3)

In this way a mixture of Gaussians (MOG) characterizes thedistribution of observed values of each pixel, especially whenthe pixel is subjected to visit several states as shown below.Every new pixel value is then used to update the model.

In particular, if among the Gaussians there is one having thenew pixel value within 2.5 standard deviation of the distribution(matching model), then its parameters are updated as follows:

(4)

(5)

where

(6)

The and parameters for the other distributions that do notsatisfy the matching condition remain the same.The prior weights of the distributions at time , are

adjusted as follows:

(7)

where is the learning rate and is 1 for the model whichmatch and 0 for the remaining models.If none of the distributions matches the value of the current

pixel, the least probable distribution is replaced with a distribu-tion having the current value as its mean value, an initially fixedhigh variance, and low prior weight.In this paper the above mixture models theory has been

adapted for the denoising purpose in the holographic recon-structed image.In particular, the proposed approach consists of a preliminary

phase involving the selection of a small number of pixels notbelonging to any object placed within the depth of field of theholographic acquisition setup. Using the terminology of the re-search area identified as machine learning, these pixels can beconsidered the reference set that allowthe system to learn the temporal behaviors of non-object pixels.During the acquisition phase, at the time instant , a statisticalmodel is built and updated according to (2)–(7) for each pixelof the holographic image.In addition, the likelihood of the set T is computed for each

pixel according to the probability computation in case of com-patible events as follows

(8)

where states for the all possible permutations of the ele-ments in taken at a time. In this way a likelihood image isgenerated and an adaptive threshold can be used to generate abinary image where pixels most probably belonging to the ob-ject under consideration are set to 1 and all the remaining pixelsare set to 0.The resulting binary image is then processed in order to de-

tected connected regions and to discard isolated pixels [5]. Theoutcome of the region connectivity analysis is a binary mask. This mask is then superimposed to the original holographic

image and, in this way, a new intermediate holographic imageis obtained:

(9)

The image is finally spatial-temporally further filtered inorder to smooth the speckle effects on the region belonging tothe objects under observation. The final image outcome ofthe proposed denoising approach is obtained in this way. Thisfiltering is done by replacing at time the pixel value inwith the value of its temporal statistics averaged with those ofthe 4-distance neighboring pixels. This is carried out for all non-zero pixels in whereas zero pixels remain unchanged. This canbe formalized as follows:

if

if

(10)

Page 9: , Member, IEEE, and P. Ferraro IEEE ProofWeb Versionpeople.isasi.cnr.it/~m.leo/pubblica/JDT2268936_PROOF.pdf · IEEE ProofWeb Version JOURNAL OF DISPLAY TECHNOLOGY 1 Automatic Digital

IEEE

Pro

of

Prin

t Ver

sion

LEO et al.: AUTOMATIC DH DENOISING BY SPATIOTEMPORAL ANALYSIS OF PIXEL-WISE STATISTICS 3

Fig. 1. Time distribution of the intensity values of a single pixel affected bynoise.

Fig. 2. (a) A photo of the Perseus statue. (b) The recorded and reconstructedholographic image.

III. EXPERIMENTAL RESULTS

The proposed approach has been tested on a holographicimage sequence of the Perseus statue [9], [10]. The hologramswere captured at long wavelength infrared (IR) range (10.6 m)in off-axis Fourier configuration [10]. A Miricle Thermoteknix307k, 640 480 detector resolution and 25 m pitch was used.At first, analyzing over time the statistical distribution of a set

of pixels in the image affected by the speckle noise has allowedthe choice of a suitable Gaussian mixture model. Fig. 1 showsthe distribution of a noisy pixel belonging to the foreground,where it is possible to notice a multimodal behavior of the in-tensity values for a time window of 100 temporally contiguousframes.This preliminary study has demonstrated that at least two

Gaussians are needed to describe the dynamics of pixel values.However, in order to correctly model also very infrequentvalues occurrences (that are usually associated to the speckle) a3-Gaussian model has been used in the following experiments.

Fig. 3. (a) The non-object likelihood image. (b) The segmented image.

Fig. 4. (a) The binary mask obtained after topological analysis of the seg-mented image. (b) The new holographic image obtained superimposing themask to the original holographic image.

Conversely, a greater number of Gaussians has been experi-mentally proven to be useless since the models generated onthe reference points experienced very low values of coefficientsassociated with the additional Gaussians.In Fig. 2(a), a photo of the Perseus statue is shown whereas,

Fig. 2(b) reports the 100th image of the recorded holographicsequence. Speckle noise is evident and strongly affects the holo-graphic image quality.In Fig. 3(a) the likelihood image relative to the 100th image

and obtained by using a set of 20 reference pixels not be-longing to the statue area is shown. In this figure, likelihoodvalues are computed as described in (8) and then rescaled inthe range for display purposes: the lightest pixels arerelated to areas of the background (highest non-object likeli-hood values) and the darkest pixels are instead related to areasbelonging to the statue (lowest non-object likelihood values).Fig. 3(b) reports the corresponding segmented binary image ex-tracted using a threshold on the likelihood values.

Page 10: , Member, IEEE, and P. Ferraro IEEE ProofWeb Versionpeople.isasi.cnr.it/~m.leo/pubblica/JDT2268936_PROOF.pdf · IEEE ProofWeb Version JOURNAL OF DISPLAY TECHNOLOGY 1 Automatic Digital

IEEE

Pro

of

Prin

t Ver

sion

4 JOURNAL OF DISPLAY TECHNOLOGY

Fig. 5. Different outcomes obtained by: (a) proposed approach; (b) Amedianfilter; (c) a discrete Fourier Filter; (d) a non-local mean filtering; (e) a Frost filter;and (f) a Lee filter.

In Fig. 4(a) the binary mask obtained after topological anal-ysis of the segmented image is shown. Fig. 4(b) presents thenew holographic image obtained as described in equation (9).In Fig. 5(a) the image obtained after the spatiotemporal fil-

tering detailed in (10) is shown.The outcomes of the proposed approach have then been quali-

tatively comparedwith those obtained using latest state of the artspatial speckle denoising techniques. In particular Figs. 5(b)–(f)show the hologram denoised by amedian filter (MF) [17], a Dis-crete Fourier Filter (DFF) [3], a nonlocal means filter (NLM)[14], a Frost filter [16] and a Lee Filter [15] respectively.

Fig. 6. [Note: Print is B/W only. ]Region used for qualitymetrics computation. Green rectangle includes a (quasi) uniform region usedfor speckle index computation. Red rectangle includes the region consideredfor Local Contrast evaluation.

All in all, from Fig. 5, the superiority of the proposed de-noising approach is evident. Its outcome is in fact much clearerthan those obtained by using other denoising strategies: on theone hand the noise is completely suppressed in the areas outsidethe object and strongly attenuated on the regions of the objectand, on the other hand, the details of the object appear preservedallowing a high visual quality.For a quantitative comparison of the various filters, three dif-

ferent metrics have been considered:1) Speckle Index [8], [9] on (quasi) homogenous areas of theobject under consideration;

2) Contrast of the whole image (Global Contrast) [6];3) Contrast [7] of a small region containing some details ofthe object (Local Contrast) [7].

Speckle index is a measure of the level of noise in the image:it is computed on a uniform region of the object in order to besure that every observed deviation from the expected referencevalue in the pixels is due to noise. That said, it is obvious thatthe lower the index, the less noisy is the image and in particular,in an ideally uncorrupted image the speckle index value is 0.Global Contrast indicates how much the object is enhanced

with respect to the whole content of the image, whereas, LocalContrast, is a measure of how much small details in the objectare preserved in the holographic reconstruction.Both the global and the local contrast should be kept as high

as possible: high values are in fact the numerical evidence thatthere is variability in the representation of the structures consti-tuting the object under observation.From the computation point of view, the Speckle index value

is computed as

(11)

where and are the standard deviation and the mean com-puted in the (quasi) uniform considered region included in thegreen rectangle in Fig. 6.Contrast measurements are instead computed as:

(12)

Page 11: , Member, IEEE, and P. Ferraro IEEE ProofWeb Versionpeople.isasi.cnr.it/~m.leo/pubblica/JDT2268936_PROOF.pdf · IEEE ProofWeb Version JOURNAL OF DISPLAY TECHNOLOGY 1 Automatic Digital

IEEE

Pro

of

Prin

t Ver

sion

LEO et al.: AUTOMATIC DH DENOISING BY SPATIOTEMPORAL ANALYSIS OF PIXEL-WISE STATISTICS 5

TABLE IVALUES OF QUALITY METRICS

where is a positive number (more precisely in thefollowing), is the standard deviation and is the kurtosismeasure defined by

(13)

where is the fourth moment about the mean and is thevariance.As mentioned above, the numerator and denominator in (12)

are computed on the whole image for Global Contrast measureand on a small selected region for Local Contrast respectively.The region chosen for Local Contrast computation is includedin the red rectangle in Fig. 6: this choice has been made con-sidering that this region contains some details of the face thatshould be preserved.A comparison between the six filters in terms of the above

metrics is shown in Table I, which provides the numerical ev-idence that the proposed approach achieved the best results interms of all the considered metrics.Going into detail, the analysis of the table shows that the pro-

posed approach is able, on the one hand, to best reduce the noise(lowest speckle index), but at the same time it also enhances therelevance of the object with respect to the background (highestGlobal contrast) and finally it preserves the structural details onthe object (highest Local Contrast).In particular, it is possible to argue that this encouraging re-

sult is due to the combined actions of two operational stepsinvolved and described in Section II: on the one hand the ini-tial temporal segmentation process increases the global imagecontrast (second last row), whereas, on the other hand, the fol-lowing spatiotemporal filtering reduces speckle on the objectregions without heavily affecting object structures (last row).This favorable effect could be exploited in contexts where ei-ther higher-contrast or lower speckle index is required.This section concludeswith some details about the processing

time of the proposed technique and a quantitative comparisonwith conventional techniques that only use spatial information.All techniques have been implemented in Matlab®, version

R2012a, running on a commercial Laptop equippedwith an IntelPentium P6200 (with a clock frequency of 2.13 GHz) and 4 GBof RAM. Regarding the proposed technique, the temporal anal-ysis, defined by (8)–(9), takes, for the computation of the likeli-

hood in each pixel, on average of 3.8 10 sec. The spatialanalysis, defined by (10), takes instead, for the computation ofthe new value in each pixel of the holographic image, an averageof 1.4 10 sec. The final estimate of the time required forthe overall processing of each pixel in the image is in the orderof 10 sec.This result should be placed in relation with the average

time per pixel observed for the application of conventionaltechniques. In particular the median filtering [17] takes onaverage of 3.9 10 s for each pixel, the Discrete FourierFiltering [3] 9.44 10 s, the NonLocal Mean filtering1.1 10 s, the Frost filter 1.12 10 and finally theLee filter 1.49 10 . Although these data demonstratethe comparability of the computational time of the proposedtechnique with most of the conventional ones, it should beemphasized that the code used for the proposed techniquecould be further computationally enhanced (for example, usingbuilt-in Matlab functions for multidimensional data handling).Furthermore the computation required for the proposed tech-

nique is largely parallelizable (in particular (8) and (9) are em-barrassing parallel since they may be parallel performed foreach pixel).

IV. CONCLUSION

In this paper a new and innovative technique to reduce thenoise in a reconstructed hologram image has been proposed.In order to overcome the limitation of the techniques in the

literature, both spatial information and temporal statistics asso-ciated with individual pixels are used to detect and suppress thespeckle noise as well as to enhance the information content.Experimental results on an holographic image sequence cap-

tured at long wavelength infrared demonstrated that the pro-posed approach gives better results in both visual quality of thecleaned images and quantitative analysis of numerical outputsin terms of three different metrics.The analysis and comparison of processing time has also been

conducted demonstrating the comparability with the state-of-the-art techniques.Future works will be addressed to further improve re-

constructed hologram image by introducing iterative regiongrowing strategies that will allow overcoming segmentationerrors that occurred at the boundaries of the object regions.Moreover a refinement of the analysis of the pixel statistics

will be experimented in order to better preserve details on thereconstructed object.

REFERENCES[1] L. Rong, W. Xiao, F. Pan, S. Liu, and R. Li, “Speckle noise reduction

in digital holography by use of multiple polarization holograms,”Chin.Opt. Lett., vol. 8, no. 7, pp. 653–655, 2010.

[2] X.-O. Cai, “Reduction of speckle noise in the reconstructed image ofdigital holography,” Optik—Int. J. for Light & Electron Opt., vol. 121,no. 4, pp. 394–399, Feb. 2010, ISSN 0030-4026.

[3] J. Maycock, B. M. Hennelly, J. B. Mcdonald, Y. Frauel, A. Castro,B. Javidi, and T. J. Naughton, “Reduction of speckle in digital holog-raphy by discrete Fourier filtering,” J. Opt. Soc. Amer. A, vol. 24, pp.1617–1622, 2007.

[4] C. Stauffer and W. Grimson, “Adaptive background mixture modelsfor real-time tracking,” in Proc. IEEE Conf. on Comp. Vision and Pat-tern Recogn. (CVPR 1999), 1999, pp. 246–252.

[5] L. Shapiro and G. Stockman, Computer Vision. : Prentice Hall, 2002,pp. 69–73.

Page 12: , Member, IEEE, and P. Ferraro IEEE ProofWeb Versionpeople.isasi.cnr.it/~m.leo/pubblica/JDT2268936_PROOF.pdf · IEEE ProofWeb Version JOURNAL OF DISPLAY TECHNOLOGY 1 Automatic Digital

IEEE

Pro

of

Prin

t Ver

sion

6 JOURNAL OF DISPLAY TECHNOLOGY

[6] H. Tamura, S.Mori, and T. Yamawaki, “Texture features correspondingto visual perception,” IEEE Trans. Syst., Man, Cybern., vol. SMC-8,no. 6, pp. 460–473, Nov. 1978.

[7] P. Memmolo, C. Distante, M. Paturzo, A. Finizio, P. Ferraro, and B.Javidi, “Automatic focusing in digital holography and its applicationto stretched holograms,” Opt. Lett., vol. 36, pp. 1945–1947, 2011.

[8] L. S. Lim, “Techniques for speckle noise removal,” Opt. Eng., vol. 20,pp. 670–678, 1981.

[9] T. J. Crimmins, “Geometric filter for speckle reduction,” Appl. Opt.,vol. 24, pp. 1438–1443, 1985.

[10] M. Paturzo, A. Pelagotti, A. Finizio, L. Miccio, M. Locatelli, A.Gertrude, P. Poggi, R. Meucci, and P. Ferraro, Opt. Lett. vol. 35, p.2112, 2010.

[11] E. Stoykova, F. Yaraş, H. Kang, L. Onural, A. Geltrude, M. Lo-catelli, M. Paturzo, A. Pelagotti, R. Meucci, and P. Ferraro, “Visiblereconstruction by a circular holographic display from digital holo-grams recorded under infrared illumination,” Opt. Lett., vol. 37, pp.3120–3122, 2012.

[12] J. Garcia-Sucerquia, J. Alexis, H. Ramı́rez, and D. V. Prieto, “Reduc-tion of speckle noise in digital holography by using digital image pro-cessing,” Optik, vol. 116, no. 1, pp. 44–48, 2005.

[13] Q. Tian, H. Wang, and Z. Ren, “Reduction of speckle noise in digitalholography,” Appl. Laser, vol. 26, no. 6, pp. 429–442, 2006.

[14] A. Uzan, Y. Rivenson, and A. Stern, “Speckle denoising in digitalholography by nonlocal means filtering,” Appl. Opt., vol. 52, pp.A195–A200, 2013.

[15] J. S. Lee, “Digital image enhancement and noise filtering by use oflocal statistics,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-2,no. , pp. 165–168, 1980.

[16] V. S. Frost, J. A. Stiles, K. S. Shanmugan, and J. C. Holtzman, “Amodel for radar images and its application to adaptive digital filteringof multiplicative noise,” IEEE Trans. Pattern Anal. Mach. Intell., vol.PAMI-4, no. , pp. 157–166, 1982.

[17] P. Memmolo, I. Esnaola, A. Finizio, M. Paturzo, P. Ferraro, and A.M. Tulino, “SPADEDH: A sparsity-based denoising method of digitalholograms without knowing the noise statistics,”Opt. Express, vol. 20,pp. 17250–17257, 2012.

Marco Leo received the Honors degree in computer science engineering fromthe University of Salento, Lecce, Italy, in 2001.From May 2001 to August 2012, he was a Researcher with the Institute of In-

telligent Systems for Automation (ISSIA-CNR) in Bari, Italy. Since September2012 he has been a Researcher with the Institute of Optics (INO-CNR) in Lecce,Italy. His main research interests are in the fields of image processing, imageanalysis, computer vision, pattern recognition, digital signal processing, neuralnetworks, graphical models, wavelet transform, and independent componentanalysis. He participated in a number of national and international researchprojects facing automatic video surveillance of indoor and outdoor environ-ments, human attention monitoring, real-time event detection in sport contexts,and nondestructive inspection of aircraft components. He is the author of morethan 100 papers in national and international journals and conference proceed-ings. He is also a coauthor of three international patents on visual systems forevent detection in sport contexts.

Cosimo Distante received the degree in computer science from the Universityof Bari, Bari, Italy, in 1997, and the Ph.D. in engineering in 2001 by working inthe field of artificial intelligent systems.Dr. Distante has been Visiting researcher at the Computer Science Depart-

ment of the University of Massachusetts (Umass at Amherst, MA) where hasworked in the context of robot learning. In 1998, he served as a teaching assistantof the Artificial Intelligence course of the Master in Science of the Universityof Massachusetts. In 2000, he received the Ph.D. in engineering from the Uni-versity of Salento Thesis: “Intelligent Sensors and Systems”. Since 2003, he isContract Professor for the courses of Pattern Recognition and Image Processingin Computer Engineering at the University of the Salento. He is expert memberfor the panel in Innovation Technology of the Ministry of the Economic De-velopment. Dr. Distante is currently with the National Institute of Optics of theCNR where he is responsible for the following research theme: “optical devicesand ICT methods for industrial applications”. His main research interests arein the field of computer vision and pattern recognition applied to the followingsectors: video surveillance, robust estimation, medical imaging and manufac-turing.

Melania Paturzo is research scientist at INO-CNR. Her research interest is inthe field of Optics. Her research activities concerns the investigation of materialproperties by means of interferometric techniques, fabrication of periodicallypoled lithium niobate samples by electric field poling process, development ofoptical techniques to get super-resolution in digital holographic microscopy,holographic display, quantitative phase-contrast microscopy for cells investi-gation, optofluidics. She is author and/or coauthor of more than 40 papers ininternational peer reviewed journals and of more than 50 conference papers.

Pasquale Memmolo was born in Avellino, Italy, in 1982. In 2005, he receivedthe first level degree in computer engineering. In 2008, he received the secondlevel degree in communication engineering, with final mark 110/110, at the“Università degli Studi di Napoli Federico II” (Napoli, Italy), working on nu-merical and optical multiplexing and de-multiplexing methods in digital holog-raphy. In 2011, he completed the Ph.D. in electronic and communication en-gineering, with a thesis entitled “Compressed Sensing: a new framework forsignals recovery and its application in Digital Holography”. Now he worksas a Post Doctoral Junior at the Center for Advanced Biomaterials for HealthCare@CRIB, Istituto Italiano di Tecnologia, Napoli, Italy, and he is a collabo-rator at the CNR—Istituto Nazionale di Ottica, Pozzuoli, Italy. He is also co-au-thor of more of 30 international journal’s publications and proceedings papers.

Massimiliano Locatelli graduated in physics at Florence University, in 2009,with a Thesis on ‘Digital Holography Applications’, developed at INO-CNR.Hegot a Ph.D. at LENS (European Laboratory for Non-linear Spectroscopy)-Uni-versity of Florence, in 2013, with a thesis on ‘Mid Infrared digital holographyand Terahertz imaging’. He is actually a researcher at LENS and INO-CNR ofFlorence. His research activity is conducted in the field of Optics (in particularInfrared and Terahertz imaging and digital holography), Biophysics and Com-plex System Physics.

Eugenio Pugliese received the Physics degree from the University of Florence,Italy, with a thesis on Mossbauer Spectroscopy, and is currently a Ph.D. studentin nonlinear dynamics and complex systems at the same university. He carriesout its research activities in the laboratories of INO-CNR in Florence. His re-search interests are in the field of chaotic dynamics of classical and quantumsystems, digital holography both in visible and IR range and optical investiga-tion of complex systems.

Riccardo Meucci graduated in physics at the University of Florence in 1982and received the Ph.D. degree in optics at the same University in 1987.From 1984 to 1987, he served as Researcher in Optics at Istituto di Ciber-

netica of the Italian National Research Council (CNR) Arco Felice (Naples),thereafter at Istituto Nazionale di Ottica (INO). Actually he is Research Di-rector at INO-CNR. His scientific activity mainly includes: chaotic instabilitiesin single mode lasers, laser transients, dynamical models for the CO laser, con-trol of chaos, spatio-temporal instabilities and delayed systems, polarization in-stabilities and digital holography in the mid-infrared region.

Pietro Ferraro (M’02–SM’07) is currently Chief Research Scientist at Isti-tuto Nazionale di Ottica del CNR, Pozzuoli, NA, Italy. Previously he workedas Principal Investigator with Alenia Aeronautics. He has published 12 bookchapters, 170 papers in journals, more than 150 papers at International Con-ferences. He Edited two books with Springer. He holds 14 patents. Among hiscurrent scientific interests are: holography, interferometry, microscopy, fabrica-tion of nanostructures, ferroelectric crystals, optical fiber sensors, fiber bragggratings, nano-microfluidics, optofluidics. Dr. Ferraro has chaired many Inter-national Conferences He is in the Editorial Board of Optics and Lasers in En-gineering (Elsevier), is Topical Editor of Optics Letters.Dr. Ferraro is Fellow of SPIE and Fellow of OSA.