Pattern Recognition 45 (2012) 393–406

Contents lists available at ScienceDirect

Pattern Recognition

0031-32

doi:10.1

n Corr

E-m

journal homepage: www.elsevier.com/locate/pr

Scale invariant small target detection by optimizing signal-to-clutter ratio inheterogeneous background for infrared search and track

Sungho Kim a,n, Joohyoung Lee a,b

a LED-IT Fusion Technology Research Center and Department of Electronic Engineering, Yeungnam University, 214-1, Dae-dong, Gyeongsan-si, Gyeongsangbuk-do 712-749,

Republic of Koreab 3-1-2, Agency for Defense Development, P.O. Box 35-3, Yuseong-gu, Daejeon 305-600, Republic of Korea

a r t i c l e i n f o

Article history:

Received 8 January 2009

Received in revised form

8 March 2011

Accepted 10 June 2011Available online 19 July 2011

Keywords:

Small target

Size-varying target

Background clutter

Human visual system

Signal-to-clutter ratio

Scale-space

Heterogeneous background

Infrared search and track

03/$ - see front matter & 2011 Elsevier Ltd. A

016/j.patcog.2011.06.009

esponding author. Tel.: þ82 53 810 3530; fax

ail address: kshrcv@gmail.com (S. Kim).

a b s t r a c t

This paper presents a novel mathematical method for incoming target detection in a cluttered

background motivated by the robust properties of the human visual system (HVS). The robust

detection of small targets is very important in IRST (Infrared Search and Track) applications for self-

defense or attacks. HVS shows the best efficiency and robustness for the task of object detection in

cluttered backgrounds. The robust properties of HVS include the contrast mechanism of figure-ground,

multi-resolution representation of an object, size adaptation of object boundary, and pop-out

phenomena in a complex environment. Based on these facts, a plausible computational model integrat-

ing these facts is proposed using Laplacian scale-space theory and an optimization method. Simulta-

neous target signal enhancement and background clutter suppression are achieved by tuning and

maximizing the signal-to-clutter ratio (TM-SCR) in Laplacian scale-space. At the first stage, Tune–Max

of the signal to background contrast produces candidate targets with estimated target scale. At the

second stage, Tune–Max of the signal-to-clutter ratio (SCR) produces maximal SCR that is used to sort

the detection results. Especially, the row-directional-local background removal filter (RD-LBRF) is

preprocessed in the horizontal region to enhance the TM-SCR method. The evaluation results of

incoming target sequence validate the detection capability of the proposed method from dim, small

targets to strong, large targets in comparison with the Top-hat method at the same rate of false alarms.

The experimental results of various cluttered background images show that the proposed TM-SCR

produces less false alarms (4.3 times reduction) compared to that of the Top-hat at the same detection

rate. Finally, TM-SCR after RD-LBRF can maximize the detection rate in horizontal regions.

& 2011 Elsevier Ltd. All rights reserved.

1. Introduction

The most important threats in sea-based infrared search andtrack (IRST) are incoming small targets such as anti-ship sea-skimming missiles (ASSM) or asymmetric ships. If a target is at afar distance, then its projected image is very small (about 2�2pixels) and dim (very low contrast), as is shown in the 1st row ofFig. 1. The 1st column shows enlarged target regions of an incomingship, and the 2nd column represents the 3D surf plots of thecorresponding target intensity images. The 3rd column representsthe 1D plots of the cross-sectional pixels around targets. The dots onthe graph represent the pixels corresponding to valid targets. As theship approaches, the size increases up to 12�12 pixels and theintensity becomes higher. Note that the target size is six times aslarge as the smallest one. It is difficult to detect size-varying targetsin IRST. Furthermore, the detection problem is more difficult if thebackgrounds have strong clutters. Note that the signal-to-clutter

ll rights reserved.

: þ82 53 810 4770.

ratios (SCR) are quite low due to the background clutter. Thesuppression of background clutter is also important to reduce thefalse alarms in IRST.

Conventional small target detection methods such as meansubtraction filter [1], median subtraction filter [2], Top-hat filter[3] , Max-Mean/Max-Median filter [4], least mean square filter [5],and matched filter [6] are used to reduce only the backgroundclutters estimated using the mean filter, median filter, openingfilter in Top-hat, and so on. Although those methods can some-what reduce background clutter, they do not consider the pro-blem of selecting filters that are suitable for the targets. In termsof the matched filter theory, the optimal filter shape should be thesame as the target shape. So, the application of such methods tothe incoming target detection problems shows limited detectionperformance enhancement since those methods use fixed filters.In a real environment, an incoming target changes its size in theimage, so the filters should adapt to the target sizes to achieverobust detection.

Several researchers proposed multiscale methods to solvethe scale problem. Gregoris et al. introduced a target detectionmethod based on the multiscale Wavelet transforms for dim

2×2

8×8

12×12

Fig. 1. Incoming ship target signature in a horizontal background: cropped IR images (1st column), 3D surf view (2nd column), and cross-sectional 1D plot (3rd column).

S. Kim, J. Lee / Pattern Recognition 45 (2012) 393–406394

target detection [7]. Multiscale images can provide valuablestructural information that can be used to distinguish targetsfrom clutter. Although the latter shows the feasibility of multi-scale analysis for size-varying target detection, it is computation-ally complex and cannot provide the exact size and locationinformation. Wang et al. proposed an efficient method for multi-scale small target detection method using template matching [8].They used a set of target templates to maximize the object-back-ground ratio. A fast orthogonal search combined with Wavelettransform showed efficient small target detection performance[25]. Recently, Wang et al. proposed to support vector machines inthe wavelet domain and showed the feasibility of multiscale smalltarget detection at low contrast backgrounds [9]. In addition, amulti-level filter-based small target detection method was imple-mented in real-time using DSP, FPGA, and ASIC technologies [26].

Previous detection methods based on multiscale approachessuch as wavelet transform or multiple templates show the feasibilityof size-varying small target detection. However, those methods donot provide optimal detection in terms of the matched filter theoryfor incoming targets. In addition, those methods do not contain anoptimization process for background clutter suppression.

The key idea of this paper is to simultaneously consider boththe size problem (target enhancement) and clutter suppressionproblem by maximizing the SCR in Laplacian scale-space moti-vated by the robust properties of human visual systems (HVS). InHVS, important information is not the amplitude of a visualsignal, but is the contrast between this amplitude at a given pointand at the surrounding location. According to neuro-physiologicalfindings, the response of ganglion cells to contrast patterns islinear in terms of contrast [10]. From this, vision scientistsconclude that contrast is the most important quantity encodedin the streams of our visual system. This fact is true in the wholedetection process. The contrast mechanism is used in spatialfiltering by Laplacian of Gaussian and in target detection proces-sing by comparing the filtered target region and backgroundregion. In addition, a multiscale representation is supported byvision research since the effect of pattern adaptation cannot beexplained by a single resolution theory. After the multiscalerepresentation, the visual system can tune in to an appropriatesize sensitive to the spatial extent, rather than to variations in thespatial frequency during object detection [11]. The multiscalerepresentation is realized via the scale-space theory. Since target

S. Kim, J. Lee / Pattern Recognition 45 (2012) 393–406 395

parameters (position, scale) are optimized by tuning (or search-ing) in scale-space and selection by maxima, we call the proposedmethod Tune–Max of SCR (TM-SCR).

This paper is organized as follows. In Section 2, we mathemati-cally formulate the small target detection problem and present theoverall framework of the proposed system. In Section 3, a new pre-processing spatial filter is presented to remove the heterogeneousbackground. In Section 4, the candidate target detection method isintroduced by maximizing the Laplacian filter responses in scale-space. In addition, the final detection method is presented based onthe Tune–Max of SCR. Various performance evaluations and resultsare explained in Section 5. We conclude this paper in Section 6.

2. Problem formulation and overview of the proposedmethod

In this section, we introduce the basic concept of SCR-basedtarget detection to use the robust properties of HVS. The measure ofSCR is useful to compare the filter performance in target detectionand defined as in Eq. (1) [13]. The conventional signal-to-noise ratio(SNR) does not consider background clutter, which is important inIRST. As shown in Fig. 2, mT and mB represent the estimated targetintensity and background intensity, respectively. sC denotes thestandard deviation of clutter plus noise in background after thefiltering process. The previous filtering methods tried to maximizethe target signal (9mT�mB9) or to minimize clutter (sC) as introducedin Section 1

SCR¼9mT�mB9

sCð1Þ

However, both approaches are not optimal in terms of SCRvalues for the filter design. An ideal filter produces maximal SCRvalues by enhancing target and reducing background noise. Ourfirst key idea is to simultaneously maximize the target signal tobackground difference and to minimize the background clutter toachieve the optimal filter selection using the SCR measure. Thesecond key idea is to insert a novel pre-processing filter beforemaximizing SCR if the test images have horizontal lines. As shown inFig. 2(a), a ship target exists on the horizontal line. The hetero-geneous background generates large background clutter that reducesthe SCR value. The novel pre-processing filter can remove structuralbackgrounds. Details will be introduced in the following sections.

Small infrared target images have blob-like structures asshown in Fig. 3(a). So, we use the parametric target model ofthe point spread function defined as in Eq. (2). It is a reasonablemodel since distant targets are blurred due to the atmosphericrefraction, dispersion, optical defocusing, lens aberration, diffrac-tion, deformation of mirror, and detector tilt [14]. The targetmodel is composed of four parameters: target center position(xc,yc), size (d, diameter), and peak intensity (Ip), as is shown in

Fig. 2. (a) An example image of IRST and (b) target regio

Fig. 3(b). s represents sigma in the Gaussian function. Since the peakintensity is automatically determined after detecting the centerposition, the critical target parameters are (xc,yc,s). So, the size-varying target detection problem can be reduced to the estimationof the parameters (xc,yc,s)

Tðx,y9xc ,yc ,sÞ ¼ Ip exp �ðx�xcÞ

2þðy�ycÞ

2

2s2

( )ð2Þ

In Eq. (1), the difference of target and background intensity(9mT�mB9) can be computed by 2D convolution of the 2D Laplacianfilter (Eq. (3)) with an input image I(x,y). As shown in Fig. 3(c), theon-center filter coefficients (þ) are used to estimate the targetsignal, while the off-surround filter coefficients (�) are used toestimate the background information. This filter is strongly relatedto the contrast mechanism (on-center, off-surround) of HVS intro-duced in the previous section. sC is calculated around the targetregion after the filtering operation to estimate the level of back-ground noise

LoGðx,y,sÞ ¼1

ps41�

x2þy2

2s2

� �e�ððx

2þy2Þ=2s2Þ ð3Þ

However, we cannot estimate the target parameters using onlyEq. (3) since it is not normalized to the scale factor s. This estimationcan be achieved using the scale-space theory established recently byLindeberg [15]. The scale-space theory can mathematically handlethe scale problem. Details of the theory are explained in Section 4.

The final small target detection problem is defined as in Eqs.(4)–(6). F(x,y,s) is a scale-normalized Laplacian image at scalefactor s and defined as in Eq. (4), where s2 is a scale normalizationfactor. Iðx,yÞ is the structural background removed image if thereis a horizontal line in the image. Eq. (5) is used to extract ðx,y,sÞ,the geometrical information of a candidate target

Fðx,y,sÞ ¼ s29LoGðx,y,sÞ Iðx,yÞ9 ð4Þ

ðx,y,sÞ ¼maxðx,y,sÞ

Fðx,y,sÞ ð5Þ

Yðx,y, ~sÞ ¼maxðs9x,yÞ

SCRðx,y,sÞ ¼maxðs9x,yÞ

Fðx,y,sÞ

sCðx,y,sÞð6Þ

After the processing of Eq. (5), we can have a set of candidatetargets. The next step is to obtain the maximal SCR value at (x,y)by varying filter scale s as in Eq. (6). The different scale factors willenhance the target or suppress background clutter. We can obtainthe maximal SCR using those scale factors. sCðx,y,sÞ is thestandard deviation of background clutter after Laplacian filteringat scale factor s and provides a measure of clutter around thecandidate target position of ðx,yÞ. Yðx,y, ~sÞ represents the maximalSCR value achieved at the candidate target in a cluttered back-ground. At this value, we can achieve enhanced target responseswith suppressed background clutter noise. We call the proposed

Background (B)

Target ((T)

n and background region for the SCR computation.

Infrared image

User input ofbackground type

Sensor pose(height, elevation

angle)

RD-LBEF

+

-

Horizontalline

estimation

Pre-filter: RD-LBRF TM-SCR

Pre-detect Final-detect

Tune-Maxof F (x,y,s)

ˆ ˆ ˆ{(x,y,s)}

Tune-Maxof SCR (x,y,s)ˆ ˆ

ˆ ˆY (x,y,s)

Threshold

Pass for generalbackground

Pass for horizontalbackground

Fig. 4. The proposed scale invariant small target detection flow.

Fig. 3. (a) Examples of real target images, (b) parametric target model, and (c) Laplacian of Gaussian for target filtering.

S. Kim, J. Lee / Pattern Recognition 45 (2012) 393–406396

method TM-SCR, Tune–Max of SCR due to the calculation process,in which Tune means searching or varying parameters, Maxmeans selecting a parameter that achieves the max response.Specific calculation methods are introduced in the following threesections. Fig. 4 summarizes the proposed small target detectionmethod. It consists of a pre-filtering part and TM-SCR-baseddetection part. The pre-filter is activated if the backgroundinformation represents a horizontal region. The horizontal line

is estimated using the sensor information. A heterogeneous back-ground is removed using a Row-Directional-Local Background Esti-mation and Removal Filter (RD-LBEF, RD-LBRF). In the TM-SCRprocess, a pre-detection part produces the candidate targets and afinal detection part calculates the maximal SCR values for eachcandidate target. Through a simple thresholding, we can get theoptimal set of targets. Section 3 presents horizontal line estimationand heterogeneous background removal method. Section 4 explains

S. Kim, J. Lee / Pattern Recognition 45 (2012) 393–406 397

the details of pre-detection based on the scale-space theory andintroduces the details of SCR optimizations.

2

2

� = 0HorizonHorizon

Sky

Sea

� �

�

�

–�

Fig. 6. Geometry of sea-based IRST system: (a) relation between sensor height and

horizontal line, (b) camera geometry with field of view and elevation angle (a¼0),

and (c) position of horizontal line when the elevation angle is a.

3. Pre-filtering: heterogeneous background removal

Normally, the LoG filter produces correct responses in homo-geneous backgrounds such as the sky. However, an importantsurveillance area in sea-based IRST is the horizontal region asshown in Fig. 5(a). Incoming targets such as ASSMs or asymmetricships exist around the horizontal line. If a LoG filter is applied tothis image, then we can get a filter response as shown in Fig. 5(b).Note the strong responses in the horizon where the targets areburied. The edge response due to the heterogeneous backgrounddegrades the SCR values, which reduces target detection rate. So,we introduce the horizontal line estimation method and struc-tural background removal method in this section.

3.1. Horizontal line estimation using sensor pose information

Since our system can provide the precise pose information ofinfrared cameras, we can estimate the horizontal line in an imagethrough geometric analysis. Fig. 6(a) shows the overall relation-ships between the sensor height (h) and earth radius (R). Theprojected horizontal line in an image can be found by calculatingthe angle (yh) as in Eq. (7). Since the earth radius (R¼about6400 km) is much larger than the sensor height (h¼10–30 m), yh

is almost 0. So, we can assume that the horizontal line is locatedat the center of an image. Fig. 6(b) represents an enlarged camerageometry diagram with field of view (b) and camera elevationangle (a¼0). If the elevation angle is increased by a (assume �b/2oaob/2) as shown in Fig. 6(c), then the field of view is divedfrom the sky region (ysky) and sea region (ysea). ysky is just aþb/2and ysea is b/2�a. So, if the image height is H, then the estimatedhorizontal line is calculated using Eq. (8)

yh ¼�cos�1 R

Rþh

� �ð7Þ

yhorizon ¼Htanysky

tanyskyþtanyseað8Þ

3.2. Structural background removal filter

If we have prior knowledge of the sky–sea background, thenwe can estimate the horizontal line using the previous method.The horizontal region is obtained by selecting 720 pixels aroundthe horizon line. A heterogeneous background can be removed orreduced using the proposed spatial filter (called Row-Directional-Local Background Estimation Filter (RD-LBEF) as in Eq. (9). Thelocal median filter with a tap size 2nþ1 can predict a hetero-geneous background that is robust to image tilt. The final result isachieved by subtracting the estimated background image from an

Targets

Fig. 5. Problems of the LoG filter in a horizontal image: (a) real infr

input image using Eq. (10). Due to the nature of operation, theproposed method is called the Row-Directional-Local BackgroundRemoval Filter (RD-LBRF). Fig. 7 shows the effect of the proposedmethod on a horizontal image. Given an input as that shown inFig. 7(a), a structural background image is obtained as in Fig. 7(b).Then, the final filter result is obtained as in Fig. 7(c) using Eq. (10).The dotted circles represent incoming targets. The RD-LBRFmethod should be activated when the surveillance area includesa horizontal region.

IRD-LBEF ðx,yÞ ¼medianfIðx�n,yÞ,Iðx�nþ1,yÞ,. . .,

Iðx,yÞ,. . .,Iðxþn�1,yÞ,Iðxþn,yÞg ð9Þ

IRD-LBRF ðx,yÞ ¼ Iðx,yÞ ¼ Iðx,yÞ�IRD-LBEF ðx,yÞ ð10Þ

4. TM-SCR-based small target detection

4.1. Introduction to the Laplacian scale-space theory

In this section, we briefly introduce the scale-space theoryestablished recently [15,16]. The scale-space and automatic scale

Strong response in horizon

ared targets on the horizontal line and (b) LoG filtering results.

Fig. 7. The effect of structural background removal filter in horizontal region: (a) cropped horizontal image, (b) estimated background image using the proposed row-

directional-local background estimation filter, and (c) background removed image. Dotted circles represent targets.

S. Kim, J. Lee / Pattern Recognition 45 (2012) 393–406398

selection theory provided ground-breaking impacts on the corre-spondence problems in computer vision such as stereo matching,object recognition, and vision-based mobile robot localization[17–19]. The scale-space represents an image as a set of imageswith different resolutions [15]. Different resolution images areacquired by consecutive convolutions with Gaussian kernels.When g(s) represents a Gaussian kernel with scale (s) and I(x,y)represents an image, a Gaussian scale-space image is representedas L(x,y,s)¼ I(x,y)g(s). The Gaussian scale-space image itself doesnot provide structural information. Scale-space derivatives aremuch more useful for the structural analysis in multiscale.Especially, the 2nd derivative of Gaussian or Laplacian of Gaussian(LoG) kernel can extract certain information of circular blobstructures. Since the Laplacian of Gaussian kernel has on-center,off-surround coefficients, it can respond to circular patterns. Theimportant thing is that the amplitude of spatial derivativesdecreases with scale. In order to maintain scale invariance, thederivative function should be normalized with respect to the scalefactor. So, the normalized Laplacian scale space (LSS) is defined asin Eq. (11), a set of scale-space images. Note that the image sizesare the same for all scale-space images, which alleviates thecorrespondence problems among scale-space images

LSSðx,yÞ ¼ fFðx,y,sÞ9s¼ s0,s1,. . .,sNg ð11Þ

F(x,y,s) represents a scale-normalized Laplacian image at scale s

defined in Eq. (4). Note that the minimum value (s0) should be0.5 for small target (around 2�2 pixels) detection. In this paper,we use scale parameters that have a range of [0.5–6] with sn¼

kns0. sn denotes the successive levels of scale-space representationwith k denoting the factor of scale change between successivelevels. For example, if n¼0–12, s0¼0.5, k¼1.2 then s can be [0.500.60 0.72 0.86 1.03 1.24 1.49 1.79 2.14 2.57 3.09 3.71].

The key idea of automatic scale selection from the Laplacianscale-space is to select a characteristic scale that shows a globalmaximum for F(x,y,s) over scales. This scale adaptation mechanism isvery important to remove the spurious clutters around corners or theedges around clouds or man-made architectures as shown in Fig. 8.For corners or edges, the normalized Laplacian scale-space showsconstant responses along the scale axis. However, the normalizedLaplacian scale-space shows peak response at the blob target. Besides

such clutter rejection property, the normalized Laplacian scale-spacecan automatically find the target scales by finding maxima from thegraph. As discussed in the previous section, if a true target exists onthe horizontal line, then the edge background structure should beremoved using the RD-LBRF method. Fig. 9 shows examples ofautomatic scale selection for the two different targets. The estimatedcharacteristic scales are 1.75 and 3.5 for the small (5�5) and large(10�10) targets, respectively.

4.2. Candidate target extraction in 2D image

It is important to achieve both selectivity and invariance forachieving a robust target detection [20]. The selectivity andinvariance are strongly correlated to the human brain for highlevel visual processing. Selectivity means that we have to knowthe specific target position and scale. Invariance means that wehave to detect targets regardless to the positional changes andscale changes. This can be solved using the Tune–Max method[17]. The Tune is simply to conduct convolution with theLaplacian kernel in scale-space. The Max is simply to select themaximal filter responses over scale-space. Since the direct 3DMax is complex, we use two step-based approaches as shown inFig. 10.

In Step 1, the spatial Tune–Max is used to estimate thepossible target position parameters. Given a test image as shownin Fig. 10(a), the spatial Tuning (convolution) with the Laplaciankernel (scale s), we can obtain the Laplacian scale-space images asshown in Fig. 10(c). Through the Max selection of F within localwindow W as in Eq. (12), we can obtain the possible targetposition (‘þ ’) as shown in Fig. 10(b). A 3�3 local window is asuitable choice considering the computational complexity

Fðx,y,siÞ4Fðxw,yw,siÞ, ðxw,ywÞAW ð12Þ

In Step 2, the scale Tune–Max is used to estimate the possibletarget scale parameter. Tuning with the Laplacian kernel in scale-space produces filter responses in thematic-scales as shown inFig. 10(c). From this, we can make the normalized Laplacianfunction at the maximum position ðxc ,ycÞ as shown in Fig. 10(d).After the Max selection of F using Eq. (13), we can estimatethe target scale as shown in Fig. 10(e). si�1, si, siþ1 represent

scale

F

F

scale

scale

F

Corner clutter:Reject

Edge clutter:Reject

Blob target:Candidate

Backgroundstructure

Characteristic scale

Fig. 8. The mechanism of clutter rejection in Laplacian scale-space.

Laplacian scale-space Scale selectionTarget size

s = 1.75

s = 3.5

Small:(5x5)

Large:(10x10)

Fig. 9. Examples of Laplacian scale-space at the target centers and corresponding automatic scale selection for the small (5�5) and large (10�10) targets. The circles in

the right figures represent the target scales at the maximum responses of the Laplacian scale-space.

S. Kim, J. Lee / Pattern Recognition 45 (2012) 393–406 399

the lower, current, upper scale values, respectively, in Laplacianscale-space

Fðx,y,siÞ4Fðx,y,si�1Þ & Fðx,y,siÞ4Fðx,y,siþ1Þ ð13Þ

The Tune–Max process in Laplacian scale-space provides twoimportant functions in pre-detection. One is the capability ofgeometric information extraction for candidate targets. As can beseen in Fig. 10, the Tune–Max generates only one set of target

+ˆ ˆ(xc,yc)

+

Test pattern

Tune-Max in space

FFd = 6s

++

+

+

+ +

LaplacianScale-space

Tune-Max in scale

Pre-detections

s = si+1

s = si

ˆ

s = si-1

Fig. 10. The Tune–Max method estimates the target parameters of position and scale. In Step 1, the Tune–Max in space can provide the specific target position. In Step 2,

the scale Tune–Max in Laplacian scale-space can provide the specific target scale.

Fig. 11. Tune–Max example of SCR computation. (a) Test target image and (b) tuning of SCR function according to scale and selection of maximum value.

S. Kim, J. Lee / Pattern Recognition 45 (2012) 393–406400

information for each target blob. So it does not need a clusteringprocess after thresholding. The conventional detection methodssuch as mean subtraction, median subtraction, and Top-hat filterrequire an additional clustering process. The other function is thecapability of clutter rejection. During the spatial Tune–Maxprocess, only the local maxima points remain. For these candidatepoints, only the scale maxima points remain through the Tune–Max process in the scale direction. Through these processes, wecan obtain target-like points with scales.

4.3. Final detection: maximization of SCR

After the process of pre-detection based on the Laplacianscale-space theory, we can have candidate targets with imageposition and size information. The last step is to maximize theSCR measure defined in Eq. (6). We use the Tune–Max methodused in the previous section to obtain the maximum of SCR. Sincethe target position is given, tuning is processed by varying onlythe scale factor s.

In the numerator of Eq. (6), the Fðx,y,sÞ values are alreadyavailable since the scale-space images are built in the pre-detectionprocess. The required computation is the denominator in Eq. (6). Thestandard deviations (sCðx,y,sÞ) of background clutter (filtered image)are calculated around the background region defined in Fig. 2,except the target region. According to several experiments, the bestchoice of background diameter is 12s, which is two times the targetdiameter (d¼ 6s). Fig. 11(b) presents an example of the SCRmaximization for the distant dim target shown in Fig. 11(a). Thecross (‘þ ’) in Fig. 11(a) is the center position of a candidate targetobtained in the pre-detection step. In Fig. 11(b), the tuning graph ofSCR ( ) is obtained using the F ( ) and sC ( ) valuesaccording to scale s. In this example, the F ( ) graph is used toestimate target size at the cross point. F shows the peak value ats¼ 0:73. So, the estimated target size is 4.43 (sz¼ 6s) pixels. sC iscalculated for each Laplacian scale-space image around the targetposition. SCR defined as F over sC shows a peak value of 11.5 at thescale ~s ¼ 2:59. The final target detection results are obtained bythresholding the maximal SCR values. If we do not perform the

σ3 = 0.98

SCR3 = 9.45

ISCR = 5.79

BSF = 23.60

σ1 = 23.22

SCR1 = 1.63

σ2 = 1.47

SCR2 = 7.26

ISCR = 4.45

BSF = 5.71

Edge response

1. Input image

2. A Lapalcian scale-space image

3. A Lapalcian scale-space image after RD-LBRF

Fig. 12. Effect of RD-LBRF in maximal SCR calculation: (a) input image with corresponding 3D surf plot, (b) a Laplacian scale-space image and maximal SCR, and (c) a

Laplacian scale-space image and maximal SCR.

IR image

Top-hat Adaptivethreshold

8-neighborclustering

Output

TM-SCR OutputThreshold

Proposed method

Baseline method

RD-LBRF

(if horizon)

Fig. 13. Test platforms of the proposed and baseline methods (Top-hat).

S. Kim, J. Lee / Pattern Recognition 45 (2012) 393–406 401

proposed SCR maximization process, then the detection will be pooror many false alarms will occur. In Fig. 11, let us assume that a SCRvalue is estimated using only the max position of F. The clutter levelis so high that the SCR is as low as 4.3. If we use an SCR threshold of8, then this important target will be missed.

In the SCR computation, it contains the concept of contrast. Ifthe filtered target signal is larger than the filtered backgroundclutter, then SCR shows high signal to background contrast. Wecan use the maximal SCR values to search targets as soon aspossible in large IRST imagery with limited processing time.

During the final detection process, the pre-filter (RD-LBRF) canimprove the SCR values if the probing area is a horizontal region.Fig. 12 shows the effect of RD-LBRF in SCR maximization. Fig. 12(a)is a test image with a small target around the horizontal line. Thecorresponding 3D surf plot visualizes the signal intensity. Thestandard deviation of background is 23.22 and SCR is 1.63.Fig. 12(b) shows a Laplacian scale-space image where a maximalSCR is achieved. The standard deviation is 1.47 and SCR is 7.26. Weuse the improvement of SCR (ISCR) and background suppressionfactor (BSF) to measure the performance of the spatial filter [13].ISCR is defined as SCRout/SCRin and BSF is defined as sin/sout. A goodfilter has high ISCR and BSF. A Laplacian scale-space has ISCR¼4.45

and BSF¼15.71. Fig. 12(c) presents a Laplacian scale-space imageafter RD-LBRF, where a maximal SCR is obtained. It has SCR¼9.45,ISCR¼5.79, and BSF¼23.60 that are larger than those of Laplacianscale-space without RD-LBRF.

5. Experimental results

In this section, we perform three kinds of experiments toevaluate the properties of TM-SCR. Its performance is comparedwith that of the Top-hat filter-based method (baseline) since it is

S. Kim, J. Lee / Pattern Recognition 45 (2012) 393–406402

well studied and shows good performance in terms of the smalltarget detection problem [3,21–24]. Fig. 13 shows the overalltarget detection methods of the proposed method and the base-line method. The proposed method is composed of RD-LBRF, TM-SCR and thresholding. RD-LBRF is activated when the inputimages have a horizontal region. The number of Laplacian scale-space images is 12, and the scale range (sigma) is from 0.6 to 6.

SCR=1.38

SCR=3.19

SCR=4.17

SCR=5.21

SCR=8.55

SCR=13.12

#3

#11

#21

#50

#74

#99

SCR=1.38

SCR=3.19

SCR=4.17

SCR=5.21

SCR=8.55

SCR=13.12

#3

#11

#21

#50

#74

#99

Fig. 14. Test results for the incoming target sequence: (a) input sequence, (b)

The baseline method is composed of Top-hat filtering, adaptivethresholding, and 8-neighbor clustering. The window size is set to7, and the adaptive thresholding is conducted based on thestandard deviation of background filter output. If a filter outputover standard deviation like Eq. (1) is larger than the threshold,then the pixel is detected as a target point. Then, the 8-neighbor-based clustering is further processed to get the target information

No target detection

No target detection

No target detection

No target detection

detection results using TM-SCR, and (c) detection results using Top-hat.

Fig. 16. Estimation of target size using TM-SCR. The dotted line represents the

measured target sizes via TM-SCR, and the solid line represents the estimated

target sizes.

Uniform Cloudy Sea Sky+Ground Ground Compound0

10

20

30

40

50

60

No.

of f

alse

ala

rms/

imag

e (D

R=9

0%)

Type of background

TM-SCRTop-hat

Fig. 17. Clutter rejection capability for six types of backgrounds. TM-SCR method

shows low number of false alarms compared with that of the Top-hat method

with the same detection rate.

S. Kim, J. Lee / Pattern Recognition 45 (2012) 393–406 403

of position and size. We change the threshold values to meet theevaluation conditions.

For comparison metrics, we use the detection rate (DR) andfalse alarms per image defined in Eq. (14). If the distance betweena ground truth and a detected position is within a threshold (5pixel), then the detection is declared as being correct. The DRrepresents how many correct targets are detected among truetargets, and the FA represents how many false targets are detectedper image on average

DR %½ � ¼Number of correctly detected targets

Number of true targets� 100

FA #½ � ¼Number of incorrectly detected targets

imagesð14Þ

In the first experiment, the detection capability is evaluated for theincoming target sequence. For fair comparison, we set the twomethods to have the same false alarms by changing the thresholds.The test sequence is acquired using a scan-based LWIR camera thatprovides 720�480 image resolution. We use a landing scenario of atarget (fighter). So, the fighter approaches from a far distance at highaltitude and passes by the IR camera at low altitude. We use 100frames that are sampled at one frame per second from the acquiredvideo. Fig. 14(a) shows partial examples of the test sequence wherethe frame numbers, ground truths of target (rectangles), SCR valuesfor input images and 3D views of the target regions are overlaid. TheSCR for an input image is calculated based on the hand-segmentedtarget region and background region. Note that the initial SCR isaround 1, which is almost the same level of background noise. TheFA values are set to 2.8/image (288/100 images). At this condition,the detection rate of TM-SCR is 100% and that of the Top-hat is 81%for 100 frames. Fig. 14(b) and (c) shows the target detection resultsusing TM-SCR and Top-hat, respectively. The rectangles representthe ground truths, while the circles represent detected targets withtheir sizes. Note that the TM-SCR detected all targets with the sameFA as that of the Top-hat. The Top-hat missed targets in frame 1 to20, and 22, whose SCRs are at the range of 1–3, and showed a poordetection result for the last frame, whose target size is around15�15 as shown in Fig. 15. From this experiment, the proposedTM-SCR shows superior detection capability from the small dimtargets to the large targets with the same level of false alarmscompared to the Top-hat method. This fact means that TM-SCR canprovide upgraded defense capability since it can quickly detectdistant targets compared to Top-hat. In addition, TM-SCR canprovide target size information. Fig. 16 shows the raw target sizesestimated by TM-SCR and filtered target sizes using the localaverage. Since the input sequence is very noisy, the estimated targetsizes (radius) are fluctuated ( ). If we apply the local meanfilter to this data, then we can get smoother size information( ). The trend of target size is increasing according to theframe index. So, we can decide that the target is approaching. We

Fig. 15. Detection results of a large target (100th fra

can conclude that the size profile is valuable information to thedecision of threats.

In the second experiment, the number of false alarms iscompared for various cluttered backgrounds to check the cap-ability of clutter rejection. Contrary to the first experiment, thedetection rates of both methods are set to the same value of 90%by changing the thresholds. The capability of clutter rejection ismeasured by the average number of false alarms per image. Thebackground types are uniform sky (image set 1, 30 images), cloud

me, size 15�15): (a) TM-SCR and (b) Top-hat.

Table 1Statistical comparison among the two proposed methods and the well-known

Top-hat method.

Criteria Method

TM-SCR RD-LBRFþTM-SCR Top-hat [24]

Detection rate (%) 93.3% (210/225) 97.8% (220/225) 31.1% (70/225)

False alarms/image 0.022 (5/225) 0.022 (5/225) 3.2 (720/225)

Threshold 6.0 6.0 5.6

S. Kim, J. Lee / Pattern Recognition 45 (2012) 393–406404

cluttered sky (image set 2, 18 images), sea with sun-glint (imageset 3, 15 images), sky with ground (image set 4, 12 images),ground only (image set 5, 14 images), and compound (sky, sea,island—image set 6, 18 images). Those image sets are acquiredusing a scan-based LWIR camera and a staring MWIR camera.Fig. 17 shows the evaluation results of TM-SCR and Top-hat. Forall cases, TM-SCR shows lower false alarms per image. Theaverage numbers of false alarms per image for TM-SCR and Top-hat are 5.94 and 25.70, respectively. So, the proposed TM-SCR canreduce the false alarms by 4.3 times more than that of Top-hat.This figure is noticeable since the detection rates of both methodsare the same. Fig. 18 shows target detection results using TM-SCRand Top-hat for the above mentioned backgrounds. TM-SCRshows better clutter rejection performance than that of Top-hatfor cloud clutter, sun-glint, and so on.

In the third experiment, we applied the proposed methods toa real test set obtained by controlling a ship target in a horizontalbackground. A small ship equipped with GPS approaches froma distant point. 225 images are acquired during the incom-ing target motion. The performance of the proposed methods(TM-SCR, RD-LBRFþTM-SCR) is compared with that of the Top-hat filter-based method. For fair comparison, the thresholds aretuned for high detection rate and low false alarms. Table 1summarizes the statistical performance in terms of detection rateand false alarms/image. Note that the proposed detection method

Set1: uniform sky

Set2: cloudy sky

Set3: sun-glint

Set4: sky with ground

Set5: ground only

Set6: compound

TM-SCR

TM-SCR

TM-SCR

TM-SCR

TM-SCR

TM-SCR

Fig. 18. Target detection examples for image set 1–6. (Left) TM-SCR-based ta

(RD-LBRFþTM-SCR) outperforms the well-known Top-hat detec-tor for real test scenario. In addition, RD-LBRFþTM-SCR shows anupgraded detection rate compared with that of the TM-SCRmethod with the same threshold. Fig. 19 shows partial compar-ison examples of the horizontal target detection problem. Initi-ally, there are two objects in the right horizontal line, rightuncontrolled object (probably merchant ship) passing from rightto left, and the left controlled object is a true incoming target.We disregarded the right object during the performance evalua-tions. The Top-hat method produces several false detectionsand misses the target of interest. However, the proposed method(RD-LBRFþTM-SCR) produces upgraded detection rate and reducedfalse alarms.

Top-hat

Top-hat

Top-hat

Top-hat

Top-hat

Top-hat

rget detection results and (right) Top-hat-based target detection results.

TM-SCR RD-LBRF+TM-SCR Top-hat

Fig. 19. Target detection comparisons among TM-SCR, TM-SCR after RD-LBRF, and Top-hat method: (a) detection results using the TM-SCR method, (b) detection

results using TM-SCR after RD-LBRF, and (c) detection results using the Top-hat method. The dotted squares denote true targets and the circles represent detected

targets.

S. Kim, J. Lee / Pattern Recognition 45 (2012) 393–406 405

6. Conclusions

In this paper, we proposed a novel method of Tune–Max of SCR(TM-SCR) in scale-space to solve both the scale problem and clutterrejection problem motivated by the robust properties of humanvisual system such as contrast mechanism, multiscale representa-tion, and scale adaptation. The key idea is to simultaneouslyenhance the target information and minimize the backgroundclutter in scale-space. In pre-detection, the candidate targets areextracted by maximizing the Laplacian scale-space images. In thefinal detection, SCRs of the candidate targets are maximized bychanging the scale parameter. If there is a horizontal background,then the TM-SCR method can be upgraded using a pre-filter, calledthe row-directional-local background removal filter (RD-LBRF)before the TM-SCR process. RD-LBRF can remove the heterogeneousbackground that leads to increasing SCR. Due to these processes, theTM-SCR-based approach can detect both dim small (long range)targets and strong large targets regardless of background clutteraccording to the results of three types of experiments. In addition,TM-SCR can continuously provide target size information, which isuseful for making decision about threats. From various evaluations,we can conclude that TM-SCR is an effective and powerful detectionmethod in IRST applications.

Acknowledgments

This research was supported by the 2011 Yeungnam Univer-sity Research Grant.

References

[1] R.C. Warren, Detection of Distant Airborne Targets in Cluttered Backgroundsin Infrared Image Sequences, Ph.D. Thesis, University of South Australia,2002.

[2] J. Barnett, Statistical analysis of median subtraction filtering with applicationto point target detection in infrared backgrounds, Proceedings of the SPIE1050 (1989) 10–15.

[3] V.T. Tom, et al., Morphology-based algorithm for point target detection ininfrared backgrounds, Proceedings of the SPIE 1954 (1993) 2–11.

[4] S.D. Deshpande, et al., Max-mean and max-median filters for detection ofsmall-targets, Proceedings of the SPIE 3809 (1999) 74–83.

[5] R. Nitzberg, et al., Spatial filtering techniques for infrared (IR) sensors,Proceedings of the SPIE 178 (1979) 40–58.

[6] W.A.C. Schmidt, Modified matched filter for cloud clutter suppression, IEEETransactions on Pattern Analysis and Machine Intelligence 12 (6) (1990)594–600.

[7] D.J. Gregoris, S.K.W. Yu, S. Tritchew, Detection of dim targets in FLIRimagery using multiscale transforms, Proceedings of the SPIE 2269 (1994)62–71.

[8] G. Wang, T. Zhang, L. Wei, N. Sang, Efficient method for multiscale smalltarget detection from a natural scene, Optical Engineering 35 (3) (1997)761–768.

[9] Z.C. Wang, J.W. Tian, J. Liu, S. Zheng, Small infrared target fusion detectionbased on support vector machines in the wavelet domain, Optical Engineer-ing 45 (7) (2006) 076401.

[10] R. VanRullen, Visual saliency and spike timing in the ventral visual pathway,Journal of Physiology (Paris) 97 (2003) 365–377.

[11] J. Fiser, S. Subramaniam, I. Biederman, Size tuning in the absence of spatialfrequency tuning in object recognition, Vision Research 41 (15) (2001)1931–1950.

[13] B. Zhang, T. Zhang, Z. Cao, K. Zhang, Fast new small-target detectionalgorithm based on a modified partial differential equation in infraredclutter, Optical Engineering 46 (10) (2007) 106401.

[14] J.–P. Ardouin, Point source detection based on point spread function sym-metry, Optical Engineering 32 (9) (1993) 2156–2164.

[15] T. Lindeberg, Feature detection with automatic scale selection, InternationalJournal of Computer Vision 30 (2) (1998) 77–116.

[16] K. Mikolajczyk, C. Schmid, Indexing based on scale invariant interest points,in: Proceedings of the Eighth IEEE International Conference on ComputerVision (ICCV’01), vol. 1, 2001, pp. 525–531.

[17] S. Kim, I.S. Kweon, Object recognition using a generalized robust invariantfeature and Gestalt’s law of proximity and similarity’’, Pattern Recognition 41(2) (2008) 726–741.

[18] D.G. Lowe, Object recognition from scale invariant features, in: Proceedingsof the Seventh IEEE International Conference on Computer Vision (ICCV’99),1999, pp. 1150–1157.

[19] S. Se, et al., Vision-based mobile robot localization using scale-invariantfeatures, in: Proceedings of the IEEE International Conference on Roboticsand Automation (ICRA’01), 2001, pp. 2051–2058.

S. Kim, J. Lee / Pattern Recognition 45 (2012) 393–406406

[20] C. Cadieu, et al., A model of V4 shape selectivity and invariance, Journal ofNeurophysiology 98 (2007) 1733–1750.

[21] W. Peizhi, et al., A method for automatic infrared point target detection in asea background based on morphology and wavelet transform, Proceedings of

the SPIE 5286 (2003) 248–253.

[22] J.-H. Luo, H.-B. Hi, J. Liu, An algorithm based on spatial filter for infrared smalltarget detection and its application to an all directional IRST system, Proceed-ings of the SPIE 6279 (2007).

[23] Z. Chen, et al., Small target detection algorithm based on average absolutedifference maximum and background forecast, International Journal ofInfrared and Millimeter Waves 28 (2007) 87–97.

[24] A. Toet, T. Wu, Small maritime target detection through false color fusion,Proceedings of the SPIE 6945 (2008).

[25] E. Abdelkawy, Donald McGaughy, Small infrared target detection using twodimensional fast orthogonal search (2D-FOS), in: Firooz A. Sadjadi (Eds.),Proceedings of the Automatic Target Recognition XIII, 2003, pp. 179–185.

[26] Z. Zhang, Z. Cao, T. Zhang, L. Yan, Real-time detecting system for infraredsmall target, Proceedings of the SPIE 6786 (2007) 678656.

Sungho Kim received his B.S. degree in Electrical Engineering from Korea University, Republic of Korea in 2000 and his M.S., Ph.D. degrees in Electrical Engineering andComputer Science from Korea Advanced Institute of Science and Technology, Korea in 2002, 2007, respectively. During 2007 and 2010, he was a Senior Researcher in theElectro-Optics Laboratory at the agency for defense development (ADD). Since 2010, he has been a Professor of Electronic Engineering at Yeungnam University. His currentresearch interests include small target detection, object recognition, human visual perception theory, mobile robot localization, and visual feature. He is a Member of theIEEE, the Institute of Electronics Engineering of Korea (IEEK), the Korea Institute of Military Science and Technology (KIMST), and Korea Robotics Society (KRS).

Joohyoung Lee received his B.S. and M.S. degrees in Electronics Engineering from Dankuk University, Republic of Korea in 1990 and 1992, respectively. Since 1992, he hasbeen a Senior Researcher in the Electro-Optics Laboratory at the agency for defense development (ADD). His research interests include analog and digital signal processingfor IRST, low-noise electronics, IRST system test and evaluation for small target detection.

本文献由“学霸图书馆-文献云下载”收集自网络，仅供学习交流使用。

学霸图书馆（www.xuebalib.com）是一个“整合众多图书馆数据库资源，

提供一站式文献检索和下载服务”的24 小时在线不限IP

图书馆。

图书馆致力于便利、促进学习与科研，提供最强文献下载服务。

图书馆导航：

图书馆首页 文献云下载 图书馆入口 外文数据库大全 疑难文献辅助工具