An Effective Segmentation of Moving Objects by a Novel Local Regions-Based Level Set

Meriem Boumehed1, Belal Alshaqaqi1, Abdelaziz Ouamri1, Mokhtar Keche1, Abdullah Salem

Baquhaizel1, Mohamed El Amine Ouis1.

(1) Signals and Images Laboratory, Department of Electronics, Faculty of Electrical Engineering, University of Science and Technology ORAN - Mohamed Boudiaf - USTO - MB , B.P. 1505 El

Mnaouer , Oran ,31000, Algeria. m_boumehd@yahoo.fr

ABSTRACT

This paper presents new local regions based level set model for segmenting multiple moving objects in video sequences captured by a stationary camera. The main idea evolves around the reformulation of well-known global energy in local way, by utilizing little squared windows centered on each point over a thin band surrounding the zero level set, hence the object contour can be reshaped into small local interior and exterior regions that lead to compute a family of adaptive local energies. Moreover, we propose to adapt the smoothness of the contours with an automatic stopping criterion. The proposed method has been tested on different real videos, and the experiment results demonstrate that our algorithm can segment effectively and accurately the moving objects.

1. INTRODUCTION

Active contour methods are of widespread interest due to a broad number of applications in diverse disciplines that include computer vision [13-15]. The main idea is to evolve a curve towards the boundary of the object so as to minimize a given energy functional in order to produce the desired segmentation. The existing active contour models can be classified into two main categories: edge-based models [16,17,19,22,24] and region-based models [6-11,20,21,23,25-27]. Typically, the former ones use image gradient for identifying object boundaries, which are generally sensitive to noise and weak edges. To overcome such problems, region information (e.g. intensity, color and texture descriptors) has been used in active contours, which usually model the foreground and background as regions with certain homogeneity constraints, and then determine an energy optimum where the model well ts the object. Since this type is based on

the statistical information inside and outside the contour, it has been found less sensitive to image noise and could deal with objects which have weak edges. In practice, implicit or non-parametric active contour models (level set methods) are broadly utilized in situations where contour topology changes in deformation, which cannot be simply handled by explicit or parametric active contours (snakes). A major class of region-based level set methods is proposed to minimize the well-known Mumford and Shah (MS) model [21]. Extended from this model, Chan and Vese [12], and Yezzi et al. [7] proposed two models which approximate the object by two-phase piecewise constant (PC) function. The energy functional of these models is minimized via the gradient descent equation with respect to the level set function [18]. However, these approaches tend to model regions using global statistics are usually not adequate for segmenting heterogeneous objects. To deal with this limitation, Vese and Chan [8] and Tsai et al. [25] proposed an advanced piecewise smooth (PS) models. These models have improved the PC model performance on intensity inhomogeneity. However, these models are usually computationally expensive. In addition, by using global statistics, the above models usually have difficulty to extract heterogeneous objects. In order to overcome this drawback, localized region-based models have been proposed, which use local information. For instance, Li et al. [1] proposed the local binary tting (LBF) model with a kernel function for more accurate and efficient segmentation of images with intensity non-homogeneity. This method embeds the local statistical information of intensity into a region-based active contour model. Some related methods were recently proposed in [3, 4], which have similar capability of handling intensity inhomogeneity as the LBF model. Lankton et al. [9] proposed a framework allows the foreground and background to be modeled in terms of smaller local regions, instead of representing them with global statistics. They reformulated global energies based on PC model in a local way. The localization can improve the segmentation, but the omission of global characteristics

978-1-4673-5540-7/13/$31.00 ©2013 IEEE

2013 8th International Workshop on Systems, Signal Processing and their Applications (WoSSPA)

191

increases the sensitivity to the contour initialization. The scale of the localization is also hard to run adaptively for different image objects. Although such methods [1, 3, 4, 9] solved the problem of intensity inhomogeneity, they suffer from the initial contour placements and multiple objects, which limit their practical applications. To deal with such drawbacks, Wang et al. [2] proposed a novel local and global intensity tting (LGIF) model in a variational level set formulation, which combines the advantages of the LBF model [1] and the PC model [6, 7, 21]. Tao and Tai [5] proposed an improved model based on graph cut optimization for the multiple piecewise constant model of Chan and Vese [6] and geodesic active contour model proposed by Caselles et al. [17]. In this paper, we propose a novel region-based level set model that can effectively deal with the problem of contour initialization and multiple objects. In addition, it can accurately segment multiple objects simultaneously. Indeed, the proposed framework presents two main contributions. First, we reformulate the well-known global mean separation energy presented by Yezzi et al. [7], by using little squared windows centered at each point over a thin band surrounding the zero level set. Second, we adapt the smoothness of the contours, and the accuracy of the objects’ perimeter of different shapes with a proposed stopping criterion. The remainder of this paper is organized as follows. In Section 2, we briefly review the level set technique presented by Osher and Sethian [18], recall the definition of the global mean separation energy. The proposed framework is presented in Section 3. Section 4 is devoted to the experimental results. Finally, conclusions are drawn in Section 5.

2. BACKGROUND

2.1. Level set formulation

The main idea of the level set method that was developed by Osher and Sethian [18] is to represent implicitly a closed contour ∈ R2 as the zero level set of a higher dimensional function . In our framework, the main purpose is to segment the moving object in a given image I(x, y) defined in the domain , by evolving the implicit function ((x, y), t) according to an appropriate partial differential equation, that contains the embedded motion of (t =0) along the normal n with an energy E. We have thus, the following evolution equation for :

0 (1)t E+ ∇ ∇ = Generally, global region-based methods minimize the global energy E by fitting a model to each region, such as Mumford-Shah [21], Chan-Vese [6], and Yezzi et al [7].

2.2. Global mean separation energy

For a given image I(x, y) defined in the domain , Yezzi et al. [7] assumed that an object of interest in I(x, y) is formed by two distinctive regions of different values in and out, that represent the global mean intensities of the interior and exterior regions of the curve . This model minimizes the energy when the interior and exterior regions are well approximated by means of in and out.

( )1 22

(2)E dxdyoutY = − −Ω

in

( )

( )( , ) . ( , )

( , )(3)

x y x y dxdyinsidex y dxdyinside

Ω=

Ω

H Iin H

( )( )

( )( )1 ( , ) . ( , )

1 ( , )(4)

x y x y dxdyoutsideout x y dxdyioutside

−Ω=

−Ω

H I

H

where H( (x, y)) is the signed distance function (SDF) given by:

( )1 ( , )

( , ) 0 ( , )

1 11 sin ( , )2

x yx y x y

x y

εε

π εε π ε

>= <−Γ

+ + ≤

H (5)

Figure 1. The proposed algorithm for localizing moving

objects

3. THE PROPOSED MODEL

In this section, we describe our proposed model of local regions based level set. The key idea is to reformulate the

192

global mean separate energy EY defined by the equation (2) in a local way. By introducing little squared windows W of size L× L centered on each point over a thin band around the zero level set, the object boundary can be reshaped into small local interior and exterior regions that are characterized by their local mean intensities values

inL and outL. This leads to compute the family of the local energies, then search the minima of the local energies by evolving the contour. In order to guarantee that remains as it was initialized, a re-initialization step is incorporated [28]. We also propose to add up a regularization step that allows us on one hand to determine an automatic stopping criterion for reaching the object boundary, and on the other hand to arrange the weight of the curvature that enables the contour to smooth easily. The main procedure is given by the diagram of Fig.1.

3.1. Construction of the squared windows

Here, we construct a squared window W with size L× L as illustrated in Fig.2, centered on each point p = (x, y) in the narrow band of width D. Typically, the value of D is very small and in the same time it ensures the well closing of the curves. The window function is given by:

2

2( )

2

2

X x LposY y Lpos

X,YX x LnegY y Lneg

= +

= +=

= −

= −

W

(6)

The length L of the window depends on the perimeter of the object in the image and its position against the camera. For instance, whether attempting to segment objects that are very small with near position, a small L should be used, and vice-versa.

Figure 2. Construction of the windows.

3.2. Computation of the local mean intensities

In this step, the local mean intensities inL and out of the interior and exterior of the local regions are defined in terms of the window function as follows:

( )( )

( ). ( ) . ( )( ). ( )

(7)X,Y H X,Y X,Y dXdYL X,Y H X,Y dXdY

=W I

in W

( )( )

( )( )( ). 1 ( ) . ( )

( ). 1 ( )(8)X,Y H X,Y X,Y dXdY

outL X,Y H X,Y dXdY−

=−

W IW

3.3. Determination of the local energies

We define the local energies by replacing the global mean intensities in equation (2), by their local equivalents from (7) and (8) as follows:

( )21 (9)2Y L inL outLη η= − −E

The optimum of the local energies is obtained when inl and outl are the most different. In some cases, this is more desirable than trying to t a constant piecewise model. Therefore, it is preferable the local foreground and background means being different rather than constant. This allows these energies to reach the object edges very well without being distracted when interior or exterior regions are not uniform. Thus, the energy function EYL is minimized with respect to for getting the corresponding local force as:

( )( ) ( )( )

( ) ( )( )( )

( , )

, .H ,

.

( , )

, . 1 H ,

X Y inLX Y X Y dXdY

inWY L inL outL

X Y outLX Y X Y dXdY

outW

−+

= − −

−

−

IW

F

I

W

(10)

3.4. Evolution of level set function

All the geometric quantities were been evaluated in the previous steps with the generic force FYL that drives the evolution of the curve over time in the narrow band. The evolution equation for is defined as:

1, , (11).t t

X Y X Y Y Lt Fϕ ϕ −= + Δ

where t is the time step bounded by the Courant-Friedrichs-Lewy (CFL) condition [29] to ensure a numerical stability over time given by :

( )

0.5max( )

(12)tFY L ε

Δ =+

Outside

Inside

193

3.5. Regularization

In this section, we propose a step of regularization, which makes our algorithm more accurate, robust, and efficient. The basic idea here evolves around the adaptation of the smoothness and the accuracy of the object boundary with an automatic stopping criterion of the level set evolution. The smoothness and the accuracy are guaranteed by the proposed regularizing term tω , which added to the local force in equation (10). This term has a strong effect on the convergence, smoothness and stability of the object boundary. Besides, it prohibits the contour from wrapping around tiny pieces of noise.

1 1 (13)*t t kω α+ +=

Here is an adaptive weight for penalizing the curvature K.

In general, the smoothness and the accuracy of the object boundary are guaranteed by penalizing the arclength of the curve and weighting this penalty by a constant parameter ranges between 0 and 1 [6, 7, 9, 10], whereas the stopping criterion is defined by a fixed iteration times so that an empirical and wide number of iterations are usually used to reliably extract the boundary of interest. Here, we propose to make the weight variable and adaptive with an automatic stopping criterion as follows:

*(1 ) 1

1*(1 ) 1

C if C Ct t t t t

tC if C Ct t t t t

α αα

α α

− − > −=+

+ − < − (14)

where C is the stopping criterion given by:

2 ( , ). ( , )1( )( , ) ( , )1

(15)X Y X Y dXdYt tCt X Y dXdY X Y dXdYt t

ϕ ϕϕ λ

ϕ ϕ−= ≥

+ − The parameter depends on the value of the stopping criterion and adapts with it to control the convergence, whereas the current value Ct increases, the contour moves inward and the evolution speed approaches to zero, so is decreased in order to let the curve converges towards the accurate object location, and when Ct decreases, is increased so the contour moves outward and the evolution speed is slow. In equation (15), the current level set function t is compared with the previous one t-

1, where the constant is approximately equal 1. Once such a criterion is satisfied, the iterating computation for the algorithm of the localization is stopped.

4. EXPERIMENTAL RESULTS

In order to analyze the effectiveness, robustness, and limitations of the proposed local-regions based level set for segmenting moving objects in video sequences, we use four complex sequences of urban traffic.

4.1. Contour Initialization

One of the major issues of active contour methods is their sensitivity to the initial contour placement that is provided by the user. To deal with this problem, the proposed model uses a fast way for initialization based on the Mixture of Gaussians method, which is adaptive, and can handle multimodal backgrounds.

4.2. Evaluation

In order to measure a quantitative evaluation of the performance, we have selected at regular intervals from each test sequence ten challenging frames, which summon almost all the known problems of the detection, and manually highlighted all the moving objects in them for constructing ground-truth frames. In the manual annotation, we highlight only the pixels belonging to vehicles and pedestrians that are actually moving at that frame. To quantify how well each method matches the ground-truth, we use three performance measures [19, 43]: sensitivity, precision, and overlap ration, which they are calculated as follows:

Pr (16)TPecision

TP FP=

+ (17)TP

SensitivityTP FN

=+

(18)TPOverlap ratio

TP FP FN=

+ +

Table.1 recapitulates the obtained results of the global mean separation model presented in Sec.2.2 with its proposed local counterpart. When applied to the entire sequence, the sensitivity, precision rate, and overlap ratio are computed as the average values for all the measured frames, which they intuitively convey the quality of the results. Typically, there is a trade-off between sensitivity and precision, sensitivity commonly increases with the number of foreground pixels detected, which in turn may lead to a decrease in precision. Namely, a good localization model should reach as a high sensitivity, an overlap value as possible without losing the precision. As it can be depicted from Table.1, the quantitative results of each sequence indicate that our local model achieves the superior performances comparing with the global model and the morphological operations, where it produces the average values of 85.76 %, 73.25 %, and 81.52 % for sensitivity, overlap, and precision, respectively, while the average values of the global one are only 55.37 %, 39.29 %, and 48.79 % for sensitivity, overlap, and precision, respectively, and the average values of the morphological operation are 65.88 %, 53.21%, and 75.14% for sensitivity, overlap, and precision, respectively. On the other hand, the qualitative results also demonstrate the benefit of the local model.

From Fig.3, it can be seen that the global model is very sensitive to the illumination variation and camera motion. It labels the non-significant regions as true foregrounds, as illustrated in Fig.3 (b), while the local counterpart

194

performs well and can segment more accurately the object, as seen in Fig.3 (d).

Table 1. Quantitative evaluation of the global model with its local counterpart

5. CONCLUSION

In this paper, we proposed a novel local region-based active contour model in a variational level set formulation for segmenting moving objects. In fact, we reformulated the global mean separation energy of Yezzi in a local way, by using squared windows of length L, centered at each point in the narrow band. This allowed us to represent the inside and the outside of the detected foregrounds by smaller local regions that are characterized by local mean intensities values. In addition, the regularity of the level set function is intrinsically preserved by the proposed regularization term, which depends on an automatic stopping criterion. A quantitative analysis was performed to compare the performances of the local and global models on one hand. On the other hand, a comparison between the local models and morphological operations was also effectuated. The obtained results demonstrated that the proposed local model achieves the superior average values of 85.76 %, 73.25 %, and 81.52 % for sensitivity, overlap, and precision, respectively. We also showed some demonstrative examples where global region-based energies failed while the localized versions gave very reasonable results.

Seq. performance measures (%)

Global model

Local model

Morphol- ogical Operation

inter Precision 38.72 81.03 72.41

Sensitivity 41.21 78.82 63.10 Overlap 29.12 72.46 54.21

dtneu_wint

Precision 42.49 86.13 72.91 Sensitivity 56.82 96.17 68.51

Overlap 32.12 80.13 52.10

dt_passat

Precision 38.12 74.81 77.00 Sensitivity 40.12 95.72 67.71

Overlap 29.83 72.39 56.32

Pet_2001

Precision 22.69 84.09 78.24 Sensitivity 45.92 72.31 64.21

Overlap 17.91 68.02 50.21

(a) Original image (b) Global model (c) Local model

Figure 3.Comparison between the global mean separation model and its local counterpart model.

195

6. REFERENCES

[1] Li, C., Kao, C., Gore, J. and Ding, Z.: ‘Implicit active contours driven by local binary fitting energy’, IEEE Proc., Computer Vision and Pattern Recognition, 2007, pp. 1-7.

[2] Wang, L., Li, C., Sun, Q., Xia, D., and Kao, C. ‘Active contours driven by local and global intensity

tting energy with application to brain MR image segmentation’, Elsevier, Computerized Medical Imaging and Graphics, 2009, (7),33, pp.520–531

[3] Zhang, K., Song, H., and Zhang, L. : ‘Active contours driven by local image tting energy’, Elsevier, Pattern Recognition, 2010, (4),43, pp.1199–1206

[4] H, C., Wang,Y., and Chen, Q.,: ‘Active contours driven by weighted region-scalable tting energy based on local entropy’, Elsevier, Signal Processing, 2011, 92, (2), pp.587–600

[5] Tao,W., Tai, X-C.,: ‘Multiple piecewise constant with geodesic active contours (MPC-GAC) framework for interactive image segmentation using graph cut optimization’, Elsevier, Image and Vision Computing, 2011,29, pp.499–508

[6] Chan, T., and Vese, L.: ‘Active contours without edges’, IEEE, Transaction on Image Processing, 2001, 10, (2), pp. 266–277.

[7] Yezzi, J. A., Tsai, A., and Willsky, A.: ‘A fully global approach to image segmentation via coupled curve evolution equations’, Elsevier, Journal of Visual Communication and Image Representation, 2002, 13, (1), pp. 195–216.

[8] Vese, L., and Chan, T.: ‘A multiphase level set framework for image segmentation using the Mumford and Shah model’, Springer, International Journal of Computer Vision, 2002, 50, (3), pp.271–293.

[9] Lankton, S., Nainb, D., Yezzi, A., and Tannenbaum, A.: ‘Hybrid geodesic region-based curve evolutions for image segmentation’, Proceedings of the SPIE Medical Imaging Symposium, 2007, 6510, (3), pp. 6510-6519.

[10] Lankton, S., and Tannenbaum, A.: ‘Localizing region-based active contours’, IEEE, Transactions on Image Processing, 2008, 17, (11), pp. 2029-2039.

[11] Rousson, M., Lenglet, C., and Deriche, R.: ‘Level set and region based propagation for diffusion tensor MRI segmentation’, SpringerLink, Computer Vision and Mathematical Methods in Medical and Biomedical Image Analysis, 2004,3117, (2004), pp.123–134.

[12] Stauffer, C., and Grimson W.: ‘Adaptive background mixture models for real time tracking’. IEEE Proc. Computer Vision and Pattern Recognition, 1999, pp. 246-252.

[13] Ardila, J., Bijker, W., Tolpekin, A., and Stein, A.: ‘Multitemporal change detection of urban trees using localized region-based active contours in VHR images’, Elsevier, Remote Sensing of Environment, 2012, 124, pp. 413–426.

[14] Zhang, K., Zhang, L. , Song, H., and Zhou, W.: ‘Active contours with selective local or global segmentation: A new formulation and level set method’, Elsevier, Image and Vision Computing, 2010, 28, (4), pp. 668–676.

[15] Dzyubachyk, O., Niessen, W., and Meijering, E.: ‘Advanced level-set based multiple-cell segmentation and tracking in time-lapse fluorescence microscopy

images’, IEEE Proc., Biomedical Imaging: From Nano to Macro, 2008, pp. 185-188.

[16] Kass, M., Witkin, A., and Terzopoulos, D.:’ Snakes: Active contour models’, Springer, International Journal of Computer Vision, 1998, 1, (4), pp. 321-331.

[17] Caselles, V., Kimmel, R., and Sapiro, G.:’ Geodesic Active Contours’, Springer, International Journal of Computer Vision, 1997, 22, (1), pp. 61-79.

[18] Osher, S., and. Sethian, J.A.:’ Fronts Propagating with Curvature Dependent Speed Algorithms Based on Hamilton-Jacobi Formulations’, Elsevier, Journal of Computational Physics, 1988, 79, (1), pp.12-49.

[19] Zhu, G.P., Zhang, Sh.Q., Zeng, Q.SH., and Wang, Ch.H.: ‘Boundary-based image segmentation using binary level set method’, SPIE, Optical Engineering, 2007 ,46, (5).

[20] Lie, J., Lysaker, M., and Tai, X.C.: ‘A binary level set model and some application to Mumford–Shah image segmentation’, IEEE, Transaction on Image Processing, 2006, 15, (5), pp.1171–1181.

[21] Mumford, D., and Shah, J.: ’Boundary detection by minimizing functional’. IEEE Proc., Computer Vision and Pattern Recognition, 1985, pp. 22-26.

[22] Li, C.M., Xu, C.Y., Gui, C.F., and Fox, M.D.: ‘Level set evolution without re-initialization: a new variational formulation’, IEEE Proc., Computer Vision and Pattern Recognition, 2005, pp. 430–436.

[23] Brox, T., and Cremers, D.: ‘On the Statistical Interpretation of the Piecewise Smooth Mumford-Shah Functional’, Springer, Scale Space and Variational Methods in Computer Vision, 2007, 4485, (2007), pp. 203-213.

[24] Paragios, N., and Deriche, R.: ‘Geodesic active contours and level sets for detection and tracking of moving objects’, IEEE ,Transaction on Pattern Analysis and Machine Intelligence, 2000, 22, (4), pp.1-15.

[25] Tsai, A., Yezzi, A., Willsky, A.S.: ‘Curve evolution implementation of the Mumford–Shah functional for image segmentation, denoising, interpolation, and magni cation’, IEEE, Transaction on Image Processing, 2001, 10, (8) pp.1169-1186.

[26] Vese, L.A., and Chan, T.F.: ‘A multiphase level set framework for image segmentation using the Mumford–Shah model’, Springer, International Journal of Computer Vision, 2002, 50, pp. 271–293.

[27] Ronfard, R.: ‘Region-based strategies for active contour models’, Springer, International Journal of Computer Vision, 1994, 13, (2), pp. 229-251.

[28] Sussman, M., Smereka, P., and Osher, S.:’A level set approach for computing solutions to incompressible two phase flow’, Journal of Computational Physics, 1994, 114, (1), pp.146-159

[29] Rosenthal, P., Molchanov, V., and Linsen, L.: ‘A Narrow Band Level Set Method for Surface Extraction from Unstructured Point-based Volume Data’, WSCG Proc., Computer Graphics, Visualization and Computer Vision, 2010, pp. 73–80.

196