a robust dynamic programming algorithm for motion detection and estimation

8
ISSN 1054-6618, Pattern Recognition and Image Analysis, 2007, Vol. 17, No. 2, pp. 175–182. © Pleiades Publishing, Ltd., 2007. A Robust Dynamic Programming Algorithm for Motion Detection and Estimation 1 M. Mozerov a, c , V. Kober b , and I. A. Ovseevich a a Laboratory of Digital Optics, Institute for Information Transmission Problems, Russian Academy of Sciences, Bol’shoi Karetnyi per. 19, Moscow, 101447 Russia e-mail: [email protected] b Department of Computer Sciences, CICESE, Ensenada 22860, B.C., Mexico e-mail: [email protected] c Computer Vision Center and Departament d’Informatica, Universitat Autonoma de Barselona (UAB), 08193 Cerdanyola, Spain e-mail: [email protected] Abstract—A new robust algorithm for motion detection and precise evaluation of the motion vectors of mov- ing objects in a sequence of images is presented. It is well known that the accuracy of estimating motion vectors estimation is limited by smoothness constraints and mutual occlusions of motion segments. The proposed method is a fusion of block-matching motion estimation and global optimization technique. It is robust to motion discontinuity and moving objects occlusions. To avoid some contradictions between global optimiza- tion techniques and piece-wise smooth values of sought motion vectors, a hidden segmentation model is uti- lized. Computer simulation and experimental results demonstrate an excellent performance of the method in terms of dynamic motion analysis. Key words: motion analysis, motion estimation, dynamic programming. DOI: 10.1134/S1054661807020010 Received March 17, 2005 1 1. INTRODUCTION Extraction of motion information is an essential part of any video processing system. Relative depth from motion, 3-D shape recovery, autonomous vehicle or robot navigation, and moving object detection usually involve various motion analysis techniques. Many such techniques, which analyze the motion on the basis of optical flow computation, have been proposed in the past two decades [1–7]; however, reliable optical flow estimation remains difficult when a smoothness con- straint is violated. The smoothness discontinuity prob- lem, as well as the occlusions problem, is closely related to that of scene segmentation. Understanding of this fact has led to development of methods that alter- nate computation of optical flow and segmentation [8– 10]. Such algorithms can be treated as variations of the expectation-maximization (EM) algorithm [11]. Among a wide variety of approaches, there exist three main catego- ries of motion estimation methods, namely, gradient- based methods [1], frequency-based approaches [2], and matching techniques [3]. In this paper, we remain in the framework of the matching concept, and a block- matching (BM) algorithm is one of the most widely used among these approaches [12–14]. 1 This article was translated by the authors. It is well known that the correspondence problem is inherently ambiguous, and some additional information must be added to solve it. The correspondence problem is also a necessary step in object segmentation and motion compensation used by MPEG-4. Various approaches have been suggested for solving the corre- spondence problem [15, 16]. The identification of a correspondence based on correlation-based methods between the same points in consecutive images is often formulated as a local (area-based) optimization prob- lem or shortest -path technique [17, 18]. The local opti- mization involves a few pixels around the correspond- ing points in different images. Many algorithms based on dynamic programming perform a 1-D optimization along an axis of the evolving 3-D volume of data to obtain a good data matching [19]. Correspondence between the same points in neigh- boring images can be considered as a global optimiza- tion problem [20]; the matching is carried out between 2-D arrays of images. A drawback of the method is owing to contradictions between a global smoothness constraint and a real signal discontinuity at borders of object segments. In this paper, the global optimization is reformu- lated as a segment optimization problem. Under the assumption that, initially, the map of segments is unknown, a hidden segmentation model is proposed. We suggest a new method based on dynamic program- ming with a local estimation of similarity like in block IMAGE PROCESSING, ANALYSIS, RECOGNITION, AND UNDERSTANDING

Upload: m-mozerov

Post on 02-Aug-2016

215 views

Category:

Documents


0 download

TRANSCRIPT

ISSN 1054-6618, Pattern Recognition and Image Analysis, 2007, Vol. 17, No. 2, pp. 175–182. © Pleiades Publishing, Ltd., 2007.

A Robust Dynamic Programming Algorithmfor Motion Detection and Estimation

1

M. Mozerov

a

,

c

, V. Kober

b

, and I. A. Ovseevich

a

a

Laboratory of Digital Optics, Institute for Information Transmission Problems, Russian Academy of Sciences, Bol’shoi Karetnyi per. 19, Moscow, 101447 Russia

e-mail: [email protected]

b

Department of Computer Sciences, CICESE, Ensenada 22860, B.C., Mexicoe-mail: [email protected]

c

Computer Vision Center and Departament d’Informatica, Universitat Autonoma de Barselona (UAB), 08193 Cerdanyola, Spain

e-mail: [email protected]

Abstract

—A new robust algorithm for motion detection and precise evaluation of the motion vectors of mov-ing objects in a sequence of images is presented. It is well known that the accuracy of estimating motion vectorsestimation is limited by smoothness constraints and mutual occlusions of motion segments. The proposedmethod is a fusion of block-matching motion estimation and global optimization technique. It is robust tomotion discontinuity and moving objects occlusions. To avoid some contradictions between global optimiza-tion techniques and piece-wise smooth values of sought motion vectors, a hidden segmentation model is uti-lized. Computer simulation and experimental results demonstrate an excellent performance of the method interms of dynamic motion analysis.

Key words

: motion analysis, motion estimation, dynamic programming.

DOI:

10.1134/S1054661807020010

Received March 17, 2005

1

1. INTRODUCTION

Extraction of motion information is an essential partof any video processing system. Relative depth frommotion, 3-

D

shape recovery, autonomous vehicle orrobot navigation, and moving object detection usuallyinvolve various motion analysis techniques. Many suchtechniques, which analyze the motion on the basis ofoptical flow computation, have been proposed in thepast two decades [1–7]; however, reliable optical flowestimation remains difficult when a smoothness con-straint is violated. The smoothness discontinuity prob-lem, as well as the occlusions problem, is closelyrelated to that of scene segmentation. Understanding ofthis fact has led to development of methods that alter-nate computation of optical flow and segmentation [8–10]. Such algorithms can be treated as variations of theexpectation-maximization (EM) algorithm [11]. Among awide variety of approaches, there exist three main catego-ries of motion estimation methods, namely, gradient-based methods [1], frequency-based approaches [2], andmatching techniques [3]. In this paper, we remain in theframework of the matching concept, and a block-matching (BM) algorithm is one of the most widelyused among these approaches [12–14].

1

This article was translated by the authors.

It is well known that the correspondence problem isinherently ambiguous, and some additional informationmust be added to solve it. The correspondence problemis also a necessary step in object segmentation andmotion compensation used by MPEG-4. Variousapproaches have been suggested for solving the corre-spondence problem [15, 16]. The identification of acorrespondence based on correlation-based methodsbetween the same points in consecutive images is oftenformulated as a local (area-based) optimization prob-lem or shortest -path technique [17, 18]. The local opti-mization involves a few pixels around the correspond-ing points in different images. Many algorithms basedon dynamic programming perform a 1-

D

optimizationalong an axis of the evolving 3-

D

volume of data toobtain a good data matching [19].

Correspondence between the same points in neigh-boring images can be considered as a global optimiza-tion problem [20]; the matching is carried out between2-

D

arrays of images. A drawback of the method isowing to contradictions between a global smoothnessconstraint and a real signal discontinuity at borders ofobject segments.

In this paper, the global optimization is reformu-lated as a segment optimization problem. Under theassumption that, initially, the map of segments isunknown, a hidden segmentation model is proposed.We suggest a new method based on dynamic program-ming with a local estimation of similarity like in block

IMAGE PROCESSING, ANALYSIS, RECOGNITION, AND UNDERSTANDING

176

PATTERN RECOGNITION AND IMAGE ANALYSIS

Vol. 17

No. 2

2007

MOZEROV et al.

matching; however, neither shape nor size of blocks arefixed. First, dynamic images are decomposed onto apoint to point similarity matrix. Then, we form atrimmed matrix for the matching process excludingsome elements from the initial matrix. Note that thisoperation reminds one of trimmed summation that isused in robust statistical procedures. Next, the dynamicprogramming operator determines connected area clus-ters with positive similarity values in such a way that aloss function is optimized and motion segments arerevealed at the same time. Computer simulation withvarious image sequences shows a good performance ofthe method in terms of motion vector accuracy.

This paper is organized as follows. In Section 2, aglobal optimization technique is described. Then, inSection 3, a hidden segmentation model is explained. InSection 4, we propose a new motion estimation algo-rithm. In Section 5, computer simulation results arepresented and discussed. The conclusion is in Sec-tion 6.

2. OPTICAL FLOW CONSTRAINT AND GLOBAL OPTIMIZATION TECHNIQUE

In dynamic image analysis, one of the first points toclarify and to formalize is the link between the intensityvariations of an image and the motion. A general andwidely used assumption is that the intensity of a pointremains constant along its trajectory. More precisely,let

f

(

x

,

y

,

t

) denote the intensity of the pixel (

x

,

y

) at time

t

. Starting from a point (

x

0

,

y

0

) at the time

t

0

, we deter-mine the trajectory (

x

0

+

u

x

δ

t

,

y

0

+

u

y

δ

t

,

t

0

+

δ

t

) of thispoint in time as follows:

(1)

where

u

= (

u

x

(

x

0

,

y

0

,

t

0

),

u

y

(

x

0

,

y

0

,

t

0

)) is the velocity vec-tor (also called flow vector) of a point (

x

0

,

y

0

) at the time

t

0

and

δ

t

is a so- called interframe interval.

In motion analysis, there are no general methods orfoolproof techniques and most algorithms work only ifcertain conditions are met. For instance, the gradient-based methods [1] suppose that the motion vector(

δ

r

x

=

u

x

δ

t

,

δ

r

y

=

u

y

δ

t

) and interframe interval

δ

t

are

f x0 y0 t0, ,( ) f x0 uxδt+ y0 uyδt+ t0 δt+, ,( ),=

small and Taylor’s expansion is applicable to (1) in thefollowing way:

(2)

using (1), we finally have

(3)

One scalar equation (3) is not sufficient for comput-ing both components of the velocity field. Several ideashave been proposed to overcome this difficulty; how-ever, equality (3) is supposed to hold for all proposedalgorithms and this significantly restricts the applicabil-ity of the gradient-based methods, making them theo-retical rather than practical.

Consider a simple example to clarify the restrictionsin applicability of the gradient-based methods.

The background in Fig. 1 represents a continuousoptical flow with a foreground sampling grid. For sim-plicity, we assume that

u

y

= 0; in other words, only ahorizontal motion is presented. Dynamic image analy-sis deals with sampled motion; and i.e., it is assumedthat consecutive frames of the dynamic image are sep-arated by a time interval

δ

t

and, in turn, in the spacedomain, a sampling interval

δ

x

must be chosen. Thus,for our example, Eq. (3) can be rewritten as

(4)

Here we can see that, for the chosen sampling inFig. 1, expression (4) does not hold (because

f

(

x

,

t

+

δ

t

) –

f

(

x

,

t

) = 0, but

u

0) and the time interval

δ

t

mustbe changed to

δ

t

'. So, our intuitive guess is that themotion vector

u

δ

t

' cannot overpass the sampling inter-val

δ

x

if we want to use (3) for motion estimation.Obviously, this assumption agrees with the linear signalmodel, which supposes that the expression (

f

(

x

+

δ

x

) –

f

(x))/δx ≈ (f(x + δx') – f(x))/δx' holds in sampling inter-val δx for all δx' ≤ δx and, in this way, motion vectorδr = uδt in Taylor’s expansion (2) must be less than orequal to the sampling interval of the processed image.Such strong restriction has obvious drawbacks, and themain one is that the time interval in most cases has afixed value and cannot be changed arbitrarily to keepthe motion vector in one pixel span. On the other hand,one pixel shift is not a motion in the framework ofimage processing and, even in the case where we cancontrol the interframe interval, many consecutiveimages should be involved to resolve the motion esti-mation problem with a cumulative error effect that isproportional to the number of frames used.

f x δrx+ y δry+ t δt+, ,( )

= f x y t, ,( ) ∂f x y t, ,( )∂x

------------------------δrx∂f x y t, ,( )

∂y------------------------δry+ +

+∂f x y t, ,( )

∂t------------------------δt,

∂f x y t, ,( )∂x

------------------------ux∂f x y t, ,( )

∂y------------------------uy

∂f x y t, ,( )∂t

------------------------+ + 0.=

f x δx+ t,( ) f x t,( )–δx

--------------------------------------------------uf x t, δt+( ) f x t,( )–

δt-------------------------------------------------+ = 0.

t1

t0x0 δx x1

δtδt’

uδt’

uδt

Fig. 1. Illustration of the optical flow constraint.

PATTERN RECOGNITION AND IMAGE ANALYSIS Vol. 17 No. 2 2007

A ROBUST DYNAMIC PROGRAMMING ALGORITHM 177

In this work, we propose an approach that is basedon matching techniques. The main step of such meth-ods usually involves only two consecutive images to becompared and the time variable of the optical flowfunction can be disregarded. Matching techniques donot strictly confine the span of motion vectors, span andoptical flow equation (1) and they can be rewritten as

(5)

where a(x, y) is an intensity function of the anchorframe (a snapshot at t0), g(x, y) is an intensity functionof the target frame (a snapshot at t0 + δt), and r(x, y) =(rx(x, y), ry(x, y)) is a sought motion vector.

A pair of images shown in Fig. 2 is an example thatcan be considered as a matching problem, but the inten-sity functions of these two images do not satisfy theoptical flow constraint of gradient-based methods.

The exact equality of the left and right sides of (5) isan ideal model. For practical application, a more conve-nient representation is given by

(6)

where δE is a mismatch error; thus, the matching prob-lem can be formulated as the global optimization prob-lem

(7)

where the symbol inf denotes the minimization of theintegral functional over the sought motion vector fieldr(x, y), Ω is the image frame domain, and n is 1 or 2depending on the measure chosen.

We are going to use the dissimilarity functionδE(x, y, rx, ry) as a local feature of correspondencematching or a local error function to compute the opti-cal flow. In the sampled space, the dissimilarity func-tion values can be described by a 4-D array Ei, j, k, l;i = 0, …, I – 1; j = 0, …, J – 1; k = –K, –K + 1, …, K;l = –L, –L + 1, …, L. Here, I and J are the sizes of

a x y,( ) g x rx x y,( )+ y ry x y,( )+,( ),=

0 δE≈ g x rx x y,( )+ y ry x y,( )+,( ) a x y,( )– n,=

g x rx x y,( )+ y ry x y,( )+,( ) a x y,( )– n x y,dd

Ω∫r x y,( )

inf

images, while K, and L are reasonable values to carryout the correspondence matching. So, in the case of adiscrete image sequence, the dissimilarity array valuesare computed as

(8)

where a(i, j) and g(i, j) are the sampled intensity func-tions of the anchor and target frames, respectively. Wesuppose that motion vectors have subpixel values andthe vectors’s sampling interval 0 < δr ≤ 1 in (8) charac-terizes subpixel accuracy of the vector estimation; n is1 or 2 depending on the measure chosen.

Optimization expression (7) assumes additionalconstraints to solve the problem. We must at least sup-pose that the motion vectors possess the property ofsmall signal variations. So, the absolute differencesbetween any adjacent elements of the motion vectorfield are assumed to be bounded by values δv:

(9)

Now, for the sampled signal, the global optimizationproblem is formulated as follows: find the motion vec-tor field ri, j ≡ ki, j , li, j with the smoothness con-straint (9) in such a way that the sum of the dissimilarityfunction (8) taken over all elements of images is mini-mal, i.e.,

(10)

where |∆ri, j | ≤ δv denotes a shortcut of the smoothnessconstraint (2).

Figure 3a represents a horizontal slice of the dissim-ilarity function of the synthetic dynamic image shownin Fig. 2. The visible sinusoid composed of black pixels(or zero values of the dissimilarity function) is the goalof the matching process. For simplicity, only a horizon-tal shift is presented in the generated pair in Fig. 2, andthe l index in motion vector r =k, l is always equal tozero.

Ei j k l, , , gi kδr+ j lδr+, ai j,– n.=

∆ri j, ki j, ki 1± j 1±,– li j, li 1± j 1±,–,( ) δv .≤≡

ri j,( ) Ei j r, ,

j 0=

J 1–

∑i 0=

I 1–

∑⎝ ⎠⎜ ⎟⎛ ⎞

,ri j,( ) ∆ri j,, δv≤

argmin=

(‡) (b)

Fig. 2. An anchor (a) and a target (b) frames of a synthetic dynamic image.

178

PATTERN RECOGNITION AND IMAGE ANALYSIS Vol. 17 No. 2 2007

MOZEROV et al.

K

–K

i

iK

–K

(a)

(b)

Fig. 3. (a) Dissimilarity function values of the synthetic dynamic image shown in Fig. 2; (b) transformed dissimilarity function val-ues after OI recurrence transformation.

The method in [20] optimizes (10) by means of twodirect and inverse consecutive recurrences,

(11)

Finally, the solution can be found by

(12)

OI Ei j r, ,( ) Ei j r, ,I Si j r, ,

I+ Si j r, ,I– Ei j r, , ,–+= =

Si j r, ,I± Si k+ j r, ,

I± Ei j r, ,+( ),k δv≤min=

S0 j r, ,I+ E0 j r, , , SI 1– j r, ,

I– EI 1– j r, , ,= =

OJ Ei j r, ,I( ) Ei j r, ,

JI Si j r, ,J+ Si j r, ,

J– Ei j r, ,I ,–+= =

Si j r, ,J± Si j l r,+,

J± Ei j r, ,I+( ),

l δv≤min=

Si 0 r, ,J+ Ei 0 r, ,

I , Si J 1– r, ,J– Ei J 1– r, ,

I .= =

ri j,( ) OJ OI Ei j r, ,( )[ ]( ),ri j,

min=

where OI and OJ are two consecutive transformationsusing recurrence operator (11) along the I- and theJ-axis, respectively. The result of the OI transformationis represented in Fig. 3b. Note that, after one transfor-mation (OI), the solution is identically equal to the opti-mal path that can be calculated by conventionaldynamic programming; however, the global optimiza-tion (10) requires the quadratic measure (IxJ), and theconventional dynamic programming procedure pro-vides only with 1D optimization. Recurrence operator(11) transforms the initial point-to-point dissimilaritymatrix (8) to more reliable values of 1D measure (I). Inthis way, after the second transformation OJ, the neces-sary quadratic optimization is performed.

3. HIDDEN SEGMENTATION MODEL

Our approach to motion segmentation is based onthe assumption that a video scene can be decomposedinto independently moving visual objects and itsdynamic image can be similarly subdivided into a set ofcorresponding segments. The necessary condition for asuccessful matching process is that the motion vectorsinside each segment possess the property of small-value variations.

As we noted above, the global optimization tech-nique can produce a bad result owingdue to contradic-tions between a global smoothness condition and thereal discontinuities at borders of segments. The seg-ments I and III shown in Fig. 4 illustrate this situation.On the other hand, the set of occluded points (the II seg-ment in Fig. 4.) has no corresponding area, and thisuncertainty affects on the accuracy of the algorithm.

To overcome the discontinuity problem, a globaloptimization (10) should be recast into the segmentoptimization as follow:

(13)

where Ωs is a definition domain for indices i, j of eachmotion segment. As we mentioned above, the segments

ri j, Ωs∈( ) Ei j r, ,

i j Ωs∈,∑⎝ ⎠

⎜ ⎟⎛ ⎞

,ri j,( ) Dri j,, δv≤

argmin=

j0

V1

V2

I II III

Img2

Img1

Fig. 4. An example of the motion segments.

PATTERN RECOGNITION AND IMAGE ANALYSIS Vol. 17 No. 2 2007

A ROBUST DYNAMIC PROGRAMMING ALGORITHM 179

are initially undefined; however, a slight modificationallows us to use the global optimization technique tosolve the problem.

Let us consider the graphical representation shownin Fig. 5 of the global optimization method in order tounderstand why this estimation can produce incorrectresults and how to use this technique to reveal hiddensegments. For simplicity, we assume that motion vec-tors vary along the I-axis. In other words, matchedpoints lie in the same row j0. This is a typical situationfor stereo pair images.

Now we can compare the true motion vectors ofri(j0) (dashed white line in Fig. 5) with the onesobtained by the global optimization method (solid darkline).

It is easy to see that the true motion vectors do notsatisfy the smoothness condition (9). In contrast, theobtained optimal path is smooth. However, suchsmoothness leads to an alteration of the sought motionvectors.

To overcome this shortcoming, a method shouldoptimize the loss function and reveal segments at thesame time. Figure 5 gives us an idea how to do this.Indeed, the true motion vectors are a set of “black” ele-ments of the matrix of point- to- point dissimilarityfunctions or all values with Ei, j, r < ETrh. In this way, weare able to reduce the number of potential motion vec-tor candidates. ThisIt is illustrated in Fig. 6.

These potential candidates form clusters withrequired smoothness conditions (9) and the size of eachmaximal (argument r) cluster is approximately equal tothe size of the corresponding motion segment (Fig. 6).The modification of recurrence operator (11) gives the

mechanism of such clusterization. Indeed, if we replacedissimilarity function Ei, j, r by the similarity measure

(14)

and then apply twice (along I and J axis) to this matrixthe following modification of recurrence operator (11):

(15)

then we obtain a matrix such that each entry ofthis matrix is the number of elements in a cluster Ωs

(Li, j, r ∈ Ωs). On the other hand, reformulating the seg-ment optimization (13) for the similarity matrix in theform

(16)

we also must obtain the number of elements in the clus-ter Ωs as the maximum value. Thus, the use of recur-rence operator (15) can formally solve the segmentoptimization problem (16). However, such a solution isnot unique (see, e.g., the largest cluster of Segment IIIin Fig. 6) and can be not optimal in terms of segmentoptimality (13), especially, if the threshold ETrh is cho-sen sufficiently large.

4. MOTION ESTIMATION ALGORITHM

To resolve the contradiction between the loss func-tion optimization and the cluster size optimization, we

Li j r, ,1 if Ei j r, , EThr,≤0 otherwise⎩

⎨⎧

=

OI Li j r, ,( ) Li j r, ,I Si j r, ,

I+ Si j r, ,I– Ii j r, , ,–+= =

Si j r, ,I± Si k+ j r, ,

I± Li j r, ,+( ) if Li j r, ,k δv≤max 0,>

0 otherwise⎩⎨⎧

=

S0 j r, ,I+ L0 j r, , , SI 1– j r, ,

I– LI 1– j r, , ,= =

Li j r, ,IJ

ri j, Ωs∈( ) Li j r, ,

i j, Ωs∈∑⎝ ⎠

⎜ ⎟⎛ ⎞

,ri j,( ) Dri j,, δv≤

argmax=

j0

+K

0

–K

I II III I

g(i)

a(i)

Fig. 5. The graphical representation of the global optimization method.

180

PATTERN RECOGNITION AND IMAGE ANALYSIS Vol. 17 No. 2 2007

MOZEROV et al.

propose a palliative similarity measure whose valuesare unity, zero, and a number very close to unity,

(17)

where Emax is the maximum of the dissimilarity func-tion, which usually coincides with the signal maxi-mum, and LThr is a threshold that controls the outputquality of the motion vectors estimation. In this work,we suggest taking the value of LThr between 1 and 0.95,because, in this case, the global peak signal-to-noiseratio (PSNR) in (16) is approximately 30–50 dB andthat span corresponds to excellent quality of imagereconstruction. Now, the final solution can be written as

(18)

where PThr is a threshold value that defines the minimalsize of objects in the frame or the minimal number ofelement in a cluster, and the symbol # denotes an unde-fined motion vector.

If we consider a pure translation motion, the pre-sented technique can be reduced to the block-matchingalgorithm. However, the block subdivision in this case

Li j r, ,z

Emax Ei j r, ,–Emax

---------------------------- if z LThr,≥=

0 otherwise,⎩⎪⎨⎪⎧

=

ri j,( )z OJ OI Li j r, ,( )[ ]( ) if z

ri j,

max Pthr,>=

# if z Pthr,≤⎩⎨⎧

=

must coincide with that of the real motion segmenta-tion, and the smoothness threshold in (9) must be zero(δv = 0).

Thus, the proposed algorithm consists of the follow-ing steps.

• Form the initial 4D matrix Li, j, r of the similarityfunction using (17).

• Perform two consecutive transformations with theuse of the recurrence operator (15) along the I- andJ-axes, respectively.

• Extract motion information with the help of (18).

5. EXPERIMENTAL RESULTS

In this section, experimental results are presented toillustrate the performance of the proposed algorithm.We are also interested in answering how well the pro-posed algorithm can fit the edges of motion segments.A computer-simulated pair of images with a knownmotion vector field is presented in Fig. 7b.

The resulting motion vectors are presented inFig. 7a. They illustrate an excellent performance of theproposed technique. Here, black areas representunmatched pixels. The method proposed in our paper isdeveloped to recognize both translation and more com-plicated affine kinds of motion. The pure translationmotion assumes the zero smoothness condition; inother words, values of the motion vector can not varywithin a motion segment. In this case, we must set thealgorithm parameter δv = 0. In our experimental sec-tion, the object motion consists only of translations in

Fig. 6. Graphical representation of the hidden segmentation model.

+K

–K

+K

–K

0

0

III III

PATTERN RECOGNITION AND IMAGE ANALYSIS Vol. 17 No. 2 2007

A ROBUST DYNAMIC PROGRAMMING ALGORITHM 181

order to emphasize the capability of the method to findmotion segments.

6. CONCLUSIONS

A new motion estimation method based on dynamicprogramming matching and a hidden segmentationmodel has beenis proposed. The method demonstratesmuch better results than those of the global optimiza-tion method, especially, in areas, which are near bor-ders of objects. Finally, note that the computationalcomplexity of correspondence matching can bereduced using a pyramidal subdivision of the initialimage frame.

REFERENCES

1. B. K. P. Horn and B. G. Schunck, “Determining OpticalFlow,” Artificial Intelligence 17, 185–204 (1981).

2. A. B. Watson and A. J. Ahumada, in Motion: Perceptionand Representation, Ed. by J. K. Tsotsos (1983).

3. P. Anandan, Measuring Visual Motion from ImageSequences (PhD Thesis, Univ. of Massachusetts,Amherst, 1987).

4. J. K. Kearney, W. B. Tompson, and D. L. Boley, “OpticalFlow Estimation—An Error Analysis of Gradient BasedMethods with Local Optimization,” IEEE Trans. on Pat-tern Analysis and Machine Intelligence 9, 229–244(1987).

5. F. Heitz and P. Bouthemy, “Multimodal Estimation ofDiscontinuous Optical Flow Using Markov RandomFields,” IEEE Trans. on Pattern Analysis and MachineIntelligence 15, 1213–1232 (1993).

6. C. Cedaras and M. Shah, “Motion Based Recognition: ASurvey,” Image and Vision Computing 13, 129–154(1995).

7. A. M. Tekalp, Digital Video Processing (Prentice HallPTR, 1995).

8. D. Murray and B. Buxton, “Scene Segmentation fromVisual Motion Using Global Optimization,” IEEE Trans.Pattern Analysis and Machine Intelligence 9, 321–335(1987).

9. J. Wang and E. Adelson, “Representing Moving Imageswith Layers,” IEEE Trans. Image Processing 3, 1103–1115 (1994).

10. N. Vasconcelos and A. Lippman, “Empirical BayesianMotion Segmentation,” IEEE Trans. Pattern Analysisand Machine Intelligence 23, 217–221 (2001).

11. A. Dempster, N. Laird, and D. Rubin, “Maximum-Like-lihood from Incomplete Data via the EM Algorithm,”J. Royal Statistical Soc. B 39, 271–284 (1977).

12. H. G. Musmann, P. Pirsch, and H. J. Grallert, “Advancesin Picture Coding,” Proc. IEEE 73, 523—548 (1985).

13. F. Dufaux and F. Moscheni, “Motion Estimation Tech-niques for Digital TV: A Review and a New Contribu-tion,” Proc. IEEE 83, 858—876 (1995).

14. K. R. Rao and J. J. Hwang, Techniques and Standards forImage, Video, and Audio Coding (Prentice-Hall, Engle-wood Cliffs, NJ, 1996).

15. H. K. Nishihara, “Practical Real-Time Imaging StereoMatcher,” Optical Engineering 23, 536–545 (1984).

16. Y. K. H. Anthony and T. C. Pong, “Cooperative Fusionof Stereo and Motion,” Pattern Recognition 28, 553–562(1995).

17. S. A. Lloyd, “A Dynamic Programming Algorithm forBinocular Stereo Vision,” GEC J. of Research 3, 18–24(1985).

18. H. Y. Chung, N. H. C. Yung, and P. Y. S. Cheung, “FastMotion Estimation with Search-Center Prediction,”Optical Engineering 40, 952–963 (2001).

19. C. Sun, “Fast Optical Flow Using Cross Correlation andShortest-Path Techniques,” in Proceedings of DigitalImage Computing: Techniques and Applications, Perth,Australia, 1999, pp. 143–148.

20. M. Mozerov, V. Kober, A. Tchernykh, and T. S. Choi,“Motion Estimation with a Modified Dynamic Program-ming,” Optical Engineering 41, 2592–2598 (2002).

(‡)

(b)

Fig. 7. (a) The motion vectors of a pair of test images pairshown in (b).

Mikhail Mozerov received hisMS degree in Physics from the Mos-cow State University in 1982 and hisPhD degree in Image Processingfrom the Institute of InformationTransmission Problems, RussianAcademy of Sciences, in 1995. Heworks at the Laboratory of DigitalOptics of the Institute of InformationTransmission Problems, RussianAcademy of Sciences. His researchinterests include signal and image

processing, pattern recognition, digital holography.

182

PATTERN RECOGNITION AND IMAGE ANALYSIS Vol. 17 No. 2 2007

MOZEROV et al.

Iosif A. Ovseyevich graduatedfrom the Moscow ElectrotechnicalInstitute of Telecommunications.Received candidate’s degree in 1953and doctoral degree in information the-ory in 1972. At present, he is EmeritusProfessor at the Institute of Informa-tion Transmission Problems of theRussian Academy of Sciences. Hisresearch interests include informationtheory, signal processing, and expertsystems. He is a Member of IEEE,

Popov Radio Society.

Vitaly Kober obtained his MSdegree in Applied Mathematics fromthe Air-Space University of Samara(Russia) in 1984, and his PhD degreein 1992 and Doctor of Sciencesdegree in 2004 in Image Processingfrom the Institute of InformationTransmission Problems, RussianAcademy of Sciences. Now he is a tit-ular researcher at the Centro de Inves-tigación Científica y de EducaciónSuperior de Ensenada (Cicese), Méx-

ico. His research interests include signal and image process-ing, pattern recognition.