Artif Life Robotics (2004) 7:182-184 �9 ISAROB 2004 DOI 10.1007/s10015-003-0272-6

Masanori Sugisaka �9 Shinobu Sato

Improvement of the precision of the gradient method and object tracking using optical flow

Received and accepted: August 5, 2003

Abstract In this study, the spatial local optimization method was improved to obtain high precision of optical flow for cases in which the object movement changes sub- stantially and a method to trace the loci of moving objects was considered. In the spatial local optimization method, the precision of the optical flow when the object movement changes substantially becomes a problem. Therefore, to make the object movement relatively small, we obtained flow vectors from the image sequence to drop the resolution of the original input image sequence to half the initial reso- lution. Flow vectors were then obtained from the original input image sequence that were smaller than the threshold value. We show that the precision of the optical flow when the object movement changes substantially is improved by this method. Method used to trace the loci of moving ob- jects was demonstrated. We obtained clusters from histo- grams of flow vectors and pursued each cluster. We show that it is possible to trace moving objects by this method.

Key words Optical flow �9 Flow vectors �9 Image sequence �9 Moving objects

1 Introduction

The ability to obtain optical flow is a theme of considerable interest in the field of image measurement, and its applica- tion is expected in various fields such as robot vision. 1-3 Typical methods used to obtain optical flow are the match- ing method and the gradient method. We chose the spatial

M. Sugisaka ([]) �9 S. Sato Department of Electric and Electronic Engineering, Faculty of Engineering, Oita University, 700 Dannoharu, Oita 870-1192, Japan Tel. +81-97-554-7831; Fax +81-97-554-7841 - e-mail: msugi@cc.oita-u.ac.j p

This work was presented, in part, at the 7th International Symposium on Artificial Life and Robotics, Oita, Japan, January 16-18, 2002

local optimization method, and have sought to improve the precision of this method for instances in which the object movement changes substantially.

We considered tracking moving objects using optical flow because we believe that it is possible to trace many objects independently. Therefore, to trace moving objects, we used a two-dimensional histogram of flow vectors, ex- tracted regions of objects, and traced loci of moving objects.

2 Optical flow detecting methods

Detecting optical flow using the gradient method is a way of determining a constraint equation based on the supposition that the distribution of brightness is kept constant between frames when moving objects in an image sequence are rigid bodies. A general constraint equation is shown in Eq. 1.

~pu ~pv bp ( b x by / (1)

Because a flow vector has two dimensions, in Eq. 1, it cannot fix the flow vector uniquely. Therefore, another con- straint equation is also usually used. Generally, the global optimization method and the local optimization methods are used with this equation.

The computational costs of the techniques were compared. Because the matching method and the global optimization method have high costs, we chose the local optimization method to obtain optical flow.

The local optimization method gives a constraint condi- tion that flow vectors v are constant in some local area S. Here, a general constraint equation, which is used in the local optimization method, is shown as Eq. 2.

8v 3v - - 0 ( x , y c S ) (2)

bx by

In this study, we obtained flow vectors from Eqs. 1 and 2. When object movement is substantial it becomes a problem when using the local optimization method. In this experi-

ment, we confirmed that the disorder of flow vectors be- comes large at movement above 4-5 pixels per frame. In such cases, we considered that the constraint conditions to obtain optical flow from a local area were not met. There- fore, we attempted to find large flow vectors above 4 pixels from the image sequence that dropped the resolution of the original input image sequence to half the resolution. Small flow vectors were then obtained using the original image sequence. When dropping the resolution of the original input image sequence to half resolution, we considered that the ability to see object movement was also halved. Smooth filtering of 9 • 9 was conducted beforehand to remove noise.

3 Object tracking using optical flow

The loci of moving objects were traced. Initially, flow vec- tors were obtained using the method described Sect. 2, and clusters from an image sequence were obtained using histo- grams of flow vectors. Because flow vectors that are in the region of a moving object similar values moving objects were examined by clustering.

Components of flow vectors less than 1 were removed as a preliminary treatment to reduce noise. A two-dimensional histogram of optical flow was obtained, as shown in Fig. la. Each region that had flow vectors with magnitudes of size difference less than 2 and differences in angle less than 45 degrees were clustered. As shown in Fig. lb, clusters were ranked in order of size so that moving objects could be traced. Therefore, loci of objects were traced by obtaining the correspondence of each cluster between frames.

183

Figure 3a shows the flow vectors obtained by a general local optimization method. Figure 3b shows the flow vectors obtained by the method using two kinds of image sequences as described in Sect. 2. The disorder of flow vectors in the part in which object movement changes substantially is im- proved in Fig. 3b compared with Fig. 3a. Mean absolute errors between experimental data and ideal data are shown in Table 1. In Table 1, u is the error of x components of flow vectors and v is the errors of y components. The result shown in Fig. 3b was found to be better than that in Fig. 3a.

4.2 Object tracking

The loci of the moving objects shown on the image se- quence in Fig. 4 were traced. In this sequence, object 1 moves up three pixels and moves left by three pixels per frame. Object 2 moves up three pixels and moves right by three pixels per frame. Object 3 moves down three pixels

Fig. 2. Image sequence

4 Experiments

4.1 Improvement of the precision of the local optimization method

In this experiment, we used the image sequence shown in Fig. 2. Flow vectors produced when making the sphere move to the right by five pixels are shown in Fig. 3.

Fig. 3. Optical flow

Fig. 1. Detecting clusters of moving objects Fig. 4. Image sequence

184

Table 1. Mean absolute errors

u (pixels) v (pixels)

a 1.9 0.5 b 1.4 0.3

per frame. The result of clustering is shown in Fig. 5. It shows that regions of the three moving objects are clustered well.

Figure 6 shows that the loci of the three moving objects can be pursued well. In this method, we found that the precision of this object tracking method depended on the precision of the clustering.

In this experiment, the loci sometimes could not be pur- sued correctly because the ranking of clusters changed. This is believed to be caused by the methodology in which the correlation between frames is not taken into consideration because we obtain clusters using independent histograms for every frame. As an improvement, it is possible to in- clude the temporal and spatial correlation between each flame in the constraint condition of numbering.

Fig. 5. Result of clustering 5 Conclusion

We found that precision of optical flow when object move- ment changed substantially was improved when using the technique described in Sect. 2. In the future, we consider that it is necessary to verify whether this technique is appro- priate for different image sequences.

To improve the precision of object tracking, we have to improve the precision of clustering. In the method used in this study, it is possible to trace the loci of objects when they translate. However, we are yet to account for movements such as turns or approaches. The ability to trace such move- ments is desirable.

References

1. Miike H, Koga K (eds) (1993) Video image processing using a personal computer. Morikita, Tokyo, pp 133-143

2. Xu G, Tsuji S (1998) 3-D vision. Kyoritsu, Tokyo, pp 111-118 3. Agui T, Nagao T (1992) Image processing and image recognition.

Shokodo, Tokyo, pp 92-98

Fig. 6. Loci of moving objects