1

A REAL-TIME ROBUST OBJECT TRACKING

A PRESENTATION OF

Prarinya Siritanawan (SIIT)Toshiaki Kondo (SIIT), Kanokvate Tungpimolrut (NECTEC), Itsuo Kumazawa (Tokyo Tech) Master of Information and Communication Technology for Embedded SystemSirindhorn International Institute of Technology

Presentation at International Advanced School on Knowledge Co-creation and Service Innovation 2012, Japan Advanced Institute of Science and Technology, March 1

2

OUTLINE

• Introduction• Hamming Distance based Gradient Orientation

Pattern Matching• Experimental Results• Conclusion• Question and Answer

INTRODUCTION

“OBJECT TRACKING IS A DETERMINATION OF LOCATION, PATH AND CHARACTERISTICS OF AN INTERESTED OBJECT”

3Subhash Challa, Mark R. Morelande, Darko Musichki and Robin J. Evans, “Fundamentals of Object

Tracking”, Cambridge University Press, 2011

4

INTRODUCTION

Applications for object tracking• Video surveillance• Human-machine interface• Robot control• Air space monitoring• Weather monitoring• Cell biology

Subhash Challa, Mark R. Morelande, Darko Musichki and Robin J. Evans, ”Fundamentals of Object Tracking”, Cambridge University Press, 2011

5

INTRODUCTION

Major problems of visual tracking are caused by• Illumination change• Occlusion• Computation time• Scaling• Rotation• Focus• Aperture

We focus on these problems

6

INTRODUCTION

• Typical visual tracking and motion estimation techniques assume that lighting conditions are constant and minimal occlusion.

• We proposed a new template matching technique.

7

INTRODUCTION

• Template matching is the intensity-based technique for measuring the similarity between template and corresponding block of image.

Template

Sample frame

Match

y

x

i jjiTjiISSD

i jjiTjiISAD

2),(),(

),(),(

Popular similarity metrics

8

INTRODUCTION

• We obtain an array of SADs or SSDs after scanning the template over the entire image.

Best matching position

Fig. 1. Inverted SSD result.

9

INTRODUCTION

• However SADs and SSD are sensitive to changing lighting conditions and occlusion. In order to develop a method that can provide the robustness to illumination change, a new template matching technique is used.

• For illumination change problem, we introduced a robust feature called Unit gradient vector (UGVs).

• To cope with the occlusion problem, we introduce the Hamming distance as a new matching method instead of SSD.

10

HAMMING DISTANCED BASED GRADIENT ORIENTATION PATTERN MATCHING

11

HAMMING DISTANCE BASED GRADIENT ORIENTATION PATTERN MATCHING• Hamming distance based Gradient Orientation

Pattern Matching (P.Siritanawan & T.Kondo)– Template matching based technique using Hamming

distance (HD) on Unit gradient vectors (UGVs).

Fig. 2. Intensity image. Fig. 3. Unit gradient vectors in x and y direction

12

TemplateSample Image

gx2 gy2gx1 gy1

Normalize

nx2 ny2nx1 ny1

Threshold Absolute Diff.

OR

1st Derivative (Sobel Operator)

return [Best matching position (x,y)]

Threshold Absolute Diff.

Sum

Step 1Extract UGVs

feature

Step 2Perform template matching by using Hamming distance

Iterate (N-M-1) blocks

Block at position

(x,y)

13

• UGV is a robust feature against Illumination changes.

UNIT GRADIENT VECTORS

Intensity image Gradient vectors Unit gradient vectors

Normal condition

Lighting change condition

14

• The unit gradient vectors (UGVs) feature can be extracted through the following normalized equations

UNIT GRADIENT VECTORS

where Ix and Iy are gradient of intensities in x and y direction

is a small constant to prevent zero division

15

MATCHING METHOD

• We introduce Hamming Distance (HD),

1st Pattern 2nd Pattern

0 0 1 1

0 0 1 1

0 0 1 1

0 0 0 1

=

HD(x,y) = 8HD counts the number of pixels that are not match

16

MATCHING METHOD

• HD uses XOR but UGVs is not binary info.• We need to transform the non-binary image

to be binary image using threshold absolute difference function (O.Pele & M.Werman),

17

MATCHING METHOD

• Then the total distance of the block at position (x,y) is given by

Fig. 4. Inverted HD result.

18

MATCHING METHODImage

featuresTemplate Occluded

ImageSimilarity

metricScore

IntensitiesSAD

= 32

Unit gradient vectors(UGV)

Pixelwise voting

HD = 8

1 4 2 66 3 4 45 6 6 22 3 5 3

1 4 0 06 3 0 05 6 0 02 3 0 0

0 0 1 10 0 1 10 0 1 10 0 1 1

0 0 2 60 0 4 40 0 6 20 0 5 3

19

EXPERIMENTAL RESULTS

20

DEMONSTRATION

21

EXPERIMENTAL RESULTS

Fig. 5. Tracking results under irregular lighting with occlusion by (a) SSD on UGVs, (b) HD on UGVs, (c) and (d) are the distributions of the corresponding similarity measurements

Best matching peak found !!

Which is the best matching position ?

?

??

?

22

CONCLUSION

• A novel pattern matching technique combines the advantages of – Unit gradient vectors (UGVs)– Hamming distance metric (HD)

• UGV is a robust feature against the time-varying lighting conditions.

• Compared with conventional matching with SAD or SSD on intensity, HD yields better results in partial occlusion scenarios. (60-70% covered).

• Efficient over the existing matching techniques on both synthetic and real image sequences.

23

PUBLICATION

1. Wattanit Hotrakool, Prarinya Siritanawan, and Toshiaki Kondo, “Real-time Gradient Orientation Pattern Matching”, International Conference on Embedded System and Information Technology, Chaing Mai, Thailand, 2010

2. Wattanit Hotrakool, Prarinya Siritanawan, and Toshiaki Kondo, “A Real-time Eye-tracking Method using Time-varying Gradient Orientation Patterns”, In proc. ECTI-CON, Thailand, 2010

3. Prarinya Siritanawan and Toshiaki Kondo, “Hamming Distance based Gradient Orientation Pattern Matching”, In proc. International Symposium of Artificial life and Robotics 17th, Chaing Mai, Oita, Japan, January 2012

4. Prarinya Siritanawan, Toshiaki Kondo, Kanokvate Tungpimolrut, Itsuo Kumazawa, “A visual tracking method using the Hamming distance”, In proc. International Conference on Information and Communication Technology for Embedded System 3rd, Bangkok, Thailand, March 2012

24

ACKNOWLEDGEMENT

This research is supported by• National Research University Project of Thailand,

Office of Higher Education Commission• Sirindhorn International Institute of Technology (SIIT)• Thailand Advanced Institute of Science and

Technology (TAIST) • Tokyo Institute of Technology• National Electronics and Computer Technology

Center (NECTEC)

25

QUESTION AND ANSWERTHANK YOU FOR YOUR ATTENTION