1

Mean shift and feature selection

ECE 738 course project

Zhaozheng YinSpring 2005

Note: Figures and ideas are copyrighted by original authors

2

Outline

Detecting, tracking and recognizing moving

objects in video sequences is a hot topic.

• Mean shift algorithm---target localization

• Feature selection---target representation

• Experiment demos

• Conclusion and future work

3

Mean shift: (1)history

Y. Cheng. “Mean Shift, Mode Seeking, and Clustering” PAMI 1995

D. Comaniciu, P Meer. “Robust analysis of feature spaces: color image segmentation” CVPR 1997

GR Bradski. “Computer vision face tracking for use in a perceptual user interface”. Intel Technology Jounal 1998.

D Comaniciu, V Ramesh, P Meer. “Real-time tracking of non-rigid objects using mean shift”. CVPR 2000 Best paper award and patent filed.

4

Mean shift: (2)Idea

Ss

Ss

xsK

sxsKxm

)(

)()(

..0

1)(

wo

xifXK

xxmshiftmean )(:

)(: xmxUpdate

5

Mean shift: (2)Idea

• Mean shift algorithm climbs the gradient of a probability distribution to find the nearest domain mode (peak)

@R. Collins CVPR 2003 @Comaniciu PAMI 2003

6

Mean shift: (3)Comments

Instead of exhaustive search in the window, the gradient information provided by the mean shift is used to reduce the time cost

Much better than moving the search window pixel by pixel and scanning row by row

7

Mean shift: (4)feature selection comes up…

• Given a likelihood image, locate the optimal location of the tracked object

• The likelihood image is generated by computing, at each pixel, the probability that the pixel belongs to the object based on the distribution of the feature

8

Feature selection: (1)Introduction

• Feature description approaches– Statistical descriptor– Structure descriptor– Spectral descriptor

e.g. intensity, color, texture, appearance, shape, motion, depth and so on.

@Bradski Intel Technology Journal 98’

9

Feature selection: (2) histogram

• Color histogram is widely used as object feature

@Bradski Intel Technology Journal 98’

Red: 1D cross section of an subsampled color probability distribution of a image

White: search window

Blue: mean shift point

10

Feature selection: (3) comments

• Foreground and background appearance changes every frame, so the object features need to update

• Color tracking affected by colored lighting, dim illumination or too much illumination

• More object information should be used to increase the tracker robustness

@Bradski Intel Technology Journal 98’

11

Experiment demos

Original frame

Probability distribution color tracking

12

Experiment demos

Non-rigid shape change lighting change on object

Object with non-identical color similar color with background

13

Conclusion and future work

• Mean shift algorithm can use view-point insensitive appearance model (like color histogram) to track non-rigid object on changing background

• Tracking success and failure will depend on the likelihood image which relates to the feature selection