Multiple People Detection and Tracking with Occlusion

Presenter: Feifei HuoSupervisor: Dr. Emile A. Hendriks Dr. A. H. J. Stijn OomesInformation and Communication Theory (ICT) GroupDelft University of Technology

Nov. 29th , 2007

Outline Definition and objective of the project 2-D Human Model Particle Filter For People Detection and Tracking Color histogram matching Experiment Results Next

Definition and Objective of the Project

Objective : Develop fast and robust algorithms that can detect, track,

and model accurately and robustly individual persons in the real 3D world

Challenge : Indoor scene with lighting condition changing Multi-person tracking with occlusion

Overview of Proposed Algorithm

Foreground binary image extraction People model definition People detection and tracking using particle filter Multi-people tracking with occlusion

Foreground Binary Image

Current Frame

Background gray image

Gray image

RGB to GRAY

Background image

RGB to GRAY

Threshold

2-D Human Model

• The geometric properties of silhouette are used to determine if the moving objects has a human shape.

• It is convenient to describe the 2-D model mathematically, where a human hypothesis is a vector of parameters whose values are positions and size.

( ) ( )Area A Area B

( , , )P x y scale

2-D Human Model How to use this shape feature?

(I) (II) (III)

(I) and (II)----Low Score, (III)---High Score.

Conclusion: Only when the position and scale of this shape

model fit people well, we can get high fitness value .

,1=

( ) 0, otherwise

A B if A B

Area A

Particle Filter For Detection and Tracking A particle set is generated with an initial distribution.

Then the observation steps take place and the weights are calculated from the observation sample.

The new set of weights form the estimation to the posterior (and therefore the prior for the next iteration).

( ) ( )( )

( )

P B A P AP A B

P B

Initialization of the detection and tracking system

Step1. Get foreground binary image

Step2. Foreground blob segmentation

Step3. Size filter to get candidate blob with people

Step4. Initialize particle filter with Gaussian distribution

Step1 Step2 Step3 Step4

Particle filter for people detection and tracking

Iteration=20

Initial Frame

Particle system Detection result

Multiple people tracking with head model

• After the convergence of the head-should-upper body model, we can set the nominal scale of the head model for tracking people.

• Head model can provide more accurate position and scale information of the people.

Iteration

( ) ( )Area A Area B

Multiple people tracking with occlusion

Demo2

Demo1

Use discriminative feature to identify different people

• Objective:

1. Find out whether person A occludes person B, or the other way

around. 2. A group of people detection and tracking.

• Solution:

1. Use color information to distinguish different people.

2. The parameters of 2-D human model are increased into positions,

size and color. P=(x, y, scale, color)

Color histogram similarity

1. Initialize color histogram before occlusion.

2. Calculate color histogram similarity when occlusion.

3. Identify individual people after occlusion.

16 16 16

1...16 16 161

ˆ ˆ ˆq= 1, , ,c cu uu

u

q q c R G B

Demo3

16 16 16

1...16 16 161

ˆ ˆ ˆp(y)= (y) 1, , ,c cu uu

u

p p c R G B

16 16 16

1

ˆ ˆ ˆ ˆ ˆ(y) [p(y),q]= (y) , , ,c cu u

u

p q c R G B

To be continued

1. Evaluation of the algorithm2. Testing with different videos

Objective:1. Optimize the parameters in the algorithm. 2. Increase the implementation speed.

Thanks for your attention !

?