visual tracking using particle filters

8/3/2019 Visual Tracking using Particle Filters

http://slidepdf.com/reader/full/visual-tracking-using-particle-filters 1/15

1 ESTIMATORS

Mestrado em Engenharia Electrotecnica e de ComputadoresMestrado em Tecnologias de Informacao Visual

Visao por Computador

Assignment n 6

Visual Tracking using Particle Filters

1 Estimators

Many problems in science and engineering require the estimation of a set of variables related with some system from a sequence of noisy measurementsmade on some variables that are related to this system.

When such estimation is to be done by a computer a state space modelfor the dynamic system written using a discrete-time formulation is the mostadequate. This means that the evolution of the system is modelled usingdifference equations, and the measurements are assumed to be available atdiscrete times. All the attention is focused on the state vector which should

contain all the relevant information to describe the system.In a (visual) tracking problem, this information is normally related to thekinematic characteristics of the target, and the measurement vector repre-sents noisy observations that are somehow related to the state vector. Al-though this is not a requirement, the measurement vector is frequently of lower dimension than the state vector.

Two models are required to analyse and infer about a dynamic system.The first one, known as the system model describes the evolution of thestate with time and the second one relates the noisy measures to the state,being known as the measurement model. Different approaches can be used toderive estimators for these kinds of problems and frequently the results are

the same. Here we are interested in the so called probabilistic approaches.These kind of approaches produce estimates under the form of probabilitydistributions, instead of deterministic values.

The probabilistic state space formulation and the requirement for updat-ing of the state information upon the reception of each new measurement

Visao por Computador 1 Paulo Menezes, 2011, DEEC



1.1 Sequencial importance sampling algorithm 1 ESTIMATORS

are well suited to a Bayesian approach. In such approach one attempts to

construct the posterior probability density function (pdf) of the state givenall the available information, which includes the set of received measure-ments. For the cases where an estimate must be obtained whenever a newmeasurement is received, a recursive filter is an adequate solution. This filteris normally divided in two stages: (1) prediction and (2) update (or correc-tion). In the first stage, the system model is used to predict the state pdf atthe next measurement time, from the previous one. Due to the presence of noise (which models the unknown disturbances that affects the system), thepredicted pdf is generally translated, deformed and spread. The receptionof a new measurement permits the adjustment of this pdf during the updateoperation. This is done using the Bayes theorem as the mechanism to update

the knowledge about the system state upon the reception of new information.There are different filtering algorithms that can be derived using a Bayesian

formulation, being the Kalman Filter (or its Extended version) one of themost used. Its requirements are that the models be linear (or differentiable)and the involved noise distributions be Gaussian.

Particle filters are in turn, particularly well-suited for vision-based track-ing applications, in particular due to the non-Gaussian characteristics of themeasurement functions involved. These are also known as Monte Carlo-basedmethods, as they involve the use of sets of random samples that approximatethe probability distributions we want to estimate.

1.1 Sequencial importance sampling algorithm

Suppose that there is a probability distribution p(x), from which it is verydifficult to draw samples, but we know that it is proportional to anotherdistribution π(x) which can easily be evaluated.

Let x(i) ∼ q(x), i = 1,...,N s be samples that are easily generated froma proposal distribution q(.), called importance density . Then, a weightedapproximation to p(.) is

p(x)≈

N s

i=1 w

(i)

δ(x − x

(i)

) (1)

where w(i) ∝ π(x(i))

q(x(i))is the normalised weight of the i-th particle.

Thus, if the samples were drawn from an importance density q(x0:k|z1:k)





and the posterior density can be approximated by

p(xk|z1:k) ≈N si=1

w(i)k δ(xk − x

(i)k )

Based on this, the Sequential Importance Sampling algorithm (SIS) isbased on the recursive propagation of the weights of the points. This recursiveupdate is performed each time a measurement is received. Its description canbe seen on algorithm 1.

Algorithm 1 Sequencial Importance Sampling Algorithm1: for i = 1 : N s do

2: x(i)k ∼ q(xk|x

(i)k−1, zk) {Draw sample, using proposal distribution}

3: w(i)k ← w

(i)k−1

p(zk|x(i)k

) p(x(i)k

|x(i)k−1)

q(x(i)k

|x(i)k−1,zk)

4: end for

5: for i = 1 : N s do

6: w(i)k ←

w(i)k

N j=1w

(i)k

{Normalise the importance weights}

7: end for

There is an important particular case of this framework that arises whenthe prior distribution is chosen as the importance distribution. In this case

the importance weights satisfy wik ∝ wik−1 p(zk|x(i)k ). It should be noted that

although this special case is widely used, the importance sampling methodis far more general than this.

The deficiency of this algorithm is that that most of the particles aftersome iterations show negligible weights. This means that most of the com-putational load used in updating these weights is worthless as these particlesmake (almost) no contribution to approximate the desired distribution.

A measure for the degeneracy of this algorithm is the effective samplesize N eff defined as

N eff =N s

1 + V ar(w∗ik )

(4)

where

w∗ik =

p(xk|z1:k)

q(xk|xk−1, zk)(5)




1.2 Re-sampling 1 ESTIMATORS

is referred as the “true weight” . This cannot be evaluated exactly, but an

estimate can be obtained

N eff =1N s

i=1(w(i)k )2

(6)

where w(i)k is the normalised weight. Note that N eff ≤ N s and small N eff

indicates severe degeneracy.Two solutions exist to reduce this effect, which are: a good choice of

importance density, and the use of re-sampling. The choice of a good im-portance density is not easily obtained for most of the cases. On the otherside, the second method is, in general, much simpler to implement being,

therefore, the most common choice.

1.2 Re-sampling

The re-sampling is a technique that can be used whenever a significant de-generacy of the samples’ weights is observed (N eff < N threshold).

The basic idea of the method is to eliminate the particles with smallweights and concentrate on particles with large weights.

This involves generating a new set xi∗kN si=1 by re-sampling (with replace-

ment) N stimes from an appropriate discrete representation of p(xk|z1:k) givenby

p(xk|z1:k) ≈N si=1

w(i)k δ(xk − x

(i)k ) (7)

in a way that P r(xi∗k = xi

k) = w jk. The obtained samples are i.i.d. sam-

ples from the discrete density and the weights are reset to 1/N s. Severalalgorithms for implementing the re-sampling are available that guaranteethe number of operations to be O(N s). The algorithm 2 shows one of thepreferred by most authors.

1.3 Sampling Importance Re-sampling Filter

This is a Monte Carlo method that can be applied to recursive Bayesianfiltering problems, and is known by several names like: Particle filter [?, ?],SIR Filter [?], Bootstrap filter [?] or CONDENSATION [?].




1.3 Sampling Importance Re-sampling Filter 1 ESTIMATORS

Algorithm 2 Re-sampling algorithm

1: c1 ← 0 {Construct the CDF}2: for i = 2 : N s do

3: c(i) ← ci−1 + w(i)k

4: end for

5: i ← 1 {Start at the bottom}6: u1 ∼ U[0, N −1

s ] {Draw a starting point}7: for j=1:Ns do

8: u( j) ← u1 + N −1s ( j − 1) {Move along the CDF}

9: while u( j) > c(i) do

10: i ← i + 111: end while

12: x( j)k ← x(i)

k {Assign particle}

13: w( j)k ← N −1

s {Reset weight}14: end for

This method can be derived from the SIS by setting the importance den-sity q(xk|x

(Ii)k−1, z1:k) = p(xk|x

(i)k−1) and applying the re-sampling at every step.

As usual it requires the knowledge of the system dynamics f (., .) and mea-surement equation h(., .). It must be possible to draw samples from the noisevk−1 and from the prior. The likelihood function p(zk|xk) must be evaluatedpoint-wise. 0011 0011 0011001100110011 0011 0011 00110011 0011 0011 00110 00 01 11 10 00 01 11 1 0 00 01 11 10011 00110 00 01 11 1 0 00 01 11 1

m e a s u r e f u n c t i o n

propagation

resampling

weighting

Figure 1: Illustration of the CONDENSATION algorithm




2 IMPLEMENTATION

This implies that the sample xik ∼ p(xk|x

(i)k−1) can be generated by draw-

ing a noise sample vik−1 ∼ pv(vk−1) and making x(i)k = f k(x(i)k−1|v(i)k−1) where pv is the probability density of vk−1.

The weights are then updated by

wik ∝ w

(i)k−1 p(zk|x

(i)k ) (8)

but as the resampling is applied in every step, we have w(i)k−1 = 1/N, ∀i, it

results inwk ∝ p(zk|x

(i)k ) (9)

These weights are normalised before the re-sampling stage as can be seenin algorithm 3 describes the SIR. One iteration of this algorithm is also

illustrated on figure 1.

Algorithm 3 Sampling Importance Re-sampling filter1: for i = 1 : N s do

2: x(i)k ∼ p(xk|x

(i)k−1){Draw particle}

3: w(i)k ← p(zk|x

(i)k )

4: end for

5: t ←N s

i=1 w(i)k

6: for i = 1 : N s do

7: wk ← t−1w(i)k

8: end for

9: Re-sample using algorithm 2

The advantages of this method are the simplicity of both computing theweights and sampling the importance density. It has the following disadvan-tages: the importance density is independent of the measure zk and thus thestate space is blindly explored, what can make this algorithm quite inefficientunder certain situations. As the re-sampling is applied in every iteration, itleads to the known sample impoverishment.

2 Implementation2.1 Monte Carlo π approximation

Considering a square of side length l and an inscribed circle (as in figure 2),which naturally has its diameter equal to the side of the square. The area of




2.2 Implement a particle filter to track a moving point on plane X-Y 2 IMPLEMENTATION

l

Figure 2: A circle inscribed in a square

the circle is

Ac = πl2

4,

and that of the square isAs = l2.

The ratio between these two areas is

Ac

As

= π4

.

If we draw randomly points over the square region, as long as their distri-bution is uniform, the ratio of number the points that fall on the circle fromthe total drawn inside the square, tends to converge to the ratio between theareas, as the number of points becomes very large. Demonstrate this usingMatlab and plot the evolution of the approximate value of π as more pointsare generated.

2.2 Implement a particle filter to track a moving point

on plane X-Y

Considering a point that’s moving and whose coordinates can be measuredby some noisy sensor. As we don’t know much about the properties of thepoint motion, we are going to use a constant velocity assumption.




3 IMPLEMENT A PARTICLE FILTER TO TRACK A SIMPLE

OBJECT ON A VIDEO SEQUENCE

As in a discrete implementation we normally use difference equations to

approximate the differential models, we can write the model as:

Vk = Vk−1

Xk = Xk−1 + Vk−1∆t

As

Vk−1 =Xk−1 − Xk−2

∆t

we can write the state evolution in a autoregressive form as

Xk = X k−1 + (Xk−1 − Xk−2)

= 2Xk−1 − Xk−2

As the vector Xk = (xk, yk), we can create an extended vector that containsthe present and past information

Xk =

xk

ykxk−1

yk−1

.

Then we can model the state evolution as Xk = FXk−1 where

F =

2 0 −1 00 2 0 −11 0 0 00 1 0 0

.

The following figure shows the matlab code for resampling a matrix of particles based on their respective weights

3 Implement a particle filter to track a simpleobject on a video sequence

Here we intend to track a U-shapped line drawn on a sheet of paper, like theone shown in figure 8.






% i n i t i a l i z e t h e p a r t i c l e s a nd w e i g h t sp= u n if r n d (0 ,50 0 ,2 ,5 00 );p a r t i c l e s =[ p ; p ] ;we igh ts =on e s (1 ,5 00) /5 00 ;

% main c y c l e

while ( 1 )p a r t i c l e s =r e s am p l e ( p a r t i c l e s , w e i g h t s ) ;p a r t i c l e s =p r e d i c t ( p a r t i c l e s ) ;measure=getmea sure () ;w e i g h t s =d o w ei g h t ( m ea su re , p a r t i c l e s ) ;

[ sta te , cov ar ]=computeMean ( ) ;end

Figure 3: Particle filter main cycle

Tracking this object on the image sequence means that we intend tocontinuously estimate its position, orientation and scale. Thus each particlewill propose an hypothesis (see figure 9) for these values and need to bevalidated by some measuring function (weighted).

For this we need to define a measuring model. A perfect estimation of

the above parameters will make a model of the tracked object to fall per-fectly onto its image. We may consider a measure of matching the distancesbetween the estimated model and the contours of the object in the image.In a particle filter each particle represents a proposal for the estimate of theparameter vector that represents the target state. Each of these proposalsmust then be evaluated to measure how close it is to the real state of thetarget we are trying to estimate. In many problems, the state of the targetcannot be measured directly, but only a manifestation of that can be per-ceived. This is the case of the current problem, where each captured imageof the video sequence is not the measure, but contains the effects of the of the target presence, together with other possible objects and background

scenarios that we will consider as distracting factors or noise.The proposed solution is based on the assumption that the target is visible

and its contours are the only one that match perfectly with a line model thatrepresents it. Therefore if we are able to place the model over it, with thesame orientation, scale, then the sum of distances between the target contourpoints will be zero. Furthermore this sum of distances will increase as the






function [ p a rt s , w e i g t ] = r e s a m p l e ( p a r t i c l e s , w e i g h t s )p a r t s =zeros ( 4 , 5 0 0 ) ;[ rows , co l s ]= size ( we igh ts ) ;cdf=zeros ( r o ws , c o l s ) ;c d f ( 1 )= w e i g h t s ( 1 ) ;for i = 2: c o l s ,

cd f ( i )=cd f ( i−1)+wei ght s ( i ) ;end

i =1;u= u n if r n d (0 ,1/ c ol s ) ;for j = 1: c ol s ,

u=u+1/c o l s ;while u > wei ght s ( i )

i=i +1;end

p a r t s ( : , j )= p a r t i c l e s ( : , i ) ;we igh ts ( j )=1/ c o ls ;

end

we igt=we igh ts ;end

Figure 4: Resampling in matlab

function p a r t s=p r e d i c t ( p a r t i c l e s , F )parts=F∗ p a r t i c l e s ;pa rt s=pa rt s+random( ’ Normal ’ , 0 ,2 , size ( p a r t i c l e s ) ) ;end

Figure 5: Predicting in matlab

model configuration goes away from the true one. The next figure showthe contour of the input image shown above. Figure 10a) represents clearlythe U-shape to be tracked but also 2 other u-like shapes, that may attractthe tracker. This would generate false tracks that can be avoided by imagepreprocessing to remove these unwanted contours or to impose restrictions onthe size (scale) of the viewed target. On figure 10b) we can see the hypothesisof the model state, being the closest to the target the one that visibly willhave smaller distances to the target contour edges.






function we igh ts =d owe igh t ( me as u re , p a r t i c l e s )% e s t a b l i s h some m ea su ri ng f u n c ti o n

for i =1: length ( p a r t i c l e s )dx=measure(1)− p a r t i c l e s ( 1 , i ) ;dy=measure(2)− p a r t i c l e s ( 2 , i ) ;% r e c a l l t h a t f or a p a r t i c l e c l os e t o t he p oi nt t h e d is ta nc e i s s ma ll

% and t h e w e ig h t a pp ro ac he s 1

wei t ( i )=exp(−dx∗dx−dy∗dy ) ;end

weights=weit ;end

Figure 6: Weighting the particles

function pos=get mea sure ( i )% e s t a b l i s h some m ea su ri ng f u n c ti o n

p os = [100∗ sin ( i /20∗pi ) ; 200∗cos ( i /40∗pi ) ] ;pos=pos+random( ’ Normal ’ , 0 ,1 0 ,2 , 1) ;end

Figure 7: Generating noisy measures

Figure 8: A possible frame of the video sequence containing the shape to

track

For computing the distance to the target contours, we would select 5points over the line-model of the target, and compute the distances di between






Figure 9: Representations of states proposed by different particles

a) b)

Figure 10: a)Extracted contours of image represented in figure 8; b) super-imposing 3 hypothesis for the model configuration on the same image.

these points and the nearest contour. Defining D(x) as

D(x) =i

di(x),

where di(x) is the distance between point i of the model and the nearestimage contour, for a given proposed configuration x, we can define the weighof each particle j as

w( j) = e−D(x( j)).

The choice of the negative exponential function is because for a perfect matchD(x( j)) = 0 so the exponential is 1, and decreases for increasing values of the sum of distances. This this is indeed the perfect shape for the weighting




3.1 Using the distance transform



function, as it gives more weight to particles that represent good proposalsand less weight for those that represent bad proposals.

3.1 Using the distance transform

To simplify the computation of measures between contours and given im-age locations, we can use the concept of distance transform. The distancetransform of a contour image, consists in an image where each pixel valuerepresents the distance between that that location and the nearest contour.Figure 11 shows the distance transform of the contour image of figure 10. Inmatlab use the bwdist function. Now to obtain the distance of the model

Figure 11: Distance transform of the contour image

points is just the matter of taking the values of the distance transform forthe locations of these points.

3.2 The measurement and state models

Lets consider the U-shape defined by the line segments [(-5,5),(-5,0)], [(-5,0),(5,0)] and [(5,0),(5,5)]. Thus selecting some points over these line seg-ments we can get the set of points defined in table 1. Now for each set of

x -5 -5 -5 -5 -5 -5 -3 -1 1 3 5 5 5 5 5 5

y 5 4 3 2 1 0 0 0 0 0 0 1 2 3 4 5

Table 1: Coordinates of measuring points defined in target coordinates

parameters proposed by each the particles we just need to apply the corre-




3.2 The measurement and state models



sponding transformation to this set of points to obtain their correspondingcoordinates.The parameters of the state vector to estimate will correspond to the x

and y coordinates of the model on the image frame, its scale and orientation.For this we need to define also the matrix F that models the evolution of these parameters.

For the current work you can consider a constant velocity model, definedin an autoregressive manner as in section 2.


visual tracking using particle filters

Documents