HUMAN TRACKING TECHNIQUES

NIRANJAN KAMATCSE 725

Different Tracking Techniques

1.Gaussian Mixture Model: 1.Construct the model of the

Background. 2.Given sequence of background

images find the mean and variance of mixture of the Gaussians.

3.Given a new pixel find out the probability of the pixel value having come from the Gaussians.

2.NCC:Normalized cross correlation 1.For each pixel find out cross

correlation with the background. 2.Normalize by dividing the

variances. 3.Sometimes better results obtained

by not normalizing.

3.Entropy: 1. The entropy is defined as the amount

of information needed to exactly specify the state of the system, given what we know about the system.

2. If the two images are exactly alike their entropy will be low.

3. The lower the entropy the higher the likelihood that the pixel belongs to the background.

4.Mutual Information: 1. Mutual information of two random

variables is a quantity that measures the mutual dependence of the two variables.

2. The higher the mutual information the higher the likelihood of the pixel belonging to the background.

3. Considered to be a stronger concept than entropy theoretically.

Gaussian Mixture Model 1.Technique developed by Chris Stauffer

and W.E.L. Grimson during CVPR 1999. 2. The technique is so powerful and

omnipresent in tracking,HCI,etc. that it won the Longuet-Higgins award at CVPR 2009.

Gaussian Mixture Model(continued)

We model the pixel history with K Gaussian Probability Density Functions.

The history of a pixel {x,y} is modeled as time series.

(vc.cs.nthu.edu.tw/.../Background_Estimation_with_Gaussian_Distribution_for_Image_Segmentation,_a_fast_approach.ppt)

Gaussian Mixture Model(continued)

The probability of observing the current pixel is given by

is the weight parameter that is used to describe by the Gaussian distribution

is a Gaussian distribution that has two parameters: is the mean of the Gaussian distribution at time t and is the covariance matrix at time instant t

Gaussian Mixture Model(continued)

A new pixel is said to match a distribution If it is within 2.5 standard deviations from

mean of the distributionThis distribution are updated with this pixel

value

is the learning rate defined by the user.I have set it as 0.1 which gives us good results.

Gaussian Mixture Model(continued)

Gaussian Mixture Model(continued)

The weight parameters of all distributions are updated as

is 1, for matched distribution 0, for unmatched distribution

Gaussian Mixture Model(continued)

If the current pixel didn’t match with any of the K distributionsThe distribution with smallest weight is replaced

by a new distribution

The new distribution withThe mean is set to the value of the current pixelThe variance is set largeThe weight is set to a small value

Gaussian Mixture Model(continued)

Define which of distributions describing the history of a pixel result from background1. Order all distributions by a factor2. B first distributions are marked as background

distribution

If a pixel is matched with one of these B distributions it is marked as a background pixel

Gaussian Mixture Model(continued)

1. Remove the singular pixels and small blobs by eroding.

2.Use connected components to segment out the persons.

Results Here are the results to a normal video

sequence. As you can see the shadows have been

removed so has the noise. Further improved results can be

obtained by edge detecting.

A Color Similarity Measure for Robust Shadow Removal in Real-Time 1. This is a paper by Daniel Grest, Jan-

Michael Frahm, and Reinhard Koch. 2. Under normal conditions it gives us

suitable results. It is suitable for indoor use. However

with the complexities of outside lighting, etc. it fails to live up to background modeling.

A Color Similarity Measure for Robust Shadow Removal in Real-Time 1. To detect foreground regions we use a

background reference image and calculate the difference image with the current one.

2. To acquire a background reference a ground truth image is incorporated as a mean image of a sequence of the interaction area without the user.

3. The segmentation of the foreground is often obtained by applying a threshold to the difference values.

A Color Similarity Measure for Robust Shadow Removal in Real-Time(Continued) 1. Instead of calculating the variance for

each pixel over time, we measure the variance of the pixel noise over the whole difference image each frame, leaving the segmented foreground region out.

A Color Similarity Measure for Robust Shadow Removal in Real-Time(Continued) Let d(x,y) = r(x,y) - a(x,y) be the

difference image value, r(x; y) the background reference image intensity and a(x,y) the current image intensity at image positions (x,y). The variance of noise in an image with size M X N is

Variance= Sum over the window ( d(i,j)-Mean Image)^2

A Color Similarity Measure for Robust Shadow Removal in Real-Time(Continued)

Assuming a Gaussian distribution of noise we set the segmentation threshold to 2 * Variance . Therefore 97.5% of noise are in the segmentation interval.

Each pixel belongs to the foreground, if |d(x,y)-Mean Image| > 2 * Variance. Now we have to remove the shadow.

A Color Similarity Measure for Robust Shadow Removal in Real-Time(Continued)

To measure the similarity of image parts it is possible to calculate the cross correlation over a given window size . Usually brightness invariance is desired, as given by the cross covariance, which is the average-free cross correlation.

A Color Similarity Measure for Robust Shadow Removal in Real-Time(Continued)

The cross covariance with window size M X N for two images a,b at image position (x,y) is then defined as

CC(a,b)(x,y) =(1/MN) Sum over window (Pixel(x,y) of Image a-Mean window a) *

(Pixel(x,y) of Image b-Mean window b) To make it contrast invariant divide by

the variance of each of the windows.

A Color Similarity Measure for Robust Shadow Removal in Real-Time(Continued)

For shadow pixels this correlation is near to one, so we eliminate the darker pixels in the segmented image, if the NCC of the pixel in the current image and the pixel in the reference image is greater than a threshold NCC. After removing pixels with the NCC thresholding, we apply a 3x3 closening operator to fill small holes in the contour.

A Color Similarity Measure for Robust Shadow Removal in Real-Time Results

Given below are the NCC results. Using only NCC doesn’t take care of moving

objects like trees blowing in wind,moving shadows,etc.

It is suitable only for indoors with restricted environment.

Using edge detection and NCC can give us improved results.

However, it is not as powerful as background modeling.

Entropy Results If two images are similar their entropies

should be close. Hence, by taking the differences in

entropy over a window, we can find the similarity.

This method doesn’t give us results as good as those for Background Modelling or even NCC.

Conclusion Background Modelling gives us the nest

results. Done on a GPU it gives us real time

speed.