ISOMAP TRACKING WITH PARTICLE FILTER

Presented by Nikhil Rane

Dimensionality Reduction

Let xi be H-dimensional and yi be L-dimensional then dimensionality reduction solves the problem xi = f (yi) where H>L

Dimensionality Reduction Techniques Linear

• PCA• Transforms data into a new coordinate system so that largest

variance in on the 1st dimension, 2nd largest along 2nd dimension …

• Classical MDS• Preserves Euclidean distances between points

Nonlinear • Isomap

• Preserves geodesic distances between points

• LLE• Preserves local configurations in data

Face Database

Principal Components Analysis (PCA)

1) Make the mean of the data zero2) Compute covariance matrix C3) Compute eigenvalues and eigenvectors

of C4) Choose the principal components5) Generate low-dimensional points using

principal components

Performance of PCA on Face-data

Classical Multidimensional Scaling (MDS) Compute Distance Matrix S

Compute inner product matrix B = -0.5JSJ where J = IN – (1/N)11T

Decompose B into eigenvectors and eigenvalues

Use top d eigenvectors and eigenvalues to form the d dimensional embedding.

Performance of MDS on face-data

Locally Linear Embedding (LLE)

Find neighbors of each data point

Compute weights that best reconstruct each data point from its neighbors

Compute low-dimensional vectors best reconstructed by the weights

Performance of LLE on Face-data

Geodesic Distance

Geodesic distance – the length of the shortest curve between two points taken along the surface of a manifold

Isometric Feature Mapping (Isomap)

Construct neighborhood graph

Compute shortest paths between points

Apply classical MDS

Performance of Isomap on face-data

Tracking vs. Detection

Detection - locating an object independent of the past information

• When motion is unpredictable

• For reacquisition of a lost target Tracking - locating an object based on past information

• Saves computation time

Recursive Bayesian Framework

Estimate the pdf of state at time t given the pdf of state at time t - 1 and measurement at time t• Predict

• Predict state of the system at time t using a system-model and pdf from time t – 1

• Update

• Update the predicted state using measurement at time t by Bayes’ rule

Kalman Filtering vs. Particle Filtering

Kalman filter assumes the pdf of the state to be Gaussian at all times and requires the measurement and process noise to be Gaussian

Particle filter makes no such assumption and in fact estimates the pdf at every time-step

Resampling

Condensation algorithm Algorithm – 1) Resample 2) Predict 3) Measure

Condensation algorithm

Isomap Tracking with Particle Filtering

Create training set of a person’s face (off-line)

Use Isomap to reduce dimensionality of the training set (off-line)

Run particle filter on test sequence to track the person

Training Data

Isomap of Training Data

Isomap Discrepancy Isomap gave dimensionality of 2 when head poses moving up

were removed. Thus, the dimensionality of 3 recovered by training data can be attributed to the non-symmetry of the face about the horizontal axis.

Weighting Particles by SSD

Weighting Particles by Chamfer distance

State evolution without resampling

State evolution with resampling

Experimental Results

Videos

Videos Continued

Conclusion and Future work Isomap provides good frame-work for pose estimation

Algorithm can track and estimate a person’s pose at the same time

Use of particle filter allows parallel implementation

Goal is to be able to build an Isomap on-line so that the particle filter tracker can learn as it tracks

Thank You!