Machine Learning for Vision-Based Motion Analysis
Learning pullback metrics for linear models
Oxford Brookes Vision Group
Oxford Brookes University17/10/2008
Fabio Cuzzolin
Learning pullback metrics for linear models
Distances between dynamical modelsLearning a metric from a training setPullback metricsSpaces of linear systems and Fisher
metricExperiments on scalar AR(2) models
Distances between dynamical models
Problem: motion classificationApproach: representing each movement as a linear dynamical modellinear dynamical modelfor instance, each image sequence can be mapped to an ARMA, or AR linear modelClassification is then reduced to find a suitable distance function in the space of dynamical distance function in the space of dynamical modelsmodelsWe can then use this distance in any distance-based classification scheme: k-NN, SVM, etc.
ji
ij
xpxpEg
),(log,),(log
Proposed distances ...
Fisher information matrixFisher information matrix [Amari] on a family of probability distributions
Kullback-Leibler divergenceKullback-Leibler divergenceGap metricGap metric [Zames,El-Sakkary]: compares graphs associated with linear systems as input-output mapsCepstrum normCepstrum norm [Martin], Subspace anglesSubspace angles [DeCock]all task specific!
Learning pullback metrics for linear models
Distances between dynamical modelsLearning a metric from a training setPullback metricsSpaces of linear systems and Fisher
metricExperiments on scalar AR(2) models
Learning metrics from a training set
it makes no sense to choose a single distance for all possible classification problems as…... labels can be assigned arbitrarily to dynamical systems, no matter what their structure is
when some a-priori info is available (training set)..
.. we can learn in a supervised fashion the “best” .. we can learn in a supervised fashion the “best” metric for the classification problem!metric for the classification problem!a math tool for this task: volume minimization of volume minimization of pullback metrics pullback metrics
Learning distances
many algorithms take an input dataset and map it to an embedded space, implicitly learning a metric (LLE, etc)
they fail to learn a full metric for the whole input space
[Xing, Jordan]: maximizes classification performance for linear maps y=A1/2 x > optimal Mahalanobis distanceoptimal Mahalanobis distance
[Shental et al]: relevant component analysisrelevant component analysis – changes the feature space by a global linear transformation which assigns large weights to “relevant dimensions”
Learning pullback metrics for linear models
Distances between dynamical modelsLearning a metric from a training setPullback metricsSpaces of linear systems and Fisher
metricExperiments on scalar AR(2) models
Learning pullback metrics
consider than a family of diffeomorphisms F between the original space M and a metric space N (can be M itself)
the diffeomorphism F induces on M a pullback metricpullback geodesics are “liftings” of the original ones
Pullback metrics - detail
)(
:
mFm
MMF
diffeomorphismdiffeomorphism on M:
MTvMTv
MTMTF
mFm
mm
)(
*
'
:
push-forwardpush-forward map:
),(),( **)(* vFuFgvug mFm diven a metric on M, g:TMTM, the
pullback metricpullback metric is
N
k
M
k
k
dmmg
mgDO
1 2
1
2
1
))((det
))((det)( Inverse volumeInverse volume:
Inverse volume maximization
the natural criterion would be to optimize the classification performancein a nonlinear setup this is hard to formulate and solvereasonable to choose a different but related objective function
finds the manifold which better interpolates the data (geodesics have to pass through “crowded” regions)
Learning pullback metrics for linear models
Distances between dynamical modelsLearning a metric from a training setPullback metricsSpaces of linear systems and Fisher
metricExperiments on scalar AR(2) models
Space of AR(2) models
given an input sequence, we can identify the parameters of the linear model which better describes itautoregressive models of order 2 AR(2)Fisher metric on AR(2)
Compute the geodesics of the pullback metric on M
21
12
2212121 1
1
)1)(1)(1(
1),(
aa
aa
aaaaaaag
A family of diffeomorphismsstretches the triangle towards the vertex with the largest lambda
332211 ,,1
)( mmmm
mFp
Effect of optimal diffeomorphism
effect of diffeomorphism on a training set of labeled dynamical models
Learning pullback metrics for linear models
Distances between dynamical modelsLearning a metric from a training setPullback metricsSpaces of linear systems and Fisher
metricExperiments on scalar AR(2) models
Exps on Mobo database
experiments on action and ID recognition on the Mobo databasesingle feature
(box width)
used NN to classify image sequences seen as AR(2)relative performance of pullback and other distances measured
Results – ID recognition
identity of 25 people from 6 different views (hard!)pullback metrics based on two different diffeomorphisms ...... are compared with other classical applicable a-priori distances
Results - action
Action recognition performance, all views considered – second best distance function
Action recognition performance, all views considered – pullback Fisher metric
Action recognition, view 5 only – difference between classification rates pullback metric – second best
Conclusions
motions as dynamical systems
classification → finding distance between systems
Having a training set we can learn the “best” such metric
formalism of pullback metrics induced by Fisher distance
design suitable family of diffeomorphism extension multilinear system easy! better objective function!
