a re-evaluation of pedestrian detection on riemannian manifolds
DESCRIPTION
Boosting covariance data on Riemannian manifolds has proven to be a convenient strategy in a pedestrian detection context. In this paper we show that the detection performances of the state-of-the-art approach of Tuzel et al. can be greatly improved, from both a computational and a qualitative point of view, by considering practical and theoretical issues, and allowing also the estimation of occlusions in a fine way. The resulting detection system reaches the best performance on the INRIA dataset, setting novel state-of-theart results.TRANSCRIPT
A Re-evaluation of Pedestrian Detection on Riemannian Manifolds
D. Tosato1, M. Farenzena1, M. Cristani1,2 and V. Murino1,2
1 Dipartimento di Informatica, University of Verona, Italy 2 Istituto Italiano di Tecnologia (IIT), Genova, Italy
The Problem
• Detecting people in images is still a hard task
• The detector must be robust and efficient to learn and test.
• Should possibly detect partially occluded pedestrians
2
Large variations of appearance
Different acquisition settings
Different light conditions Resolution
Occlusions
The goal of this work is …
• building an effective pedestrian detection framework for video surveillance applications,
• exploiting the power of covariance matrices as object descriptors,
• also dealing with pedestrian occlusions which are frequent in crowded scenes.
Related work: • O. Tuzel, F. Porikli, P. Meer. Pedestrian detection via classification on Riemannian manifolds.
IEEE PAMI, 2008.
• V. Arsigny, P. Fillard, X. Pennec, N. Ayache. Geometric means in a novel vector space structure on symmetric positivedefinite matrices. SIAM Journal on Matrix Analysis and Applications 29(1), 2008.
3
Background information
• A human is described with a set of covariance matrices.
• Covariance matrices live on a Riemannian manifold and typical machine learning techniques are not usable.
• Covariances have to be projected on local manifold views (vectorial spaces) for detection purposes.
• Classifiers are learned on the local views and combined with boosting.
4
Overview
• From the work of Tuzel et al., we developed and tested a more efficient and effective strategy for pedestrian detection.
• We increase the accuracy and efficiency of the original work by addressing some empirical and theoretical issues:
1. A more informative selection of weak learners (WLs).
2. An effective training set building procedure avoiding risks of overtraining.
3. A more efficient way of working on Riemannian Manifolds.
4. A more effective choice of the regressors as WLs.
5. A procedure to manage partially occluded pedestrians.
The boosting procedure: LogitBoost [J. Friedman, T. Hastie, R. Tibshirani, Ann Statist., 2000]
6
• LB is a boosting framework which fits iteratively an additive symmetric logistic model to get the posterior over the classes.
• Given {Xi,yi} i=1,..,N, the probability of Xi being in class 1 (human) is
where
is the strong classifier composed by a set of weak learners { f
l }
• The update step combines the weak classification response coming from the current linear regressor application
• Each WL focuses on a sub-window whose size and positions is selected from a bunch of candidates sizes and positions sampled uniformly over the whole pedestrian image
7
LogitBoost
Binary weak classifier Regressor
• We build a prior map on which stable regions are highlighted.
• WLs are selected sampling this prior distribution over the whole pedestrian image.
A more informative selection of the weak learners {Opt1}
8
Sampling Prior Map
Stable
Unstable
This speed up the learning process and minimize the selection of patches on the background area.
Avoiding overtraining {Opt2}
Construction of an ordered training set of negative examples, which decreases the risk of overtraining and improves the classification efficiency
9
Input Image
Edge Map
Edge Frequency Map
HARD
EASY
Avoiding overtraining: {Opt2} effects
10
Tuzel et al. [PAMI 2008] Ours
Cascade complexity is reduced of about 15% in its most used (first levels) part.
Analysis on a Riemannian Manifold {Opt3}
• Problem: working efficiently on a Riemannian Manifold
• Hybrid framework exploiting the properties of affine-invariance regarding the local mapping operations
and similarity-invariance for the projection operation
• In the original work the projection is done using the mean calculated iteratively
• The (approximate) closed form gives similar result 11
A more powerful weak classification strategy {Opt4}
• {Opt4} leads to a reduction of the 58% of the weak classifiers number maintaining a state-of the-art performance level.
• The generalization from a linear to a polynomial regressor is linear in the number of variables.
14
Default linear regression Polynomial regression
Pedestrian occlusion detection {Opt5}
• Analyzing the distribution of the weak classifier responses, we can detect occlusions
• The positive and negative weak responses are analyzed separately, peaks highlights when and where the detected person is occluded
16
To recap …
{Opt1} builds a light cascade, weak learners selection is based on a prior map
{Opt2} avoids the overtraining using a proper selection and ordering of the samples in the training phase
{Opt3} introduces an hybrid framework to work efficiently on Riemannian Manifolds
{Opt4} exploits the more powerful polynomial regressors as weak learner
{Opt5} can manage partial occlusions of peds
17
Performance evaluation on INRIA Pedestrian Dataset
18
1774 peds (doubled by mirroring) 1671 person-free 64x128 pixel window
Detection Error Tradeoff (DET) curve
Qualitative results for the occluded pedestrian detection
20
Occluded estimated area
Occlusion-free area
81% of accuracy on a subset of 200 examples the ETHZ Pedestrian dataset
Computational considerations
• The main computational cost is due to SVD factorization needed for the projection of the covariance matrices
• A sliding window over the whole image is used for testing. • Our C++ implementation:
– Processing time on a 320x240 pix • image: 0.1 s • sampling step: 3 pixel • number of scales considered: 1 (original scale)
– Processing time on a 320x240 • image: 0.8 s • sampling step: 3 pixel • number of scales considered: 3
21
Conclusions
• We have proposed a set of improvements for the original pedestrian detection algorithm of Tuzel et al.
• Three improvements regards the weak classifiers and the cascade levels, one improvement regards the management of the Riemannian Manifold space
• We also proposed a method to cope with partially occluded pedestrians
• Results show that we improve the original algorithm in terms of DET curve, while the computational cost remains of the same order of complexity
• Future work regards the embedding of the occlusion detection in the training phase and an extension of this framework to the multi-class object detection problem.
22