multi-attribute spaces: calibration for attribute fusion and similarity search university of oxford...
TRANSCRIPT
Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search
University of Oxford 5th December 2012
Walter Scheirer, Neeraj Kumar, Peter N. Belhumeur, Terrance E. Boult,CVPR 2012
Attributes based image description
4-Legged
Orange
Striped
Furry
White
Symmetric
Ionic columns
Classical
Male
Asian
Beard
Smiling
Slide Courtesy: Neeraj Kumar
Attribute Classifiers
Attribute and Simile Classifiers for Face VerificationN. Kumar, A. C. Berg, P. N. Belhumeur, and S. K. Nayar ICCV 2009
FaceTracer: A Search Engine for Large Collections of Images with Faces N. Kumar, P. N. Belhumeur, and S. K. Nayar ICCV 2009
Attributes Fusion
FaceTracer: “smiling asian men with glasses”Slide Courtesy: Neeraj Kumar
Score Normalization: Problem
• Necessary to prevent high confidence for one attribute from dominating the results.
• Ideal normalization technique should,1) Normalize scores to a uniform range say, [0,1]2) Assign perceptual quality to the scores.
• Positive and negative distributions of different classifiers do not necessarily follow same distribution.
• Fitting a Gaussian or any other distribution to scores satisfies condition 1 but doesn’t satisfy condition 2.
Negative Scores Distributions Positive Scores Distributions
Score Normalization: Solution
• Model distance between positive scores and the negative scores .
• If we knew distribution of negative scores, we could do a hypothesis test for each positive score using that distribution.
• Unfortunately, we don’t know anything about overall negative distribution.
But, we know something about tail of the negative score distribution.
Extreme Value Theory
• Central Limit Theorem:
• The “mean” of a sufficiently large iid random variables will be distributed according to Normal distribution
• Extreme Value Theory:
• The maximum of a sufficiently large iid random variable will be distributed according to Gumbell, Frechet or Weibull distribution.
• If the values are bounded from above and below, the the values are distributed according to “Weibull” distribution.
Weibull Distribution
• Weibull Distribution
CDF
k and λ are shape and location parameters respectively.
PDF CDF
Extreme Value Theory: Application
Tail
Overall Negative Score Distribution
Maximum values of random variables
• Tail of negative scores can be seen as a collection of maxima of some random variables. • Hence it follows Weibull distribution according to Extreme Value Theory.
W-score normalization: Procedure
For any classifier,
• Fix the decision boundary on the scores(Ideally this should be at score = 0 )
• Select maximum N (tail size) samples from negative side of the boundary.
• Fit a Weibull Distribution to these tail scores.
• Renormalize scores using Cumulative Density Function (CDF) of this Weibull distribution.
Results: Dataset
• “Labeled Faces In The Wild” dataset.
• About 13,000 images of 5000 celebrities.
• 75 different attribute classification scores available from “Attribute and Simile Classifiers for Face Verification”. Kumar et al. ICCV 09.
Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments.
Results
Multi Attribute Fusion:
• Joint score can be computed as multiplication of individual attribute probabilities.
• Attributes may not be independent.
• Low probability due to:• bad classifier • absence of images belonging to an attribute.
• Instead of product, authors propose use l1 norm of probabilities as a fusion score.
Results
Similarity Search:
• Given an image and a set of attributes, find nearest images.
• Perceived difference between images in different ranges might be similar.
• Distances between query attribute and its nearest neighbor needs to be normalized.
• Normalize query attribute scores on query image.• Get nearest neighbor distances.• Fit Weibull distribution to distances.
Summary
• Provides way of normalizing scores intuitively.
• Provides way for combining attributes.
• Relies on finding the right threshold and tail size. Requires fair bit of tuning.
Questions?