a feature-based kernel for object classification p. moreels - j-y bouguet intel
TRANSCRIPT
A feature-based kernel for object classification
P. Moreels - J-Y BouguetIntel
One scenario for image query : Relevance Feedback
Relevance Relevance FeedbackFeedback
ImageImageDatabaseDatabase
QueryQuery
+ “Baby with house”
SimilaritySimilarityMetricMetric
CandidateCandidateResultsResults Final ResultFinal Result
SVM classifier
???
Outline
• Motivation
• Features and distances
• Starting point: the pyramid match kernel
• Extension : the max-kernel.
• Experiments
• Conclusions
Similarity Metric between pairs of images
• Need to derive a measure of similarity between images
How similar are those two images?
sscore Similarity
Similarity Metric between pairs of images
• Need to derive a measure of similarity between images
How similar are those two images?
sscore Similarity
ss It is desired that:
Image descriptors: features
• Smaller volume of information
• Robustness to deformations (lighting, rotation, affine transformations)
• Detector: difference-of-gaussians
• Descriptor: SIFT = 128D collection of local gradients
Match-based Similarity
IDEA: First establish correspondence between the two sets of points and then compute a “distance” metric based on the matching result
STEP 1: Establish CorrespondenceSTEP 1: Establish Correspondence
2
2
2
Matches),(
2d
d
jiji
es
pp
STEP 2: Compute similaritySTEP 2: Compute similarity
PROBLEM: Until now, no matching-based image similarity metric has been shown to satisfy the MERCER conditions
Our main contribution
• Derive a new image similarity metric that is based on point correspondence and satisfies the Mercer condition
• Methodoly: generalize another metric developed by Grauman at MIT (pyramid kernel) while preserving its Mercer quality
Pyramid match – K.Grauman (ICCV05)
• Only appearance is considered
• Image represented in terms of multi-scale histograms
Appearance
space
Matching process
• Soft matches by histogram intersection• Fine resolution to coarse resolution• More weight at fine resolution: 2-level =1/size(bin)
Level 0
Level 1
Level 2
Final score (Kernel)
count intersection at current level
discards matches alreadycounted at previous levels
- Fine resolution first- Coarse resolution last
• This kernel verifies Mercer condition ! (Odone et all, TIP, 2005)
more weight givento best matches
Issues
should be matchedat this level
level 0
level 1
level 2
level 3
counted only here
• Boundary problems
x 2
x 2
x 2
Issues
level n
• 2level=size(bin) approximates poorly the distance between 2 points
• Weight function f(d)= 1/d over-emphasizes small distances
w = weight = 1/size(bin)
c = correct weight = 1/d
000
...
00
00
(w-c)/c
From discrete to continous
2k is a poor approximation increase the number of
resolution steps
Boundary problems
use translation of bins
level 0
level 1
level 2
level 3
• Verifies Mercer condition
(1+)
1
0
2
3
STEP 1: Establish CorrespondenceSTEP 1: Establish CorrespondenceOur kernel
• This kernel is easy to compute• Uses exact distances• No over-emphasis of low distances• still verifies Mercer condition
matches SORTED),(
21 )],([
1)I,K(I
ji jidf
best matches first
6
18.01][
d
dfwhere
Experiments – distance accuracy
• Random sets of 2D points• Compares distances based on: our kernel, pyramid match,
Earth Mover’s Distance (EMD = optimal solution to the matching problem, based on simplex)
Distances measured between simulated imagesfor pyramid match, max-kernel and EMD
Corresponding probability density function
Caltech database – 101 categories
Some data
• ~50 to ~300 images per class
• Performance of the competitors:– Chance : 1%– Fei-Fei & Perona : 16%– Berg & Malik : 48%– Holub & Perona : 40%– Grauman & Darrell : 43%
• Classification using a SVM
Classification results
• category vs. bg:
performance = 89%
• 10 random categories
performance: 61%
Classification results
• 7 good, 7 bad categories• Performance: 45%
Conclusions
• The MaxKernel is more accurate than pyramid match, more practical than EMD
• Good approximation of the optimal distance
• Verifies Mercer SVM classification OK
• Initial classification performance in same ballpark as the competition
• TODO: add some geometry – e.g. Hough transform to filter out wrong correspondences