p robing the l ocal -f eature s pace of i nterest p oints wei-ting lee, hwann-tzong chen department...
TRANSCRIPT
![Page 1: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/1.jpg)
PROBING THE LOCAL-FEATURE SPACE OF INTEREST POINTS
Wei-Ting Lee, Hwann-Tzong Chen
Department of Computer ScienceNational Tsing Hua University, Taiwan
ICIP 2010
![Page 2: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/2.jpg)
OUTLINE• Introduction• Approach– Locality-Sensitive
Hashing (LSH) – Sketching the
Feature Space• Experiments– Fast Matching
• Conclusion
![Page 3: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/3.jpg)
INTRODUCTION
Local feature have been extensively used to represent image for various problem
Lots of local feature detector and local feature descriptor have been proposed recent years
![Page 4: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/4.jpg)
Recent History
Maximally Stable Extremal Regions (MSER) [1] BMVC 2002
Difference-of-Gaussian and Scale-Invariant-Feature-Transform (SIFT) [2]
IJCV 2004
Affine invariant detector [3] , [4] IJCV 2004 , TPAMI 2005
Histogram of oriented gradients (HOG) [5]CVPR 2005
‘Visual words’ [6] ‘codebooks’ [7] ICCV 2003 , BMVC 2003
For example
![Page 5: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/5.jpg)
• Present an empirical analysis of the feature space of interest points detected in natural image
• Perform an approximate method for the fast matching between two sets of interest points detected in two images
• Show that the complexity of matching M points to N points can be reduced from O(MN) to O(M+N)
INTRODUCTION
![Page 6: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/6.jpg)
Locality-Sensitive Hashing
• p-stable Distribution:
![Page 7: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/7.jpg)
Locality-Sensitive Hashing based on 2-Stable Distribution
![Page 8: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/8.jpg)
Hash Family
a : random vector sampled from a Gaussian distribution
b : real value chosen uniformly from the range [0 , r]
r : line width
The dot-product a‧v projects each vector to the real line
![Page 9: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/9.jpg)
Building Hash table
![Page 10: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/10.jpg)
Building Hash table
Choose the width r based on the minimum and maximum
=?
θ
a‧b = |a| |b|
Index function
t = 5 , K=3
[5] [5] [5] = 125 = (5-1) * 52 + (5-1) * 51 + 4 * 50 + 1 = 4 * 25 + 20 + 4 + 1 = 125
![Page 11: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/11.jpg)
Sketching the Feature Space
Berkeley segmentation database [14]
Use difference of Gaussian (DOG) [2] & Hessian-affine [3] detector detect about 200,000 interest points
Extract image patches by SIFT descriptor [2]
Create a hash table (L = 1) with five projection(K = 5) and 15 segments on each dot-product real line (t = 15)
The total number of buckets is 155 = 759,375
![Page 12: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/12.jpg)
Entropy = 4.2251(a) DOG
Entropy = 4.0622(b) Hessian-affine
Sketching the Feature SpaceDistribution and Entropy
![Page 13: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/13.jpg)
Collect three image patches of different size 16x16 , 32x32 , 64x64
Each set consist of 200,000 patches.
Natural image patches (from Berkeley segmentation database )
Noise image patches (Randomly-generated noise patches)
Sketching the Feature Space
![Page 14: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/14.jpg)
Distribution and Entropy
![Page 15: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/15.jpg)
Fast Matching
3
3
3 3
3 3
3
3
Referenceimage
RemainingImage (test)
![Page 16: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/16.jpg)
Fast MatchingWe create L = 16 hash tables to probe the 128-dimensional SIFT-feature space
Each table is equipped with five 2-stable Projections , and the projected values are quantized into 15 segments,
i.e., K = 5 and t = 15
For LSH, we use two threshold values of dot-product, θ = 0.95 and θ = 0.97,to determine whether a pair of feature vectors in the same bucket yields a match
LSH is 2 to 15 times faster than matching by exhaustive search
a b = |a| |b| ‧If a = b , then = 1
![Page 17: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/17.jpg)
Fast Matching
DoG detector + SIFT descriptor Hessian-affine detector + SIFT descriptor
![Page 18: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/18.jpg)
DoG detector + SIFT descriptor
2-stable LSH matching vs. exhaustive matching
![Page 19: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/19.jpg)
2-stable LSH matching vs. exhaustive matching
Hessian-affine detector + SIFT descriptor
![Page 20: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/20.jpg)
Conclusion
Using the approximate nearest-neighbor probing scheme derived from 2-stable Locality-Sensitive Hashing
Make use of the efficient representation of the SIFT feature space, and present a fast feature-matching method for finding correspondences between two sets of interest points.
And,Have been used by Whiteorange !!
![Page 21: P ROBING THE L OCAL -F EATURE S PACE OF I NTEREST P OINTS Wei-Ting Lee, Hwann-Tzong Chen Department of Computer Science National Tsing Hua University,](https://reader033.vdocuments.site/reader033/viewer/2022051316/56649f4f5503460f94c70fd0/html5/thumbnails/21.jpg)
THANK YOU SO MUCH