local features - khu.ac.krcvlab.khu.ac.kr/cvg10.pdf · matching local self-similarities across...
TRANSCRIPT
Local Features
Correspondence Problem & Keypoint
• Correspondence: matching points, patches, edges, or regions across images
≈
Keypoint Matching
K. Grauman, B. Leibe
Af Bf
A1
A2 A3
Tffd BA ),(
DoG – Efficient Computation
• Computation in Gaussian scale pyramid
K. Grauman, B. Leibe
s
Original image 4
1
2s
Sampling with
step s4 =2
s
s
s
Find local maxima in position-scale space of Difference-of-Gaussian
K. Grauman, B. Leibe
)()( ss yyxx LL
s
s2
s3
s4
s5
List of (x, y, s)
Results: Difference-of-Gaussian
K. Grauman, B. Leibe
Challenges in Keypoints
• Scale change
• Rotation
• Occlusion
• Illumination
……
SIFT
• Scale-space extrema detection
• Keypoint localization
• Orientation assignment
• Keypoint descriptor
Scale-space extrema detection
• Find the points, whose surrounding patches (with some scale) are distinctive
• An approximation to the scale-normalized Laplacian of Gaussian
Maxima and minima in a
3*3*3 neighborhood
Eliminating edge points
Orientation assignment
• Assign an orientation to each keypoint, the keypoint descriptor can be represented relative to this orientation and therefore achieve invariance to image rotation
• Compute magnitude and orientation on the Gaussian smoothed images
Orientation assignment
• A histogram is formed by quantizing the orientations into 36 bins;
• Peaks in the histogram correspond to the orientations of the patch;
• For the same scale and location, there could be multiple keypoints with different orientations;
T. Tuytelaars, B. Leibe
Orientation normalization
• Compute orientation histogram
• Select dominant orientation
• Normalize: rotate to fixed orientation
0 2 p
SIFT vector formation • Computed on rotated and scaled version of window
according to computed orientation & scale
– resample the window
• Based on gradients weighted by a Gaussian of variance half the window (for smooth falloff)
SIFT vector formation • 4x4 array of gradient orientation histogram weighted by
magnitude
• 8 orientations x 4x4 array = 128 dimensions
• Motivation: some sensitivity to spatial layout, but not too much.
• Gaussian weight
showing only 2x2 here but is 4x4
A result
Maximally Stable Extremal Regions [Matas ‘02]
• Select regions that stay stable over a large parameter range
K. Grauman, B. Leibe
Maximally Stable Extremal Regions [Matas ‘02]
• Select regions that stay stable over a large parameter range
K. Grauman, B. Leibe
Maximally Stable Extremal Regions [Matas ‘02]
• Select regions that stay stable over a large parameter range
K. Grauman, B. Leibe
Maximally Stable Extremal Regions [Matas ‘02]
• Select regions that stay stable over a large parameter range
K. Grauman, B. Leibe
Maximally Stable Extremal Regions [Matas ‘02]
• Select regions that stay stable over a large parameter range
K. Grauman, B. Leibe
Maximally Stable Extremal Regions [Matas ‘02]
• Select regions that stay stable over a large parameter range
K. Grauman, B. Leibe
Maximally Stable Extremal Regions [Matas ‘02]
• Select regions that stay stable over a large parameter range
K. Grauman, B. Leibe
Maximally Stable Extremal Regions [Matas ‘02]
• Select regions that stay stable over a large parameter range
K. Grauman, B. Leibe
Maximally Stable Extremal Regions [Matas ‘02]
• Select regions that stay stable over a large parameter range
K. Grauman, B. Leibe
Maximally Stable Extremal Regions [Matas ‘02]
• Select regions that stay stable over a large parameter range
K. Grauman, B. Leibe
Maximally Stable Extremal Regions [Matas ‘02]
• Select regions that stay stable over a large parameter range
K. Grauman, B. Leibe
Maximally Stable Extremal Regions [Matas ‘02]
• Select regions that stay stable over a large parameter range
K. Grauman, B. Leibe
Maximally Stable Extremal Regions [Matas ‘02]
• Select regions that stay stable over a large parameter range
K. Grauman, B. Leibe
Maximally Stable Extremal Regions [Matas ‘02]
• Select regions that stay stable over a large parameter range
K. Grauman, B. Leibe
Maximally Stable Extremal Regions [Matas ‘02]
• Select regions that stay stable over a large parameter range
K. Grauman, B. Leibe
Maximally Stable Extremal Regions [Matas ‘02]
• Select regions that stay stable over a large parameter range
K. Grauman, B. Leibe
Maximally Stable Extremal Regions [Matas ‘02]
• Select regions that stay stable over a large parameter range
K. Grauman, B. Leibe
Example Results: MSER
39 K. Grauman, B. Leibe
Self-similarity Descriptor
Matching Local Self-Similarities across Images and Videos, Shechtman and Irani, 2007
Self-similarity Descriptor
Matching Local Self-Similarities across Images and Videos, Shechtman and Irani, 2007
Self-similarity Descriptor
Matching Local Self-Similarities across Images and Videos, Shechtman and Irani, 2007
Local Descriptors
• Most features can be thought of as templates, histograms (counts), or combinations
• The ideal descriptor should be – Robust
– Distinctive
– Compact
– Efficient
• Most available descriptors focus on edge/gradient information – Capture texture information
– Color rarely used
K. Grauman, B. Leibe
SIFT Repeatability
Lowe IJCV 2004
SIFT Repeatability
Object Recognition
Computer Vision
3D Object Recognition
3D Shape Reconstruction
RGBD Camera based object recognition
3D Model based object recognition
3D Model based object recognition
3D Model based object recognition
Paper list
“Frustum PointNets for 3D Object Detection from RGB-D Data”
Charles R. Qi, Wei Liu, Chenxia Wu, Hao Su, Leonidas J. Guibas
“You Only Look Once: Unified, Real-Time Object Detection”
Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi
“Squeeze-and-Excitation Networks”
Jie Hu, Li Shen, Gang Sun
“Adversarial Complementary Learning forWeakly Supervised Object
Localization”
Xiaolin Zhang, Yunchao Wei, Jiashi Feng, Yi Yang, Thomas Huang