feature detection. description localization more points robust to occlusion works with less texture...
TRANSCRIPT
Feature Detection
Feature Detection
Description
Localization
More PointsRobust to occlusionWorks with less texture
More RepeatableRobust detection
Precise localization
More RobustDeal with expected variationsMaximize correct matches
More SelectiveMinimize wrong matches
Trade‐offs
small big
small big
Harris Corner Detection
Flat Edge Corner
no change in
all direction
no change along
the edge direction
large change
Concept
Shifting the window in any direction should yield a large change in appearance
C. Harris and M. Stephens (1988). "A combined corner and edge detector". Proceedings of the 4th Alvey Vision Conference. pp. 147–151.
Harris Corner Detection
2
2,
( , ) x x y
x y x y y
I I IM w x y
I I I
2,
( , ) ( , ) ( , ) ( , )x y
E u v w x y I x u y v I x y
IntensityShiftedintensity
Windowfunction
( , ) ( , ) , ,u vI x u y v I x y I x y u I x y v
By Taylor expansion
2
,
( , ) ( , ) , ,u vx y
E u v w x y I x y u I x y v
( , ) ,u
E u v u v Mv
일반적으로 x, y 는 window 의 center
Window-averaged change of intensity for the shift [u,v]:
This produces
written in matrix form
where,
Harris Corner Detection
( , ) ,u
E u v u v Mv
1, 2 are eigenvalues of M. then the following inferences can be made
1. 2 >> 1 or 1 >> 2 : “Edge”2. 1 and 2 are large, 1 ~ 2 : “Corner”3. 1 and 2 are small : “Flat”
2det traceR M k M
1 2
1 2
det
trace
M
M
(k – empirical constant, k = 0.04-0.06)
Eigenvalue analysis
Since the exact computation of the eigenvalues is computationally expensive, the following function is suggested
Harris Corner Detection
• R depends only on eigenvalues of M
• R is large for a corner
• R is negative with large magnitude for an edge
• |R| is small for a flat region
2det traceR M k M
1 2
1 2
det
trace
M
M
(k – empirical constant, k = 0.04-0.06)
Corner
Edge
Edge
Flat
0R
0R
0R
R is small
2
1
Harris Corner Detection
Examples
SIFT
Descriptor1. Orientation assignment2. Keypoint descriptor
Detector1. Scale-space extrema detection2. Keypoint localization and filtering
Scale-invariant feature transform
Lowe, David G. (1999). "Object recognition from local scale-invariant features". Proceedings of the International Conference on Computer Vision. 2. pp. 1150–1157.
- Choosing features that are invariant to image scaling and rotation
SIFT
Convolve withGaussian
Downsample
# of scales/octave => empirically
Find extremain 3D DoG space
Scale-space extrema detection
SIFT
Constructscale-space
Takedifferences
SIFT
• Compare a pixel with its 26 neighbors in 3*3 regions at the current and ad-jacent scales
• Identify Min and Max
Scale-space extrema detection
SIFT
Sub-pixel Localization
Fit Trivariate quadratic to find sub-pixel extrema
Taylor Series Expansion
Differentiate and set to 0 to get location
SIFT
There are still a lot of points, some of them are not good enough.
Filter Edge and Low Contrast
SIFT
Reject points with bad contrast DOG smaller than 0.03 (image values in [0, 1])
Filter Edge and Low Contrast
SIFT
Reject points with strong edge response in one direction only To check if ratio of principal curvature is below some threshold, r
Filter Edge and Low Contrast
SIFT
( 1, ) ( 1, )
( , 1) ( , 1)
L x y L x yGradientVector
L x y L x y
A histogram is formed by quantizing the orientations into 36 bins;• Compute the orientation histogram within a region around the keypoint (16 16)Ⅹ• Compute gradient magnitude and orientation using finite differences
Orientation assignment
SIFT
Peaks in the histogram correspond to the orientations of the patch; - for all peaks with value >= 0.8 max bin
Orientation assignment
SIFT
Keypoint descriptor• Rotate the gradients and coordinates by the previously computer
orientation• Thresholded image gradients are sampled over 16x16 array of lo-
cations in scale space• Create array of orientation histograms• 8 bins x 4 x 4 histogram array = 128 dimensions
SIFT
Structure from Motion
FeaturePoints
Detection
FeaturePoints
Matching
RelatingImage
Reconstruction
CameraCalibration
Dense Matching
BundleAdjustment
Feature points
Fundamental matrix
Camera matrixSparse reconstructed pointCalibration matrix
Correspondence point sets
Reconstructed point
3D model
Structure from Motion
Flow