feature detection. description localization more points robust to occlusion works with less texture...

Feature Detection

Feature Detection

Description

Localization

More PointsRobust to occlusionWorks with less texture

More RepeatableRobust detection

Precise localization

More RobustDeal with expected variationsMaximize correct matches

More SelectiveMinimize wrong matches

Trade‐offs

small big

small big

Harris Corner Detection

Flat Edge Corner

no change in

all direction

no change along

the edge direction

large change

Concept

Shifting the window in any direction should yield a large change in appearance

C. Harris and M. Stephens (1988). "A combined corner and edge detector". Proceedings of the 4th Alvey Vision Conference. pp. 147–151.

Harris Corner Detection

2

2,

( , ) x x y

x y x y y

I I IM w x y

I I I

2,

( , ) ( , ) ( , ) ( , )x y

E u v w x y I x u y v I x y

IntensityShiftedintensity

Windowfunction

( , ) ( , ) , ,u vI x u y v I x y I x y u I x y v

By Taylor expansion

2

,

( , ) ( , ) , ,u vx y

E u v w x y I x y u I x y v

( , ) ,u

E u v u v Mv

일반적으로 x, y 는 window 의 center

Window-averaged change of intensity for the shift [u,v]:

This produces

 written in matrix form

where,

Harris Corner Detection

( , ) ,u

E u v u v Mv

1, 2 are eigenvalues of M. then  the following inferences can be made

1. 2 >> 1 or 1 >> 2 : “Edge”2. 1 and 2 are large, 1 ~ 2 : “Corner”3. 1 and 2 are small : “Flat”

2det traceR M k M

1 2

1 2

det

trace

M

M

(k – empirical constant, k = 0.04-0.06)

Eigenvalue analysis

Since the exact computation of the eigenvalues is computationally expensive, the following function is suggested

Harris Corner Detection

• R depends only on eigenvalues of M

• R is large for a corner

• R is negative with large magnitude for an edge

• |R| is small for a flat region

2det traceR M k M

1 2

1 2

det

trace

M

M

(k – empirical constant, k = 0.04-0.06)

Corner

Edge

Edge

Flat

0R

0R

0R

R is small

2

1

Harris Corner Detection

Examples

SIFT

Descriptor1. Orientation assignment2. Keypoint descriptor

Detector1. Scale-space extrema detection2. Keypoint localization and filtering

Scale-invariant feature transform

Lowe, David G. (1999). "Object recognition from local scale-invariant features". Proceedings of the International Conference on Computer Vision. 2. pp. 1150–1157.

- Choosing features that are invariant to image scaling and rotation

SIFT

Convolve withGaussian

Downsample

# of scales/octave => empirically

Find extremain 3D DoG space

Scale-space extrema detection

SIFT

Constructscale-space

Takedifferences

SIFT

• Compare a pixel with its 26 neighbors in 3*3 regions at the current and ad-jacent scales

• Identify Min and Max

Scale-space extrema detection

SIFT

Sub-pixel Localization

Fit Trivariate quadratic to find sub-pixel extrema

Taylor Series Expansion

Differentiate and set to 0 to get location

SIFT

There are still a lot of points, some of them are not good enough.

Filter Edge and Low Contrast

SIFT

Reject points with bad contrast DOG smaller than 0.03 (image values in [0, 1])

Filter Edge and Low Contrast

SIFT

Reject points with strong edge response in one direction only To check if ratio of principal curvature is below some threshold, r

Filter Edge and Low Contrast

SIFT

( 1, ) ( 1, )

( , 1) ( , 1)

L x y L x yGradientVector

L x y L x y

A histogram is formed by quantizing the orientations into 36 bins;• Compute the orientation histogram within a region around the keypoint (16 16)Ⅹ• Compute gradient magnitude and orientation using finite differences

Orientation assignment

SIFT

Peaks in the histogram correspond to the orientations of the patch; - for all peaks with value >= 0.8 max bin

Orientation assignment

SIFT

Keypoint descriptor• Rotate the gradients and coordinates by the previously computer

orientation• Thresholded image gradients are sampled over 16x16 array of lo-

cations in scale space• Create array of orientation histograms• 8 bins x 4 x 4 histogram array = 128 dimensions

SIFT

Structure from Motion

FeaturePoints

Detection

FeaturePoints

Matching

RelatingImage

Reconstruction

CameraCalibration

Dense Matching

BundleAdjustment

Feature points

Fundamental matrix

Camera matrixSparse reconstructed pointCalibration matrix

Correspondence point sets

Reconstructed point

3D model

Structure from Motion

Flow