overvie · descriptors • gaussian derivative-based descriptors – differential invariants (...
TRANSCRIPT
![Page 1: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/1.jpg)
Overview
• Introduction to local features
• Harris interest points + SSD, ZNCC, SIFT
• Scale & affine invariant interest point detectors• Scale & affine invariant interest point detectors
• Evaluation and comparison of different detectors
• Region descriptors and their performance
![Page 2: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/2.jpg)
Scale invariance - motivation
• Description regions have to be adapted to scale changes
• Interest points have to be repeatable for scale changes
![Page 3: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/3.jpg)
Harris detector + scale changes
|)||,max(|
|})),((|),{(|)(
ii
iiii HdistR
bababa εε <=
Repeatability rate
![Page 4: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/4.jpg)
Scale adaptation
=
=
1
1
2
2
2
2
1
1
1 sy
sxI
y
xI
y
xI
Scale change between two images
Scale adapted derivative calculation
![Page 5: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/5.jpg)
Scale adaptation
=
=
1
1
2
2
2
2
1
1
1 sy
sxI
y
xI
y
xI
Scale change between two images
)()(11
2
2
2
1
1
1 σσ sGy
xIsG
y
xI
nn ii
n
ii KK
⊗
=⊗
Scale adapted derivative calculation
σsns
![Page 6: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/6.jpg)
Scale adaptation
)(σiLwhere are the derivatives with Gaussian convolution
⊗
)()(
)()()~( 2
2
σσσσ
σyyx
yxx
LLL
LLLG
![Page 7: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/7.jpg)
Scale adaptation
)(σiLwhere are the derivatives with Gaussian convolution
⊗
)()(
)()()~( 2
2
σσσσ
σyyx
yxx
LLL
LLLG
⊗
)()(
)()()~( 2
22
σσσσ
σsLsLL
sLLsLsGs
yyx
yxx
Scale adapted auto-correlation matrix
![Page 8: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/8.jpg)
Harris detector – adaptation to scale
})),((|),{()( εε <= iiii HdistR baba
![Page 9: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/9.jpg)
Multi-scale matching algorithm
1=s
3=s
5=s
![Page 10: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/10.jpg)
Multi-scale matching algorithm
1=s
8 matches
![Page 11: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/11.jpg)
Multi-scale matching algorithm
1=s
3 matches
Robust estimation of a global affine transformation
![Page 12: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/12.jpg)
Multi-scale matching algorithm
1=s
3 matches
3=s
4 matches
![Page 13: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/13.jpg)
Multi-scale matching algorithm
1=s
3 matches
3=s
5=s
4 matches
16 matchescorrect scale
highest number of matches
![Page 14: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/14.jpg)
Matching results
Scale change of 5.7
![Page 15: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/15.jpg)
Matching results
100% correct matches (13 matches)
![Page 16: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/16.jpg)
Scale selection• We want to find the characteristic scale of the blob by
convolving it with Laplacians at several scales and looking for the maximum response
• However, Laplacian response decays as scale increases:
Why does this happen?
increasing σoriginal signal(radius=8)
![Page 17: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/17.jpg)
Scale normalization
• The response of a derivative of Gaussian filter to a perfect step edge decreases as σ increases
1
πσ 2
1
![Page 18: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/18.jpg)
Scale normalization
• The response of a derivative of Gaussian filter to a perfect step edge decreases as σ increases
• To keep response the same (scale-invariant), must multiply Gaussian derivative by σ
• Laplacian is the second Gaussian derivative, so it must be • Laplacian is the second Gaussian derivative, so it must be multiplied by σ2
![Page 19: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/19.jpg)
Effect of scale normalizationUnnormalized Laplacian responseOriginal signal
Scale-normalized Laplacian response
maximum
![Page 20: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/20.jpg)
Blob detection in 2D
• Laplacian of Gaussian: Circularly symmetric operator for blob detection in 2D
2
2
2
22
y
g
x
gg
∂∂+
∂∂=∇
![Page 21: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/21.jpg)
Blob detection in 2D
• Laplacian of Gaussian: Circularly symmetric operator for blob detection in 2D
∂∂+
∂∂=∇
2
2
2
222
norm y
g
x
gg σScale-normalized:
![Page 22: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/22.jpg)
Scale selection
• The 2D Laplacian is given by
• For a binary circle of radius r, the Laplacian achieves a
222 2/)(222 )2( σσ yxeyx +−−+ (up to scale)
• For a binary circle of radius r, the Laplacian achieves a maximum at 2/r=σ
r
2/rimage
La
pla
cia
n re
spon
se
scale (σ)
![Page 23: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/23.jpg)
Characteristic scale
• We define the characteristic scale as the scale that produces peak of Laplacian response
characteristic scale
T. Lindeberg (1998). Feature detection with automatic scale selection. International Journal of Computer Vision 30 (2): pp 77--116.
![Page 24: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/24.jpg)
Scale selection
• For a point compute a value (gradient, Laplacian etc.) at several scales
• Normalization of the values with the scale factor
e.g. Laplacian |)(| 2
yyxx LLs +
• Select scale at the maximum → characteristic scale
• Exp. results show that the Laplacian gives best results
|)(| 2
yyxx LLs +
∗s
scale
![Page 25: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/25.jpg)
Scale selection
• Scale invariance of the characteristic scale
s
norm
. Lap
.
scale
![Page 26: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/26.jpg)
Scale selection
• Scale invariance of the characteristic scale
s
∗∗ =⋅ 21 sss
norm
. Lap
.
norm
. Lap
.
• Relation between characteristic scales
scale scale
![Page 27: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/27.jpg)
Scale-invariant detectors
• Harris-Laplace (Mikolajczyk & Schmid’01)
• Laplacian detector (Lindeberg’98)
• Difference of Gaussian (Lowe’99)
Harris-Laplace Laplacian
![Page 28: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/28.jpg)
Harris-Laplace
multi-scale Harris points
invariant points + associated regions [Mikolajczyk & Schmid’01]
selection of points at
maximum of Laplacian
![Page 29: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/29.jpg)
Matching results
213 / 190 detected interest points
![Page 30: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/30.jpg)
Matching results
58 points are initially matched
![Page 31: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/31.jpg)
Matching results
32 points are matched after verification – all correct
![Page 32: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/32.jpg)
LOG detector
Convolve image with scale-normalized Laplacian at several scales
))()((2 σσ yyxx GGsLOG +=
Detection of maxima and minima of Laplacian in scale space
![Page 33: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/33.jpg)
Hessian detector
=
yyxy
xyxx
LL
LLxH )(Hessian matrix
2xyyyxx LLLDET −=Determinant of Hessian matrix
Penalizes/eliminates long structures � with small derivative in a single direction
![Page 34: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/34.jpg)
Efficient implementation
• Difference of Gaussian (DOG) approximates the Laplacian )()( σσ GkGDOG −=
• Error due to the approximation
![Page 35: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/35.jpg)
DOG detector
• Fast computation, scale space processed one octave at a time
David G. Lowe. "Distinctive image features from scale-invariant keypoints.”IJCV 60 (2).
![Page 36: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/36.jpg)
Local features - overview
• Scale invariant interest points
• Affine invariant interest points
• Evaluation of interest points
• Descriptors and their evaluation
![Page 37: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/37.jpg)
Affine invariant regions - Motivation
• Scale invariance is not sufficient for large baseline changes
A
detected scale invariant region
A
projected regions, viewpoint changes can locally be approximated by an affine transformation A
![Page 38: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/38.jpg)
Affine invariant regions - Motivation
![Page 39: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/39.jpg)
Affine invariant regions - Example
![Page 40: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/40.jpg)
Harris/Hessian/Laplacian-Affine
• Initialize with scale-invariant Harris/Hessian/Laplacian points
• Estimation of the affine neighbourhood with the second moment matrix [Lindeberg’94]
• Apply affine neighbourhood estimation to the scale-invariant interest points [Mikolajczyk & Schmid’02, Schaffalitzky & Zisserman’02]
• Excellent results in a comparison [Mikolajczyk et al.’05]
![Page 41: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/41.jpg)
Affine invariant regions
• Based on the second moment matrix (Lindeberg’94)
⊗==
),(),(
),(),()(),,( 2
2
2
DyDyx
DyxDx
IDDI LLL
LLLGM
σσσσ
σσσσµxx
xxx
xx 2
1
M=′
• Normalization with eigenvalues/eigenvectors
![Page 42: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/42.jpg)
Affine invariant regions
LR xx A=
′=′LR Rxx
Isotropic neighborhoods related by image rotation
L2
1
L xx LM=′R
2
1
R xx RM=′
![Page 43: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/43.jpg)
• Iterative estimation – initial points
Affine invariant regions - Estimation
![Page 44: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/44.jpg)
• Iterative estimation – iteration #1
Affine invariant regions - Estimation
![Page 45: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/45.jpg)
• Iterative estimation – iteration #2
Affine invariant regions - Estimation
![Page 46: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/46.jpg)
• Iterative estimation – iteration #3, #4
Affine invariant regions - Estimation
![Page 47: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/47.jpg)
Harris-Affine versus Harris-Laplace
Harris-LaplaceHarris-Affine
![Page 48: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/48.jpg)
Harris-Affine
Harris/Hessian-Affine
Hessian-Affine
![Page 49: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/49.jpg)
Harris-Affine
![Page 50: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/50.jpg)
Hessian-Affine
![Page 51: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/51.jpg)
Matches
22 correct matches
![Page 52: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/52.jpg)
Matches
33 correct matches
![Page 53: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/53.jpg)
Maximally stable extremal regions (MSER) [Matas’02]
• Extremal regions: connected components in a thresholded image (all pixels above/below a threshold)
• Maximally stable: minimal change of the component (area) for a change of the threshold, i.e. region remains (area) for a change of the threshold, i.e. region remains stable for a change of threshold
• Excellent results in a recent comparison
![Page 54: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/54.jpg)
Maximally stable extremal regions (MSER)
Examples of thresholded images
high threshold
low threshold
![Page 55: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/55.jpg)
MSER
![Page 56: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/56.jpg)
Overview
• Introduction to local features
• Harris interest points + SSD, ZNCC, SIFT
• Scale & affine invariant interest point detectors• Scale & affine invariant interest point detectors
• Evaluation and comparison of different detectors
• Region descriptors and their performance
![Page 57: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/57.jpg)
Evaluation of interest points
• Quantitative evaluation of interest point/region detectors– points / regions at the same relative location and area
• Repeatability rate : percentage of corresponding points
• Two points/regions are corresponding if– location error small– area intersection large
• [K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir & L. Van Gool ’05]
![Page 58: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/58.jpg)
Evaluation criterion
H
%100#
# ⋅=regionsdetected
regionsingcorrespondityrepeatabil
![Page 59: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/59.jpg)
Evaluation criterion
H
%100#
# ⋅=regionsdetected
regionsingcorrespondityrepeatabil
%100)1( ⋅−=union
onintersectierroroverlap
2% 10% 20% 30% 40% 50% 60%
![Page 60: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/60.jpg)
Dataset
• Different types of transformation– Viewpoint change– Scale change– Image blur– JPEG compression– Light change– Light change
• Two scene types– Structured– Textured
• Transformations within the sequence (homographies)– Independent estimation
![Page 61: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/61.jpg)
Viewpoint change (0-60 degrees )
structured scene
textured scene
![Page 62: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/62.jpg)
Zoom + rotation (zoom of 1-4)
structured scene
textured scene
![Page 63: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/63.jpg)
Blur, compression, illumination
blur - structured scene blur - textured scene
light change - structured scene jpeg compression - structured scene
![Page 64: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/64.jpg)
Comparison of affine invariant detectors
60
70
80
90
100
repe
atab
ility
%
Harris−Affine
Hessian−Affine
MSER
IBR
EBR
Salient
800
1000
1200
1400
num
ber
of c
orre
spon
denc
es
Harris−Affine
Hessian−Affine
MSER
IBR
EBR
Salient
Viewpoint change - structured scenerepeatability % # correspondences
15 20 25 30 35 40 45 50 55 60 650
10
20
30
40
50
viewpoint angle
repe
atab
ility
%
15 20 25 30 35 40 45 50 55 60 650
200
400
600
800
viewpoint anglenu
mbe
r of
cor
resp
onde
nces
reference image 20 6040
![Page 65: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/65.jpg)
Scale changerepeatability % repeatability %
Comparison of affine invariant detectors
reference image 4reference image 2.8
![Page 66: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/66.jpg)
• Good performance for large viewpoint and scale changes
• Results depend on transformation and scene type, no one best detector
Conclusion - detectors
• Detectors are complementary– MSER adapted to structured scenes– Harris and Hessian adapted to textured scenes
• Performance of the different scale invariant detectors is very similar (Harris-Laplace, Hessian, LoG and DOG)
• Scale-invariant detector sufficient up to 40 degrees of viewpoint change
![Page 67: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/67.jpg)
Overview
• Introduction to local features
• Harris interest points + SSD, ZNCC, SIFT
• Scale & affine invariant interest point detectors• Scale & affine invariant interest point detectors
• Evaluation and comparison of different detectors
• Region descriptors and their performance
![Page 68: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/68.jpg)
Region descriptors
• Normalized regions are– invariant to geometric transformations except rotation– not invariant to photometric transformations
![Page 69: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/69.jpg)
Descriptors
• Regions invariant to geometric transformations except rotation– rotation invariant descriptors– normalization with dominant gradient direction– normalization with dominant gradient direction
• Regions not invariant to photometric transformations– invariance to affine photometric transformations– normalization with mean and standard deviation of the image patch
![Page 70: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/70.jpg)
Descriptors
Extract affine regions Normalize regionsEliminate rotational
+ illumination
Compute appearancedescriptors
SIFT (Lowe ’04)
![Page 71: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/71.jpg)
Descriptors
• Gaussian derivative-based descriptors– Differential invariants (Koenderink and van Doorn’87)– Steerable filters (Freeman and Adelson’91)
• SIFT (Lowe’99)• SIFT (Lowe’99)• Moment invariants [Van Gool et al.’96]
• Shape context [Belongie et al.’02]
• SIFT with PCA dimensionality reduction• Gradient PCA [Ke and Sukthankar’04]
• SURF descriptor [Bay et al.’08]
• DAISY descriptor [Tola et al.’08, Windler et al’09]
![Page 72: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/72.jpg)
Comparison criterion
• Descriptors should be– Distinctive– Robust to changes on viewing conditions as well as to errors of
the detector
• Detection rate (recall) 1• Detection rate (recall)– #correct matches / #correspondences
• False positive rate– #false matches / #all matches
• Variation of the distance threshold – distance (d1, d2) < threshold
1
1
[K. Mikolajczyk & C. Schmid, PAMI’05]
![Page 73: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/73.jpg)
Viewpoint change (60 degrees)
0.6
0.7
0.8
0.9
1
#cor
rect
/ 21
01
esift* *shape context
gradient pcacross correlation
complex filtershar−aff esift
steerable filters
gradient moments
sift
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
1−precision
#cor
rect
/ 21
01
![Page 74: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/74.jpg)
esift* *
Scale change (factor 2.8)
0.6
0.7
0.8
0.9
1
#cor
rect
/ 20
86
shape context
gradient pcacross correlation
complex filtershar−aff esift
steerable filters
gradient moments
sift
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
1−precision
#cor
rect
/ 20
86
![Page 75: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/75.jpg)
Conclusion - descriptors
• SIFT based descriptors perform best
• Significant difference between SIFT and low dimension descriptors as well as cross-correlation
• Robust region descriptors better than point-wise descriptors
• Performance of the descriptor is relatively independent of the detector
![Page 76: Overvie · Descriptors • Gaussian derivative-based descriptors – Differential invariants ( Koenderink and van Doorn’87 ) – Steerable filters ( Freeman and Adelson’91 ) •](https://reader034.vdocuments.site/reader034/viewer/2022050516/5fa020465b4ac6207917146f/html5/thumbnails/76.jpg)
Available on the internet
• Binaries for detectors and descriptors– Building blocks for recognition systems
http://lear.inrialpes.fr/software
• Carefully designed test setup– Dataset with transformations– Evaluation code in matlab– Benchmark for new detectors and descriptors