lecture 5&6 –finding features, affine invariance,...
TRANSCRIPT
![Page 1: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/1.jpg)
CAP5415-ComputerVisionLecture5and6-FindingFeatures,
AffineInvariance,SIFT
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
1
![Page 2: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/2.jpg)
Outline
• ConceptofScale– Pyramids– Scale-spaceapproachesbriefly
• Scaleinvariantregionselection• SIFT:animageregiondescriptor
• DavidLowe,IJCV2004paper[Pleasereadit!]
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
2
![Page 3: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/3.jpg)
Reminder:Motivation
• ImageMatching– Fundamentalaspectofmanyproblems
• ObjectRecognition• 3DStructures• StereoCorrespondence• MotionTracking• ….
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
3
![Page 4: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/4.jpg)
Reminder:Motivation
• ImageMatching– Fundamentalaspectofmanyproblems
• ObjectRecognition• 3DStructures• StereoCorrespondence• MotionTracking• ….
Whatarethedesiredfeaturestoconductthesetasks?
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
4
![Page 5: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/5.jpg)
Review oftheCornerDetectionLecture5&6–FindingFeatures,AffineInvariance,SIFT
55
rotation
translation
scaling
Cornersareinvariantto
✔
✔
✖
![Page 6: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/6.jpg)
SCALE
• Theextrema inasignalanditsfirstafewderivatives provideausefulgeneralpurposedescriptionformanykindsofsignals– Edges– Corners
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
66
![Page 7: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/7.jpg)
SCALE
• Theextrema inasignalanditsfirstafewderivatives provideausefulgeneralpurposedescriptionformanykindsofsignals– Edges– Corners
• Inphysics,objectsliveonarangeofscales.
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
77
![Page 8: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/8.jpg)
SCALE
• Theextrema inasignalanditsfirstafewderivatives provideausefulgeneralpurposedescriptionformanykindsofsignals– Edges– Corners
• Inphysics,objectsliveonarangeofscales.• Neighborhoodoperationscanonlyextractlocalfeatures(atascaleofatmostafewpixels),however,imagescontaininformationatlargerscales
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
88
![Page 9: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/9.jpg)
Scale-Space
• Anygivenimagecancontainobjectsthatexistatscalesdifferentfromotherobjectsinthesameimage
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
9
![Page 10: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/10.jpg)
Scale-Space
• Anygivenimagecancontainobjectsthatexistatscalesdifferentfromotherobjectsinthesameimage
• Or,thatevenexistatmultiplescalessimultaneously
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
10
![Page 11: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/11.jpg)
Scale-Space
• Anygivenimagecancontainobjectsthatexistatscalesdifferentfromotherobjectsinthesameimage
• Or,thatevenexistatmultiplescalessimultaneously
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
11
![Page 12: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/12.jpg)
Multi-scaleRepresentationLecture5&6–FindingFeatures,AffineInvariance,SIFT
12
SmallScaleFeatures
SmallerFilterMasks
LargeScaleFeatures
LargerFilterMasks
Image
![Page 13: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/13.jpg)
Multi-scaleRepresentationLecture5&6–FindingFeatures,AffineInvariance,SIFT
13
SmallScaleFeatures
SmallerFilterMasks
LargeScaleFeatures
LargerFilterMasks
Image
ComputationalCostincreases!
Doublingthescaleleadstofour-foldincreaseinthenumberofoperationsin2D.
![Page 14: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/14.jpg)
Multi-ScaleRepresentation:Pyramid
• Pyramid isonewaytorepresentimagesinmulti-scale– Pyramidisbuiltbyusingmultiplecopiesofimage.– Eachlevelinthepyramidis1/4ofthesizeofpreviouslevel.
– Thelowestlevelisofthehighestresolution.– Thehighestlevelisofthelowestresolution.
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
14
![Page 15: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/15.jpg)
Multi-ScaleRepresentation:PyramidLecture5&6–FindingFeatures,AffineInvariance,SIFT
15
![Page 16: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/16.jpg)
Pyramidcancaptureglobalandlocalfeatures
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
16
![Page 17: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/17.jpg)
Gaussian PyramidsLecture5&6–FindingFeatures,AffineInvariance,SIFT
17Source: Forsyth
![Page 18: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/18.jpg)
Laplacian PyramidsLecture5&6–FindingFeatures,AffineInvariance,SIFT
18Source: Forsyth
![Page 19: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/19.jpg)
Laplacian PyramidsLecture5&6–FindingFeatures,AffineInvariance,SIFT
19
1
1
-2 1 1-2
Laplacian ofGaussian(LoG):GaussianfirsttosmoothimagesthenperformLaplacian operation
52(G� ⇤ I) = I ⇤ 52G�
![Page 20: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/20.jpg)
Laplacian PyramidsLecture5&6–FindingFeatures,AffineInvariance,SIFT
20
![Page 21: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/21.jpg)
Laplacian PyramidsLecture5&6–FindingFeatures,AffineInvariance,SIFT
21
�G
�
�x
(x, y) = � x
2⇡�4e
�(x2+y
2)/(2�2)
![Page 22: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/22.jpg)
Laplacian PyramidsLecture5&6–FindingFeatures,AffineInvariance,SIFT
22
�G
�
�x
(x, y) = � x
2⇡�4e
�(x2+y
2)/(2�2)
52G
�
(x, y) =1
2⇡�4
(x2 + y
2 � 2�2)
�
2e
�(x2+y
2)/(2�2)
Repeatsamederivativeforycomponent,andthentakesecondderivatives.
![Page 23: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/23.jpg)
Laplacian PyramidConstructionLecture5&6–FindingFeatures,AffineInvariance,SIFT
23
![Page 24: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/24.jpg)
Laplacian PyramidConstructionLecture5&6–FindingFeatures,AffineInvariance,SIFT
24
![Page 25: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/25.jpg)
Decimation andInterpolationLecture5&6–FindingFeatures,AffineInvariance,SIFT
25
LowPass filter(i.e.,Gaussian)
![Page 26: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/26.jpg)
Decimation andInterpolationLecture5&6–FindingFeatures,AffineInvariance,SIFT
26
LowPass filter(i.e.,Gaussian)
y(n) = x(n) ⇤ h(n) =X
k
h(k)x(n� k)
![Page 27: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/27.jpg)
Decimation andInterpolationLecture5&6–FindingFeatures,AffineInvariance,SIFT
27
LowPass filter(i.e.,Gaussian)
y(n) = x(n) ⇤ h(n) =X
k
h(k)x(n� k)
z(n) = y(2n)
![Page 28: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/28.jpg)
Decimation andInterpolationLecture5&6–FindingFeatures,AffineInvariance,SIFT
28
LowPass filter(i.e.,Gaussian)
y(n) = x(n) ⇤ h(n) =X
k
h(k)x(n� k)
z(n) = y(2n)
z(n) =X
k
h(k)x(2n� k)
![Page 29: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/29.jpg)
DifferenceofGaussian(DoG)
• ItisacommonapproximationofLoG (betterruntime).
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
29
![Page 30: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/30.jpg)
DifferenceofGaussian(DoG)
• ItisacommonapproximationofLoG (betterruntime).
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
30
D�,↵(x, y) = L(x, y,↵)� L(x, y,↵�)
![Page 31: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/31.jpg)
DifferenceofGaussian(DoG)
• ItisacommonapproximationofLoG (betterruntime).
• ItisthedifferencebetweenablurredcopyofimageIandanevenmoreblurredcopyofI.
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
31
D�,↵(x, y) = L(x, y,↵)� L(x, y,↵�)
![Page 32: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/32.jpg)
DifferenceofGaussian(DoG)
• ItisacommonapproximationofLoG (betterruntime).
• ItisthedifferencebetweenablurredcopyofimageIandanevenmoreblurredcopyofI.
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
32
D�,↵(x, y) = L(x, y,↵)� L(x, y,↵�)
5G�(x, y) ⇡G↵�(x, y)�G�(x, y)
(↵� 1)�2
![Page 33: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/33.jpg)
ScaleSelection-AutomatedLecture5&6–FindingFeatures,AffineInvariance,SIFT
33
• Find scale that gives local maxima of some function f in both position and scale.
f
region size
Image 1f
region size
Image 2
s1 s2K. Grauman,
![Page 34: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/34.jpg)
GoodFunction?Lecture5&6–FindingFeatures,AffineInvariance,SIFT
34
• A “good” function for scale detection:has one stable sharp peak
f
region size
bad
f
region size
bad
f
region size
Good !
• Forusualimages:agoodfunctionwouldbeaonewhichrespondstocontrast(sharplocalintensitychange)
![Page 35: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/35.jpg)
PatchSizeCorrespondingtoScaleLecture5&6–FindingFeatures,AffineInvariance,SIFT
35
)),(( )),((11
ss ¢¢= xIfxIfmm iiii !!
How to find corresponding patch sizes?
![Page 36: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/36.jpg)
PatchSizeCorrespondingtoScaleLecture5&6–FindingFeatures,AffineInvariance,SIFT
36K.Grauman,B.Leibe)),((
1sxIf
mii ! )),((1
sxIfmii ¢!
![Page 37: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/37.jpg)
PatchSizeCorrespondingtoScaleLecture5&6–FindingFeatures,AffineInvariance,SIFT
37K.Grauman,B.Leibe)),((
1sxIf
mii ! )),((1
sxIfmii ¢!
![Page 38: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/38.jpg)
PatchSizeCorrespondingtoScaleLecture5&6–FindingFeatures,AffineInvariance,SIFT
38K.Grauman,B.Leibe)),((
1sxIf
mii ! )),((1
sxIfmii ¢!
![Page 39: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/39.jpg)
PatchSizeCorrespondingtoScaleLecture5&6–FindingFeatures,AffineInvariance,SIFT
39K.Grauman,B.Leibe)),((
1sxIf
mii ! )),((1
sxIfmii ¢!
![Page 40: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/40.jpg)
PatchSizeCorrespondingtoScaleLecture5&6–FindingFeatures,AffineInvariance,SIFT
40K.Grauman,B.Leibe)),((
1sxIf
mii ! )),((1
sxIfmii ¢!
![Page 41: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/41.jpg)
PatchSizeCorrespondingtoScaleLecture5&6–FindingFeatures,AffineInvariance,SIFT
41
• Functionresponsesforincreasingscale(scalesignature)
K.Grauman,B.Leibe)),((
1sxIf
mii ! )),((1
s ¢¢xIfmii !
![Page 42: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/42.jpg)
NormalizationLecture5&6–FindingFeatures,AffineInvariance,SIFT
42
![Page 43: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/43.jpg)
ScaleInvariantFeatureTransform(SIFT)
• Lowe.,D.2004,IJCV
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
43cited>43K
![Page 44: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/44.jpg)
ScaleInvariantFeatureTransform(SIFT)
• Showsexistence,importance,andvalueofinvarianttypesofdetector
• Demonstratestherichnessthatfeaturedescriptorscanbringtofeaturematching
InsteadofusingLoG (Laplacian ofGaussian) likeinHarrisandHessian-basedoperators,SIFTuses
DoG (DifferenceofGaussian).
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
44
![Page 45: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/45.jpg)
ScaleInvariantFeatureTransform(SIFT)
• Image content is transformed into local feature coordinates that are invariant to translation, rotation, scale, and other imaging parameters
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
45
![Page 46: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/46.jpg)
ScaleInvariantFeatureTransform(SIFT)
• Image content is transformed into local feature coordinates that are invariant to translation, rotation, scale, and other imaging parameters
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
46
![Page 47: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/47.jpg)
OverallProcedureataHighLevelLecture5&6–FindingFeatures,AffineInvariance,SIFT
47
Scale-SpaceExtremaDetection
Searchovermultiplescalesandimagelocations
KeyPointLocalization
Fitamodeltodeterminelocationandscale.SelectKeyPoints basedona
measureofstability.
OrientationAssignment
Computebestorientation(s)foreachkeyPoint region.
KeyPointDescription
Uselocalimagegradientsatselectedscaleandrotationtodescribeeach
keyPoint region.
![Page 48: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/48.jpg)
SIFT DescriptorLecture5&6–FindingFeatures,AffineInvariance,SIFT
48
Basic idea:• Take n x n (i.e., n=16) square window (around a feature/interest point)• Divide them into m x m (i.e., m=4) cells• Compute gradient orientation for each cell• Create histogram over edge orientations weighted by magnitude
0 2anglehistogram
![Page 49: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/49.jpg)
SIFT DescriptorLecture5&6–FindingFeatures,AffineInvariance,SIFT
49
16histogramsx8orientations=128features
![Page 50: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/50.jpg)
SIFT DescriptorLecture5&6–FindingFeatures,AffineInvariance,SIFT
50
Full version• Divide the 16x16 window into a 4x4 grid of cells (2x2 case shown below)• Compute an orientation histogram for each cell• 16 cells * 8 orientations = 128 dimensional descriptor• Threshold normalize the descriptor:
suchthat:
0.2
![Page 51: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/51.jpg)
PropertiesofSIFTLecture5&6–FindingFeatures,AffineInvariance,SIFT
51
Extraordinarilyrobustmatchingtechnique
– Canhandlechangesinviewpoint– Canhandlesignificantchangesinillumination– Efficient(realtime)– Lotsofcodeavailable
• http://people.csail.mit.edu/albert/ladypack/wiki/index.php/Known_implementations_of_SIFT
![Page 52: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/52.jpg)
Revisit SIFTStepsLecture5&6–FindingFeatures,AffineInvariance,SIFT
52
(1) Scale-space extrema detection– Extract scale and rotation invariant interest points (i.e.,
keypoints). (2) KeyPoint localization
– Determine location and scale for each interest point.– Eliminate “weak” keypoints
(3) Orientation assignment– Assign one or more orientations to each keypoint.
(4) KeyPoint descriptor– Use local image gradients at the selected scale.
![Page 53: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/53.jpg)
(1)Scale-SpaceExtrema DetectionLecture5&6–FindingFeatures,AffineInvariance,SIFT
53
Wewanttofindpointsthatgiveusinformationabouttheobjectsintheimage.
Theinformationabouttheobjectsisintheobject’sedges.
Wewillrepresenttheimageinawaythatgivesustheseedgesasthisrepresentationsextrema points.
![Page 54: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/54.jpg)
(1)Scale-SpaceExtrema DetectionLecture5&6–FindingFeatures,AffineInvariance,SIFT
54
• Harris-Laplace
Find local maxima of:– Harris detector in space – LoG in scale
• SIFT
Find local maxima of:– Hessian in space – DoG in scale
scale
x
y
Hessian
DoG
scale
x
y
¬ Harris ®
LoG
![Page 55: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/55.jpg)
(1)Scale-SpaceExtrema DetectionLecture5&6–FindingFeatures,AffineInvariance,SIFT
55
![Page 56: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/56.jpg)
(1)Scale-SpaceExtrema DetectionLecture5&6–FindingFeatures,AffineInvariance,SIFT
56
• Extractlocalextrema(i.e.,minimaormaxima)inDoG pyramid.
-Compareeachpointtoits8neighborsatthesamelevel,9neighborsinthelevelabove,and9neighborsinthelevelbelow(i.e.,26total).
( )sD
( )skD
( )s2kD
![Page 57: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/57.jpg)
(1)Scale-SpaceExtrema DetectionLecture5&6–FindingFeatures,AffineInvariance,SIFT
57
• Laplacian of Gaussian kernel– Scale normalized (x by scale2)– Proposed by Lindeberg
• Scale-space detection– Find local maxima across scale/space– A good “blob” detector
![Page 58: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/58.jpg)
(2)KeyPoint Localization
• There are still a lot of points, some of them are not good enough.
• The locations of keypoints may be not accurate.
• Eliminating edge points.
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
58
![Page 59: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/59.jpg)
(2)KeyPoint LocalizationLecture5&6–FindingFeatures,AffineInvariance,SIFT
� Reject (1) pointswithlowcontrast(flat)(2)poorlylocalizedalonganedge(edge)
![Page 60: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/60.jpg)
(2)KeyPoint LocalizationLecture5&6–FindingFeatures,AffineInvariance,SIFT
� Reject (1) pointswithlowcontrast(flat)(2)poorlylocalizedalonganedge(edge)
if
Reject
WhereandpracticallySIFTusesr=10.
Tr(H)
|H| <(r + 1)2
r
r =�1
�2
![Page 61: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/61.jpg)
InaccurateKeyPoint LocalizationLecture5&6–FindingFeatures,AffineInvariance,SIFT
61
xSampling
DetectedExtrema
TrueExtrema
Poorcontrast
![Page 62: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/62.jpg)
InaccurateKeyPoint LocalizationLecture5&6–FindingFeatures,AffineInvariance,SIFT
62
TheSolution:
( ) xxDxx
xDDxD
TT
T !!
!!!
!2
2
21
¶¶
+¶¶
+=
Taylorexpansion:
Minimizetofindaccurateextrema:
xD
xDx !!
¶¶
¶¶
-=-1
2
2
ˆ
Brown&Lowe2002
Ifoffsetfromsamplingpointislargerthan0.5-Keypoint shouldbeinadifferentsamplingpoint.
![Page 63: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/63.jpg)
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
63
Localextremas
![Page 64: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/64.jpg)
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
64
Localextremas Removelowcontrastfeatures
![Page 65: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/65.jpg)
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
65
Localextremas Removelowedges
![Page 66: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/66.jpg)
SIFTDescriptorLecture5&6–FindingFeatures,AffineInvariance,SIFT
66
![Page 67: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/67.jpg)
(3)OrientationAssignmentLecture5&6–FindingFeatures,AffineInvariance,SIFT
67
m(x, y) = (L(x +1, y)− L(x −1, y))2 + (L(x, y +1)− L(x, y −1))2
θ(x, y) = a tan2((L(x, y +1)− L(x, y −1)) / (L(x +1, y)− L(x −1, y)))
36 bins (i.e., 10o per bin)
• Histogramentriesare weighted by(i)gradientmagnitudeand(ii)aGaussianfunctionwithσ equalto1.5timesthescaleofthekeypoint.
0 2pi
L(x, y,σ ) = G(x, y,σ )* I(x, y)
Create histogram of gradient directions, within a region around the keyPoint, at selected scale:
![Page 68: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/68.jpg)
(3)OrientationAssignmentLecture5&6–FindingFeatures,AffineInvariance,SIFT
68
![Page 69: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/69.jpg)
(3)OrientationAssignmentLecture5&6–FindingFeatures,AffineInvariance,SIFT
69
![Page 70: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/70.jpg)
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
70
(3)OrientationAssignment
![Page 71: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/71.jpg)
(3)OrientationAssignmentLecture5&6–FindingFeatures,AffineInvariance,SIFT
71
![Page 72: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/72.jpg)
(3)OrientationAssignmentLecture5&6–FindingFeatures,AffineInvariance,SIFT
72
![Page 73: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/73.jpg)
(4)KeyPoint DescriptorLecture5&6–FindingFeatures,AffineInvariance,SIFT
73
• Partial Voting: distribute histogram entries into adjacent bins (i.e., additional robustness to shifts)– Each entry is added to all bins, multiplied by a weight of 1-d,
where d is the distance from the bin it belongs.
![Page 74: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/74.jpg)
(4)KeyPoint DescriptorLecture5&6–FindingFeatures,AffineInvariance,SIFT
74
• Descriptordependsontwomainparameters:– (1)numberoforientations– nxnarrayoforientationhistograms
![Page 75: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/75.jpg)
EvaluatingResultsLecture5&6–FindingFeatures,AffineInvariance,SIFT
75
0.7
Howcanwemeasuretheperformanceofafeaturematcher?
0 1
1
false positive rate
truepositive
rate# true positives matched
# true positives
0.1# false positives matched
# true negatives
features that really do have a match
the matcher correctlyfound a match
the matcher said yes when the right answer was no
![Page 76: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/76.jpg)
EvaluatingResultsLecture5&6–FindingFeatures,AffineInvariance,SIFT
76
0.7
Howcanwemeasuretheperformanceofafeaturematcher?
0 1
1
false positive rate
truepositive
rate# true positives matched
# true positives
0.1# false positives matched
# true negatives
features that really do have a match
the matcher correctlyfound a match
the matcher said yes when the right answer was no
ROC curve (“Receiver Operator Characteristic”)
![Page 77: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/77.jpg)
ChangeofIlluminationLecture5&6–FindingFeatures,AffineInvariance,SIFT
77
Change of brightness => doesn’t effect gradients (difference of pixels value).
Change of contrast => doesn’t effect gradients (up to normalization).Saturation (non-linear change of illumination) =>affects magnitudes much more than orientation.=> Threshold gradient magnitudes to 0.2 and
renormalize.
![Page 78: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/78.jpg)
Illuminationinvariance?Lecture5&6–FindingFeatures,AffineInvariance,SIFT
78
1. Beforeyoustart,youcannormalizetheimages.2. Differencebasedmetricscanbeused(Haar,SIFT,…)
![Page 79: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/79.jpg)
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
79
![Page 80: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/80.jpg)
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
80
• Extractfeatures
![Page 81: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/81.jpg)
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
81
• Extractfeatures• Computeputativematches
![Page 82: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/82.jpg)
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
82
• Extractfeatures• Computeputativematches• Loop:
– Hypothesize transformationT(smallgroupofputativematchesthatarerelatedbyT)
![Page 83: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/83.jpg)
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
83
• Extractfeatures• Computeputativematches• Loop:
– Hypothesize transformationT(smallgroupofputativematchesthatarerelatedbyT)
– Verifytransformation(searchforothermatchesconsistentwithT)
![Page 84: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/84.jpg)
Lecture5&6–FindingFeatures,AffineInvariance,SIFT
84
• Extractfeatures• Computeputativematches• Loop:
– Hypothesize transformationT(smallgroupofputativematchesthatarerelatedbyT)
– Verifytransformation(searchforothermatchesconsistentwithT)
![Page 85: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/85.jpg)
ObjectRecognitionLecture5&6–FindingFeatures,AffineInvariance,SIFT
85
Fortrainingimages:ExtractingkeypointsbySIFT.Creatingdescriptorsdatabase.
Performingdetailedgeometricfitcheckforeachcluster.
Forqueryimages:ExtractingkeypointsbySIFT.Foreachdescriptor- findingnearestneighborinDB.
Findingclusterofat-least3keypoints.
![Page 86: Lecture 5&6 –Finding Features, Affine Invariance, SIFTbagci/teaching/computervision17/Lec5and6.pdf · 2017-09-26 · Lecture 5&6 –Finding Features, Affine Invariance, SIFT 47](https://reader034.vdocuments.site/reader034/viewer/2022042215/5ebd3bc0e047dd36551efc85/html5/thumbnails/86.jpg)
ImageRegistrationLecture5&6–FindingFeatures,AffineInvariance,SIFT
86