feature description and matching - cornell university
Post on 03-Apr-2022
10 Views
Preview:
TRANSCRIPT
MatchingfeaturepointsWeknowhowtodetectgoodpointsNextquestion:Howtomatchthem?
Twointerrelatedquestions:1. Howdowedescribeeachfeaturepoint?2. Howdowematchdescriptions?
?
Featurematching
⢠Measurethedistancebetween(orsimilaritybetween)everypairofdescriptors
đđ đđ
đđ đ(đĽ", đŚ") đ(đĽ", đŚ#)
đđ đ(đĽ#, đŚ") đ(đĽ#, đŚ#)
Invariancevs.discriminability
⢠Invariance:â Distancebetweendescriptorsshouldbesmallevenifimageistransformed
⢠Discriminability:â Descriptorshouldbehighlyuniqueforeachpoint(farawayfromotherpointsintheimage)
Invariance
⢠Mostfeaturedescriptorsaredesignedtobeinvarianttoâ Translation,2Drotation,scale
⢠Theycanusuallyalsohandleâ Limited3Drotations(SIFTworksuptoabout60degrees)â Limitedaffinetransformations(somearefullyaffineinvariant)â Limitedillumination/contrastchanges
Howtoachieveinvariance
Designaninvariantfeaturedescriptorâ Simplestdescriptor:asingle0
⢠Whatâsthisinvariantto?⢠Isthisdiscriminative?
â Nextsimplestdescriptor:asinglepixel⢠Whatâsthisinvariantto?⢠Isthisdiscriminative?
SSD
⢠Useasdescriptorthewholepatch⢠Matchdescriptorsusingeuclidean distance⢠đ đĽ, đŚ = ||đĽ â đŚ||#
NCC- NormalizedCrossCorrelation
⢠Lightingandcolorchangepixelintensities⢠Example:increasebrightness/contrast⢠đź2 = đźđź + đ˝â˘ Subtractpatchmean:invariancetođ˝â˘ Dividebynormofvector:invariancetođźâ˘ đĽâ˛ = đĽâ< đĽ >
⢠đĽâ˛â˛ = 92||92||
⢠similarity=đĽ22 â đŚâ˛â˛
Basiccorrespondence
⢠Imagepatchasdescriptor,NCCassimilarity⢠Invariantto?
â Photometrictransformations?â Translation?â Rotation?
⢠Finddominantorientationoftheimagepatchâ Thisisgivenbyxmax,theeigenvectorofM correspondingtolmax (the
larger eigenvalue)â Rotatethepatchaccordingtothisangle
Rotationinvarianceforfeaturedescriptors
FigurebyMatthewBrown
Take40x40squarewindowarounddetectedfeatureâ Scaleto1/5size(using
prefiltering)â Rotatetohorizontalâ Sample8x8squarewindow
centeredatfeatureâ Intensitynormalizethe
windowbysubtractingthemean,dividingbythestandarddeviationinthewindow
CSE576:ComputerVision
MultiscaleOrientedPatcheS descriptor
8 pixels
AdaptedfromslidebyMatthewBrown
⢠Finddominantorientationoftheimagepatchâ Thisisgivenbyxmax,theeigenvectorofM correspondingtolmax (the
larger eigenvalue)â Rotatethepatchaccordingtothisangle
Rotationinvarianceforfeaturedescriptors
FigurebyMatthewBrown
Detour:Imagetransformations
⢠Whatdoesitmeantorotateapatch?⢠Eachpixelhascoordinates(x,y)⢠RotationrepresentedbyamatrixR⢠Pixelâsnewcoordinates:
⢠Iâ(xâ,yâ)=I(x,y)
x0
y0
ďż˝= R
xy
ďż˝
Detour:Imagetransformations
⢠Whatifdestinationpixelisfractional?⢠Flipcomputation:foreverydestinationpixelfigureoutsourcepixelâ Useinterpolationifsourcelocationisfractional
⢠Iâ(xâ,yâ)=I(x,y)
xy
�= R�1
x0
y0
ďż˝
Take40x40squarewindowarounddetectedfeatureâ Scaleto1/5size(using
prefiltering)â Rotatetohorizontalâ Sample8x8squarewindow
centeredatfeatureâ Intensitynormalizethe
windowbysubtractingthemean,dividingbythestandarddeviationinthewindow
CSE576:ComputerVision
MultiscaleOrientedPatcheS descriptor
8 pixels
AdaptedfromslidebyMatthewBrown
Towardsabetterfeaturedescriptor
⢠Matchpatternofedgesâ Edgeorientationâ cluetoshape
⢠Beresilienttosmalldeformationsâ Deformationsmightmovepixelsaround,butslightly
â Deformationsmightchangeedgeorientations,butslightly
Invariancetodeformationbyquantization
g(â) =
8>>>><
>>>>:
0 if 0 < â < 2âĄ/N1 if 2âĄ/N < â < 4âĄ/N2 if 4âĄ/N < â < 6âĄ/N
. . .N ďż˝ 1 if 2(N ďż˝ 1)âĄ/N
T.Tuytelaars,B.Leibe
RotationInvariancebyOrientationNormalization
⢠Computeorientationhistogram⢠Selectdominantorientation⢠Normalize:rotatetofixedorientation
0 2p
[Lowe,SIFT,1999]
TheSIFTdescriptor
⢠Computeedgemagnitudes+orientations⢠Quantizeorientations(invariancetodef)⢠Divideintospatialcells⢠Computeorientationhistogramineachcell(spatialinvariance)
DistinctiveImageFeaturesfromScale-InvariantKeypoints.Lowe.InIJCV2004
Basicidea:⢠DoG forscale-spacefeaturedetection⢠Take16x16squarewindowarounddetectedfeature
⢠Computegradientorientationforeachpixel⢠Throwoutweakedges(thresholdgradientmagnitude)⢠Createhistogramofsurvivingedgeorientations
ScaleInvariantFeatureTransform
AdaptedfromslidebyDavidLowe
0 2p
angle histogram
SIFTdescriptorCreatehistogram
⢠Dividethe16x16windowintoa4x4gridofcells(2x2caseshownbelow)⢠Computeanorientationhistogramforeachcell⢠16cells*8orientations=128dimensionaldescriptor
AdaptedfromslidebyDavidLowe
SIFTvectorformation⢠Computedonrotatedandscaledversionofwindowaccordingtocomputedorientation&scaleâ resamplethewindow
⢠BasedongradientsweightedbyaGaussian
Ensuresmoothness⢠Trilinear interpolation
â agivengradientcontributesto8bins:4inspacetimes2inorientation
Reduceeffectofillumination⢠128-dimvectornormalizedto1⢠Thresholdgradientmagnitudestoavoidexcessiveinfluenceofhighgradientsâ afternormalization,clampgradients>0.2â renormalize
PropertiesofSIFTExtraordinarilyrobustmatchingtechnique
â Canhandlechangesinviewpoint⢠Uptoabout60degreeoutofplanerotation
â Canhandlesignificantchangesinillumination⢠Sometimesevendayvs.night(below)
â Fastandefficientâcanruninrealtimeâ Lotsofcodeavailable:
http://people.csail.mit.edu/albert/ladypack/wiki/index.php/Known_implementations_of_SIFT
Summary⢠Keypoint detection:repeatableanddistinctiveâ Corners,blobs,stableregionsâ Harris,DoG
⢠Descriptors:robustandselectiveâ spatialhistogramsoforientationâ SIFTandvariantsaretypicallygoodforstitchingandrecognition
â But,neednotsticktoone
Featurematching
GivenafeatureinI1,howtofindthebestmatchinI2?1. Definedistancefunctionthatcomparestwo
descriptors2. TestallthefeaturesinI2,findtheonewithmin
distance
FeaturedistanceHowtodefinethedifferencebetweentwofeaturesf1,f2?
â Simpleapproach:L2 distance,||f1 - f2 ||â cangivegoodscorestoambiguous(incorrect)matches
I1 I2
f1 f2
f1 f2f2'
FeaturedistanceHowtodefinethedifferencebetweentwofeaturesf1,f2?
⢠Betterapproach:ratiodistance=||f1 - f2 ||/||f1 - f2â||⢠f2 isbestSSDmatchtof1 inI2⢠f2âis2nd bestSSDmatchtof1 inI2⢠giveslargevaluesforambiguousmatches
I1 I2
top related