15-11-2011

1

Geometricverifica-onofmatching

Thecorrespondenceproblem

•  Thecorrespondenceproblemtriestofigureoutwhichpartsofanimagecorrespondtowhichpartsofanotherimage,a;erthecamerahasmoved,-mehaselapsed,and/ortheobjectshavemovedaround.

•  Giventwoormoreimagesofthesame3Dscene,takenfromdifferentpointsofview,atdifferent-mes,andwithobjectsinthesceneingeneralmo-onrela-vetothecamera,thecorrespondenceproblemis:

–  tofindasetofpointsinoneimagewhichcanbeiden-fiedasthesamepointsinanotherimagei.e.verifyiftheybelongtoaconsistentconfigura-on

•  Imagecontentistransformedintolocalfeaturesthatareinvarianttotransla-on,rota-on,andscaleandthecorrespondenceisassessedbycheckingifthelayoutofasubsetoffeaturesissimilarinthetwoimages.

15-11-2011

2

•  Aclassicalsolu-ontothecorrespondenceproblemistheRANSACalgorithm[Fischler81].RANSACpermitsthees-ma-onofparametersofamathema-calmodelbyrandomsampling.

•  Thebasicassump-onisthatthedataconsistsof"inliers",i.e.,datawhosedistribu-oncanbeexplainedbysomesetofmodelparameters,and"outliers"whicharedatathatdonotfitthemodel.RANSACalsoassumesthat,givena(usuallysmall)setofinliers,thereexistsaprocedurewhichcanes-matetheparametersofamodelthatop-mallyexplainsorfitsthisdata:

–  Itassumesthatobjectsareplanar,(validformanystructuresonbuildingsandmanmadeobjects;sufficientforsmallviewpointvaria-onson3Dobjects)

–  Itisnon‐determinis-cinthatitproducesareasonableresultonlywithacertainprobability,withthisprobabilityincreasingasmoreitera-onsareallowed.

–  Themaindisadvantageisthatnoupperboundexistsonthe-merequiredtocomputetheparameters.

RANSACRANdomSAmpleConsensus

TheRANSACalgorithm

•  Input:–  asetofobserveddatavalues;–  aparameterizedmodelwhichcanexplainorbefiYedtotheobserva-ons;–  confidenceparameters.

•  Generatek(apredeterminednumber)modelhypotheses,eachofthemiscomputedusingaminimalsubsetmofpoints

•  Foreachmodelhypothesis–  Drawasampleofm pointsfromdataatrandom–  Theparametersofthemodelarereconstructedfromthesetofpoints–  Computetheresidualswithrespecttoalldatapoints.Pointswithresidualslessthan

somethresholdt areclassifiedashypothe(calinliers–  Thees-matedmodelisreasonablygoodifsufficientlymanypointshavebeen

classifiedashypothe-calinliers.

–  Themodelisrees-matedfromallhypothe-calinliers(itwasonlybeenes-matedfromtheini-alsetofhypothe-calinliers).

–  Themodelisevaluatedbyes-ma-ngtheerroroftheinliersrela-vetothemodel.•  end

•  Ateachitera-onamodelisproducedthateitherisrejectedbecausetoofewpointsareclassifiedasinliersoraisarefinedmodelwithacorrespondingerrormeasure.Therefinedmodelisacceptedifitserrorislowerthanthelastsavedmodel.

15-11-2011

3

Example:fi\ngaline

Leastsquaresfit

FromD.Forsyth

• Selectsampleofmpointsatrandom:toes-matealine,2pointsareminimal.

15-11-2011

4

• Calculatemodelparametersthatfitthedatainthesample

• Calculateerrorfunc-onforeachdatapoint

15-11-2011

5

• Selectdatathatsupportcurrenthypothesis(pointswithresidualslessthansomethresholdareclassifiedasitsinliers)

• Repeatsampling

• Calculatemodelparametersthatfitthedatainthesample

• Calculateerrorfunc-onforeachdatapoint

• Selectdatathatsupportcurrenthypothesis(pointswithresidualslessthansomethresholdareclassifiedasitsinliers)

15-11-2011

6

• Repeatsampling

• Calculatemodelparametersthatfitthedatainthesample

• Calculateerrorfunc-onforeachdatapoint

• Selectdatathatsupportcurrenthypothesis(pointswithresidualslessthansomethresholdareclassifiedasitsinliers)

• …….Repeatun-lk

• Selectthehypothesiswiththemaximalnumberofinliersandre‐es-matethemodelparameterusingitsiden-fiedinliers.

Finalfi\ng

15-11-2011

7

Example:fi\ngaffinetransforma-on

•  Affinetransformof[x,y]to[u,v]:

•  Rewritetosolvefortransformparameters(6):

u, v x, y

Whatdowedoaboutthe“bad”matches?

FromS.Seitz,R.SzeliskiandA.Efros

15-11-2011

8

Whatdowedoaboutthe“bad”matches?

FromS.Seitz,R.SzeliskiandA.Efros

Selectonematch,countinliers

15-11-2011

9

Selectonematch,countinliers

Leastsquaresfit

Find“average”transla-onvector

15-11-2011

10

Howmanyitera-ons/samples?

•  Suppose:

wnumberofinliersindata/numberofpointsindatanisthenumberofpointsneededfores(ma(ngamodelselectedindependently,wnistheprobabilitythatallnpointsareinliers1−wnistheprobabilitythatatleastoneofthenpointsisanoutlierptheprobabilitythatthealgorithmproducesausefulresul

•  Theore-callythenumberofitera-onsk(thenumberofsamples)canbeobtainedfrom:

Itmustbechosenhighenoughtokeepthisbelowdesiredfailurerate.

Slidecredit:DavidLowe

•  Fordatawithmanyoutliers,therequirednumberofsamplesincreasesdrama-cally.Inprac-cethetheore-cales-matesareop-mis-candtheactualnumberofrequiredsamplesismuchhigher.

•  Theore-calnumberofsampleskneededtoensure95%confidencethatatleastoneoutlierfreesample:

Propor-onofinliersw[%]

samplesize(#

pointstakenatra

ndom

)

15-11-2011

11

RANSACsummary

•  Advantages–  Generalmethodsuitedtolargerangeofproblems–  Easytoimplement–  Independentofnumberofdimensions

•  Disadvantages–  Onlyhandlesmoderatenumberofoutliers(<50%)–  Noupperboundexistsonthe-merequiredtocomputetheseparameters−  Requiresalargenumberofsamplesfordatawithmanyoutliersthusheavycomputa-on−  Needstoknowtheoutlierra-otoes-matethenumberofsamples.Ifnopriorinforma-on,a

conserva-venumberneedtobeused,forinstance,60%forwidebaselinematchingresultstoensuresuccessfulrun.

−  Requiresathresholdfordeterminingwhetherpointsareinliers

•  Variousimprovementstostandardapproach[Nister,2004;Matas2005,SuYer2005,…….].Manyvariantsavailable

–  PROSAC:ProgressiveRANSAC[Chum,2005]–  Preemp-veRANSAC[Nister,2005]–  ….

ExampleApplica-ons

Mobiletouristguideself‐localiza-on,object/buildingrecogni-on,photo/videoaugmenta-on

AachenCathedral

[Quack,Leibe,VanGool,CIVR’08]

FromQuack,Leibe,VanGool,CIVR’08

15-11-2011

12

MoviePosterRecogni-on

Content‐basedretrievalfrommobilephone

FromQuack,Leibe,VanGool,CIVR’08

ImageAuto‐Annota-on

Le;:WikipediaimageRight:closestmatchfromFlickr

MoulinRouge

TourMontparnasse Colosseum

ViktualienmarktMaypole

OldTownSquare(Prague)

FromQuack,Leibe,VanGool,CIVR’08

15-11-2011

13

GeneralizedHoughTransform

•  Generaliza-onofHoughtransformforanarbitrarycontourorshape[Ballard,1981]–  Choosereferencepointforthecontour(e.g.thecenter)–  Foreachpointonthecontourrememberwhereitislocatedw.r.t.tothereferencepoint

(i.e.rememberradiusrandangleφrela-vetothecontourtangent)–  Recogni-on:wheneveryoufindacontourpoint,calculatethetangentangleand‘vote’

forallpossiblereferencepoints

•  Toassessatransforma-onbetweenimages,thesameideacanapplytolocalfeaturesversusatransforma-on

Slide credit: Bernt Schiele

• Ifthereistheneedtorecognizeclustersofjustasmallnumberofconsistentfeaturesamongalargenumberoffeaturematchhypotheses(e.g.1%vs99%)RANSACdoesnotwork.GeneralizedHoughtransformistheappropriatesolu-on.

•  Thekeytoefficiencyistohaveeachfeaturedetermineasmanyparametersaspossible–  Forexample,linescanbedetectedmuchmoreefficientlyfromsmalledgeelements(or

pointswithlocalgradients)thanfromjustpoints–  Forobjectrecogni-on,eachfeatureshouldpredictloca-on,scale,andorienta-on

•  ExamineallclustersinHoughtransformwithatleastmfeatures

•  Performleast‐squaresaffinefittomodel.

•  Discardoutliersandperformtop‐downcheckforaddi-onalfeatures.

•  Evaluateprobabilitythatmatchiscorrect

–  UseBayesianmodel,withprobabilitythatfeatureswouldarisebychanceifobjectwasnotpresent

–  Takesaccountofobjectsizeinimage,texturedregions,modelfeaturecountindatabase,accuracyoffit(D.Lowe,2001)

•  TheHoughtransformcanextractfeaturegroupingsfromcluYerinlinear-me

Slide credit: David Lowe

TheGeneralizedHoughalgorithm

15-11-2011

14

FromK.Grauman,B.Leibe

Example:recogni-onwithlocalfeatures

•  Foreveryfeature,storeallpossible“occurrences”

– Objectiden-ty– Pose– Rela-veposi-on

•  Fornewimage,letthematchedfeaturesvoteforpossibleobjectposi-ons

ExampleApplica-ons

FromPhilbinCVPR’07

Query Resultsfrom5kFlickrimages(demoavailablefor100kset)

Large‐ScaleRetrieval

15-11-2011

15

•  ModelsforplanarsurfaceswithSIFTkeys•  Planarsurfacescanbereliablyrecognizedatarota-on

of60°awayfromthecamera

•  Affinefitapproximatesperspec-veprojec-on

•  Only3pointsareneededforrecogni-on

Planarrecogni-on

FromK.Grauman,B.Leibe

3DObjectRecogni-on

•  Extractoutlineswithbackgroundsubtrac-on•  Only3keypointsareneededforrecogni-on,soextrakeypointsproviderobustness

•  Affinemodelisnolongerasaccurate

FromK.Grauman,B.Leibe