On Multi-scale differential features for face recognition

S.Ravela Allen R. HansonCenterfor IntelligentInformationRetrieval Vision Laboratory

Dept.of ComputerScience,Universityof MassachusettsatAmherst,MA, 01002�ravela,hanson� @cs.umass.edu�

AbstractThis paper describes an algorithm that uses multi-scale

Gaussian differential features (MGDFs) for face recogni-tion. Results on standard sets indicate at least 96% recog-nition accuracy, and a comparable or better performancewith other well known techniques. The MGDF based tech-nique is very general; its original application includedsimilarity retrieval in textures, trademarks, binary shapesand heterogeneous gray-level collections.

1 IntroductionFacerecognitiontechnologiescansignificantly impact

authentication,monitoring and image indexing applica-tions. This paperpresentsan algorithmto computesim-ilarity of facesasa whole. Thetaskis to querya databaseusing the imageof a faceand then have the systemei-ther ascertainits identity, or retrieve the top � similarmatches.As such,the techniqueis generalandhashith-erto beenusedsuccessfullyin imageretrieval taskssuchas finding similar scenes,trademarks,binary shapesandtextures[23, 24, 25]. Theapproachis basedonthetwo hy-potheses;first thatvisualappearanceof a faceplaysanim-portantrole in judging similarity andsecond,multi-scaledifferential featuresof the imagebrightnesssurfaceformeffectiveappearancefeatures.

Thefirst hypothesisis basedon theobservationthatvi-sualappearanceis an importantcuewith which we judgesimilarity. We readilyrecognizeobjectsthatsharea visualappearanceassimilar, andin theabsenceof otherevidence,arelikely to rejectthosethatdonot. A precisedefinitionof�

This material is basedon work supportedin part by the followingsponsors:1. Departmentof Commerceunder cooperative agreementnumberEEC-9209623,2. DefenseAdvancedResearchProjectsAgen-cy/ITO underARPA ordernumberD468, issuedby ESC/AXA contractnumberF19628-95-C-0235,3. Air ForceOffice of ScientificResearchundergrantnumberF49620-99-I=0138,4. NationalScienceFoundationIRI-9619117andMultimediaCDA-9502639.Additionally this work hasbeenmadepossibledueto thesupportof the Library of Congress.Anyopinions,findingsandconclusionsor recommendationsexpressedin thismaterialaretheauthor’s anddo not necessarilyreflectthoseof thespon-sors.

visualappearanceis difficult. Thephysicalandperceptu-al phenomenathatdefineappearancearenot well known,and even when thereis agreement,suchas the effect ofobject(3D)shape,surfacetexture,illumination,albedoandviewpoint, it is non-trivial to decomposean imagealongthesecomponents.However, early recognitionalgorithm-s [9, 16, 32] broughtforwardthenotionthat thesimilaritybetweencomputationalrepresentationsof imagedbright-nesssurfaces,in many cases,correlateswith similaritiesinvisualappearanceof objects.Therefore,it is notunreason-ableto develop appearancerepresentationsandsimilaritymeasuresto suit the semanticsof the retrieval or recogni-tion task.

In this paper, an appearancerepresentationfor facerecognitionusingdistributionsof local featuresof the im-agebrightnesssurfaceis constructed.Local featuresareobtainedby applyingoperatorsto the imagethat, equiva-lently, canbethoughtof astunablespatial-frequency filter-s, statisticaldescriptorsof the brightnesssurface,or ap-proximationsof the local shapeof the imagebrightnesssurface. Specifically, multi-scaledifferential featuresareused [3, 5, 7, 11, 15, 23, 24, 25, 26, 21, 28, 29] andthischoiceis motivatedby arguments[3, 7] thatthelocalstruc-ture of an imagecan be representedin a stableand ro-bustmannerby theoutputsof asetof multi-scaleGaussianderivativefilters (MGDFs)appliedto animage.In ordertodeduceglobal similarity betweentwo faceimages,multi-scaledifferential featuresare composedinto histogramsandcorrelated.

Thefirst partof this paperbeginswith a brief review ofscale-spacetheoryunderlyingMGDFsandendswith anal-gorithmto deduceglobalsimilarity. In thesecondpart,thisalgorithmis appliedto facerecognition.Usingthedatabas-esandprotocolfor evaluationdescribedby Simet. al. [30],thispaperdemonstratesthatthealgorithmpresentedhereisat leastaseffective whencomparedto severalothermeth-ods.

1.1 Related Work

Facerecognitionhasreceived significantattentionandit is beyond the scopeof this paperto fully investigatethe available techniques.Insteadwe describetechniquesthataremostrelevant to our approach.Sim et al [30] usea relatively simple techniqueof matchingdecimatedim-ageswith extremelygoodresults.Althoughour approachis completelydifferentwe usetheir evaluationmethodol-ogy. Othertechniquesfor facerecognitionhave alsobeendevelopedusingprojectionprofiles[33], deformablesur-faces[14], hiddenMarkov models(HMM) [27], andself-organizingmaps[10]. Noneof thesetechniquesarerelatedto the onespresentedhere,but comparisonscanbe madeby readingtheresultspresentedhereandthoseresultspre-sentedby LawrenceandSim[10, 30]. ResultsontheFER-ET collectionwith othertechniquesmayalsobe found inPhillips [19].

From an appearancerepresentationstandpoint,princi-pal componentanalysis(PCA) basedtechniquesaremorerelevant. PCA waspioneeredby Kirby andSirovich [9]asa representationfor faceswhich wasalsodevelopedin-to an effective facerecognitionsystemby Turk andPent-land [32], with generalizationsto multiple views [16, 18],illumination changes[16], andreplicatedon otherobject-s [16]. Sincethesuccessof eigendecompositiondepend-s on the objectsbeingcorrelatedan attemptwasmadetoovercomethis restrictionby Swetset. al [31, 36]. Theyextendthe traditionalPCA methodto multiple classesofobjectsusingFischer’s discriminantanalysis[1]. Theap-proachpresentedin this paperis differentbecauseEigendecompositionsare not usedto characterizeappearance.Further, the methodpresentedhereusesno learninganddoesnot requireconstantsizedimages.In fact,oneof theconclusionsdrawn from thispaperis thatascale-spacede-composition(ratherthan an eigenone) performsequiva-lently well. That is, an unbiasedrepresentationperformsaswell (if not better)thanthelearnedrepresentation.

Appearancefeaturescan also be extractedin the fre-quency domainandin this sensearecommonlyrelatedtotexture features. In the context of imageretrieval Ma et.al. [12] useGaborfilters to retrieve imageswith similartexture.Gaborjets[34] havealsobeenusedfor facerecog-nition. We find that a comparisonbetweenGaussianandGaborfilters is instructive. Gaborfilters are sine modu-latedGaussianfunctions,which canbe tunedto respondto a bandwidtharounda certaincenterfrequency. Theyexhibit compactnessin spaceand frequency, are optimalin the senseof the uncertaintyprinciple (time-bandwidthproduct)andarecomplete.Gaborfilters arenot equivari-ant with rotations,andseparableimplementationsareex-pensive. In contrast,Gaussianderivativesexhibit thesametime-bandwidthpropertyand althoughthey have infinite

support,they canbe safelytruncatedat aroundfour stan-darddeviations. While Gaussianderivativeshave coupledbandwidthandcenterfrequency, in practiceseparatetun-ing is notnecessary. Rather, thederivativesprovidea“nat-ural” samplingof thefrequency space,becausethey repre-sentthe ordersof approximationin a Taylor seriessense.Thesignificantadvantageof usingtheGaussianderivativesis that, they areequivariantwith rotations[4] eliminatingtheneedfor explicitly orientedfilters andalsosupporttheformulationof rotationalinvariants. Gaussianderivativesare separableand efficient implementationsare possible.Thereareseveralotherinterestingpropertiesandtheread-er is referredto [6, 23] for a morebasicreview.

2 Computing Global SimilarityThe stepsinvolved in deducingsimilarity betweena

query face image and a databaseimage are as follows:Databaseimagesarefiltereda priori with Gaussianderiva-tives,andthen,at eachpixel, the gradientorientationandsurfacecurvatureis computed.A query imageis filteredthesameway andmulti-scalehistogramsof curvatureandorientationarecorrelatedto measuresimilarity. In theau-thenticationtasktheidentityof thebestmatchingimageinthedatabaseis ascribedto thequeryandin themonitoringtask,the top � arepresentedto the user. Below, the useof differential featuresand the stepsin the algorithm arediscussed.2.1 Differential features:

The simplestdifferential featureis a vectorof spatialderivatives. For example, given an image � , and somepoint, ��� the first two ordersof spatialderivativescanbeusedas a feature(vector). This vectorapproximatestheshapeof the local intensitysurfacein the senseof a sec-ond orderTaylor approximation.Including higherordersproducesa morepreciseapproximation.Derivativescap-tureusefulstatisticalinformationabouttheimage.Thefirstderivativesrepresentthegradientor ”edgeness”of the in-tensityandthesecondderivativescanbeusedto representcurvatures(bars,blobsandsoon).

However it is importantthatderivativesbecomputedina stablemanner. Derivativeswill be stableif, insteadofusingjust finite differences,they arecomputedby filteringan imagewith normalizedGaussianderivative filters (ac-tually any � functionwill do [3]). In two dimensions,aGaussianderivative is thederivativeof thefunction

�� �� ������� ���� ����� � ��! � �#"%$&�� ���In the frequency domain,a Gaussianderivative filter is

a band-passfilter, asshown in Figure1 (one-dimensionalcase).Computingderivativesby filtering with a Gaussianderivative at a certainscale,therefore,implies that only

−0.5 0 0.50

Frequency

Mag

nitu

de o

f fre

quen

cy r

espo

nse

Gaussian derivative filters in the frequency domain

Order of derivative54321

Figure1: Gaussianderivative filters in the frequency do-main.

a limited bandof frequenciesare beingobserved. Thus,in order to describethe original imagemore completely,a multi-scalerepresentationis necessary. Samplingthescale-spaceof theimagebecomesessential.

2.2 Gaussian scale-space:Thenecessityof a multi-scalerepresentationdescribed

above canbeconcludedfor any smoothband-limitingfil-ter by usingthecommutativity of differentiationandcon-volution. The Gaussianhappenedto be a convenien-t function; it hasnaturalscaleparameterization,smooth-nessandself-similarityacrossscales.However, theGaus-sian is more than just convenient. Thereare compellingtheory and implementationrelated argumentsfor usingmulti-scaleGaussianderivativesto form appearancefea-tures. In particular, it has been shown by several au-thors [3, 5, 11, 26, 35], that under certain generalcon-straints,the (isotropic)Gaussianfilter formsa uniqueop-eratorfor representinganimageacrossthespaceof scales.Structures(suchasedges)observedat a coarserscalecanberelatedto structuresalreadypresentat a finer scaleandnotasanartifactof thefilter. In general,theGaussian(lin-ear)scale-spaceservesasan unbiased(without usinganyotherinformation)front end(pre-processor)for represent-ing theimagefrom whichdifferentialfeaturesmaybecom-puted. It is beyond the scopeof this documentto engagein a full discussionaboutthe scale-spaceimagerepresen-tation and,instead,the readeris referredto the followingpapers[3, 11, 26, 35, 23]. Otherreasonsfor choosingtheGaussianarepresentedin Section1.1.2.3 Curvature and Orientation:

Several differential featurescan be constructedfromderivativesandseveral representationsandmethodshavebeendeveloped[21, 28, 25, 29, 24, 23, 22] for recognitionandretrieval. Thechoiceof thesefeaturesdependsonsev-

eralfactors,primary(amongthese)is toleranceto rotation,illumination,scalesincevariationsin theseaffectsappear-ance.Herewe arguefor two particularfeatures.

Since the task is to robustly characterizethe 3-dimensionalintensitysurface(X, Y, Intensity),localcurva-turesareappropriatebecausethesurfaceis uniquelydeter-minedfrom them. In particular, two principal curvatures,namelythe isophoteandflowline curvescanbecomputedat a point,andrepresentthecurvaturesof theiso-intensitycontoursandthegradientintegralcurves.In fact,principalcurvaturesarenothingmorethanthesecondorderspatialderivativesexpressedin acoordinateframe(gauge[3]) de-terminedby theorientationof the local intensitygradient.Theprincipalcurvaturesof theintensitysurfaceareinvari-antto imageplanerotations,monotonicintensityvariationsandfurther, their ratiosare,in practice,quite tolerantto s-calevariationsof the entire image. The isophote(N) andflowline (T) curvaturesaredefinedas[8, 3]:

� � ')(+* � �-,.�0/1�-,2/3!4� �, �-/-/3!4� �/ �-,2,�5 (1)6 � '87 * � ,2/ � �, !4� �/ � " � , � / � /0/ !4� , , � 5 (2)

' � � �, " � �/ �:9<;= (3)

�-,��>�-, �?����� and �-/@�>�0/ �?����� arethefirst orderpar-tial spatialderivativesof imageI aroundpointp, computedusingGaussianderivative at scale � . Similarly, �-,2,��A�-, / ,and �0/-/ are the correspondingsecondderivatives. Theisophotecurvature N and flowline curvature T are thencombinedinto a ratio calledtheshapeindex, expressedasfollows [8, 2, 15]: B�DC E �GF !IHJ 7LKNMOKNPRQTS�UQV9WUYX � TheindexvalueC is undefinedwheneitherN andT areboth zero,andis, therefore,notcomputed.This is interestingbecauseveryflat portionsof animage(constantor constantslopeinintensity)areeliminated.The shapeindex is in the range[0,1]. Nastar[15] alsousestheshapeindex for recognitionandretrieval. However, hisapproachusescurvaturescom-putedata singlescale.Clearly, astheexperimentssuggest(seeSection3), this is not enough.

Thesecondfeatureusedis local orientation.Local ori-entationis the directionof the local gradient.Orientationis independentof curvature,is stablewith respectto scaleand illumination changes.The orientationis simply de-finedas Z �[KNMOKNP � � / �A� , � NotethatP is definedonly atthoselocationswhereC is andignoredelsewhere.As withtheshapeindex P is rescaledandshiftedto lie betweentheinterval [0,1].

Feature Histograms: Histogramsof the shapeindexand orientationare usedto representthe distributions offeaturesover an image. Histogramsform a global rep-resentationbecausethey capturethe distribution of lo-cal featuresand they are the simplestways of estimat-ing a non parametricdistribution. In this implementa-

tion, curvatureandorientationaregeneratedat several s-calesandrepresentedasa onedimensionalrecordor vec-tor. Therepresentationof theimageI is thevector \�]��@^_�` � H � �2�0� _a` �cb��:� _ed � H � �0�2� _ed ��bc�<f .

_�`and

_edare

the curvatureand orientationhistogramsrespectively. Itshouldbe notedthat [28] usehistogramsof variousdif-ferential features.However, the differencebetweenthe t-wo approachesis thattheirmethodusesmulti-dimensionalhistogramsof featuresthatdoesnotincludecurvature.Fur-ther, their representationsarecomputedat a singlescale.Multi-dimensionalhistogramstendto bevery sparse,andfurther, arecomputationallymoreexpensive to match.Webelievethatusingonedimensionalhistogramsatseverals-cales(andstringingthemtogether)providesa sufficientlyrich representationof theimage.

Figure2: Examplesof theFERET(firstpair)andORL(nextfour pairs)sets.

Matching feature histograms: Two representationsare comparedusing normalizedcross-covariancedefined

as g ]Gh � i�jlk?mnpo iqjGk�mrsGs i jlk?mn sGsGsGs i jGk�mr sGs Where \�tvuTw] �I\ ] !8x � KNP \ ] � .Thereareother possiblemeasures,suchas the Kulback-Leibler [1] and Mahalanobis[13] distanceswhich couldbeused.Thequeryhistogramvector \zy is comparedwitheachdatabasehistogramvector \ ] . Thecorrespondingim-agesarerankedby their score.We call this algorithmthe1D curvature/orientationor CO-1algorithm.

3 Face RecognitionTwo variationsof the algorithmarecomparedfor face

recognition.Thefirst is CO-1,wherehistogramsarebuiltovertheentireimage(CO-1).Thesecondis PCO-1,wheretheimageis partitionedinto threetiles roughlycoveringathird of the imageandhistogramsfor eachtile aregener-

atedseparatelyandconcatenated(PCO-1). Assumingtheimagesareroughly facesegmentedto begin with, the toptile correspondsto the foreheadregion, the middle tile tothemid-faceandthebottomtile correspondingto thechinregion.

Datasets: The following three datasetsare usedforevaluations. 1. ORL Set [17]: the ORL (Olivetti Re-searchLab) collectionis a publicly availablecollectionof400 faces. This collection contains40 individuals. Thedatabasecontainssmallview, gesture,andintensityvaria-tion. Seethesecondthroughfourth facepair of Figure2.2. FERETSet[19]: TheFERETdatasetis maintainedbyNIST and the CDROM contains3737images. However,our testswererepeatedin exactly the sameconfigurationasSim [30] andthereforewe only used275 imagesof 40individuals. Theseimagescontainbust photographswithvaryingbustcoverage,andsmall facialgestureandimageillumination changes.Seefirst facepair in Figure 2. 3.UMASS TeaCrowd Set [20]: The UMassTeaCrowd setconsistsof 119imagesof facesextractedfrom a livevideofeedof camerasmonitoringa TeaParty. Therearetotal of15peoplein thiscollection.Thesefacescontaingesture,il-lumination,andview variations,in additionto motionblurandocclusion.SeeFigure3.

Evaluation: The evaluationmethodologyfollows theone describedby Sim et. al. [30]. During eachtrial adatabaseis randomlysplit into a trainingsetanda testset.Theconfigurationsof trainingsetpertrial useseither5 ex-emplarsper personor the greatestnumberlessthanhalfthenumberof facesavailablefor thatperson,whichever issmaller. The remainingfacesfor the personbecomethetestset.Eachof thesetestsetimagesbecomesa query. Aqueryis matchedwith all of thetrainingsetandtheidenti-ty of thebestmatchingtrainingsetimageis ascribedto thequery. Over a large (100) numberof trials the proportionof correctlyidentifiedpeopleis reportedastherecognitionrate.For example,in theORL seta trial will consistof 200trainingandtestimageseach.Thus,over100trials 20,000queries(testset)arematchedwith a randomtraining/testpick at every trial.

Examples: In Figure2, queriesandcorrespondingex-emplarimages(selectedduring sometrial) they matchtoareshown. The first facepair is drawn from the FERETset. Notethat theseimageswerenot processedto localizethefaceportionalone.Theremainingfour pairsin Figure2show resultsfrom the ORL set. Note that the secondpairin the secondrow in Figure2 is a mismatch.The correctidentity is not recovered,but qualitatively boththesefacesshareasignificantsimilarity in appearance.

In Figure 3, several examplesfrom the TeaCrowd setareshown from a retrieval perspective. Each”row” of thisFigurecontainssix images,thefirst beingthequeryandthe

Figure3: Examplesof theTeaCrowd setfrom a retrieval point of view.

remainderbeingthe imagesmatchedin rank order. Eachimageis labeledby its matchscoreto thequery(1.0ismax-imum). Theseexamplesshow recognitionfrom a retrievalpointof view. Thequeriesincludegesturevariations,scalevariations,occlusions,motionblur andview variations.

Analysis: The performanceof the algorithm is de-picted in Table 1. On all threesetsthe performanceisvery goodandcomparableto otheralgorithms,specifical-ly, thosebasedon Principalcomponentanalysis[32] andCMUs [30] technique.Thereaderis referredto Sim’s pa-per [30] for additionalcomparisonswith othertechniques(they performworsethanCMUs technique). In Table1,column2 indicatesthe evaluationparametersused. In al-l methods5 exemplarsare usedand when it is not pos-sible to do so, only half the available are used. In ourtechniquenothing is done to the imagesin termsof in-tensitystretching,warpings,faceextractionor generatingsyntheticimages.In contrastin Sim’s techniquebasedonmatchingthumbnails,syntheticimagesaregeneratedfromexemplars(rotatedandslightly scaledversions)andthesebecomepartof the trainingset. A query’s scoreagainsta

databaseindividual is themeanover thescoresthat it getsfor all trainingsamplesof theindividual. Wepick themax-imum. The implementationof Eigenfacesreportedin thesamepaperalsousessyntheticimagesfrom theexemplars,40EigenvaluesandtheL2 normto comparethequeryvec-tor. In this case,like our method,the identity of the bestmatchingimageis ascribedto thequery. Note that there-sultsreportedherefor Eigenfacesarethebestof theresultsreportedby Sim et. al. [30](also seeLawrence’s compar-isons[10]).

Thealgorithmpresentedherehastwo principalparam-eters;scalesandthebin sizesof thehistograms.Thegraphin Figure 4, depictsthe performanceof the systemwithvariationin scalefor theORL setusingtheCO-1algorith-m (othersetshavesimilar results).For thisgraphthenum-berof curvatureandorientationbinswereeachfixedat40.TheX-axisof thisgraphis abyte-encodednumberthatin-dicatesthescalesused.TheLSB meansascalevalueof 1,thenext leastsignificantbit correspondsto ascalevalueof{ �

andsoon throughstepsof{ �

, to anMSB valuerep-resenting| { �

. Thevalid numbersfor this byteare1-255,

Technique EvaluationParameters ORL FERET TeaCrowdUMASSPCO1 5 samples,0 synthetic 98% 96% 96%

CMU L0, 5 samples,10synth 97% 96% .IP.UMASSCO1 5 samples,0 synth 95% 90% 90%

Eigen-face 40 vector, L2, 5 samples,10synth 95% 90% .IP.

Table1: Theperformanceof MGDF methodswith PCAandCMUs techniques

Figure4: Theperformanceon theORL set.For this graph40 binswereusedin thehistogram.

1 implying the useof only scale1, 255 implying the useof all 8 scales.The Y-axis of this graphdepictsrecogni-tion rateover100trials. Thustherecognitionperformancewith respectto scalesis exhaustively plotted. Therearethreeplots in Figure 4. The lower onecorrespondingtothe useof 1 exemplar, the middle onecorrespondingto 3exemplars,andthetop onecorrespondingto 5 exemplars.

Several conclusionscan be drawn from this figure.First, the performanceimproves categorically with in-creasein exemplars,andthis is truefor all variationsof thealgorithmspresentedhere.Second,asinglescale,which ischaracterizedby largedipsin theplot is indicativeof poorperformance,andshows thenecessityfor multiple scales.Third, all eightscalesarenotnecessary. It canbeobservedfor examplethat a packed setof scalesof smallerextent(suchasbit code96) give approximatelythesameperfor-manceasusingall scales(suchasbit code255). Finally, adensepackingof scalesis not essentialeither. A sequenceof scalesthatis denselypacked,suchas

�0�0� �.�}�.� �2�0� , caus-es only marginal changesin accuracy in relation to onethat is coarser, suchas

�0�0� � E � E �0�2�:�In mostcaseswe find

thatanoctavespacingis sufficient,andtwo octavesepara-tion resultsin lessthan �1~ drop in recognitionaccuracy.

This suggeststhat the multi-scalerepresentationcanhaveasomewhatlargesamplewidth acrossscales.This is goodnews becauseit implies that significant”compression”intherepresentationis possible.Theshapeof this graphre-peatsitself for variousbin combinations.

Figure5: RecognitionPerformancewith variationin Binsizesfor CO-1onORL set.All scaleswereused.

Figure6: RecognitionPerformancewith variationin Binsizesfor PCO-1onORL set.All scalesareused.

The secondfactor that wasvaried is the bin size. Forthe experimentsconductedall the scaleswere usedwith5 exemplarsand the bin sizesweresystematicallyvariedfrom 10 to 100for curvatureandorientationindependent-ly, thereforegiving amatrixof 100combinations.Surpris-ingly, therecognitionratesheldverystable:PCO-1variedbetween97.2%and98.2%(seeFigure6; andCO-1 (seeFigure5) between94.1%and95.2%.Thevariancefor anygivenobservationover the trials waslessthan1%. Final-ly, in termsof computation,it takesa few milli-secondsto recognizeapproximately200imagesfrom thedatabase,andin contrastit takesabout0.4 secondson a �}E}E}� _+�

Figure7: Facelocalizationandrectificationfor recognitionin akiosk.

PentiumII processorwith sufficientmemory.

4 Summary and ConclusionsTheresultspresentedin this paperarevery exciting for

the following reasons.First, thecurvatureandorientationbasedmethodperformswell; especiallybecausethereisno learninginvolvedwith respectto any of theparameters.Arguably, arepresentationbasedonthedifferentialdecom-positionof theimageat multiplescalesis giving compara-bleperformanceto onebasedon learningacompactrepre-sentationfrom thedata,namelyPCA.Thus,we find thesefeaturesto begoodfrom anappearancesimilarity point ofview. Second,while scaleis important,it seemsin faces,the changeof the feature(blur) with scaleis ratherslow.This is why densesamplingof scalesis notnecessary. Thisis goodfor a multi-scalerepresentation.Third, the appli-cationof a”spatial” partitiondramaticallyimprovesthere-sults,suggestingthatexplicit representationof spacemaybe necessaryand might be the principal reasonwhy therecognitionratesimprove. In conclusion,we believe thattherepresentationpresentedhereis turningout to bequiteversatile.

Weareextendingthiswork towardsconstructingakioskthatcanbeusedfor authenticationusinginexpensivecam-eras(QuickCams).Our presentapproachis to pre-processacquiredimagesby localizing facesand detectingfacialfeatures.Oncedetectedfacial featurescanbe usedto es-tablish a coordinatebasis from which partitions can becomputedfor PCO-1. One way to do this is to simplyrectify the facefor orientationand scale. Further, facialfeaturedetectionprovidescoarseinferenceof facial viewandthus,matchingcanbe speededup to nearbyviews inthedatabase.For example,in Figure7 threeimagestakenat thekiosk areshown. Thefirst is thefull imagetakenbythecamera,thesecondthedetectedfacewith anoverlayoffacialfeatures,andboxesaroundthe(final) localizationofeyes.Thethird is theorientationrectifiedview of thefacethatsimultaneouslyusestheorientationhistogramandtheinter-eye angleto rectify the face.While completeexper-imentationis forthcoming,in the context of this paper, itmaybenotedthatfacialfeaturesarelocalizedusingmulti-scaledifferentialfeatureswith naturalscaleselection.

AcknowledgementThe authorsthank JamesAllan, W. Bruce Croft, R.

Manmatha,andEdwardRisemanfor supportandfeedbackthroughvariousstagesof thiswork. TheauthorsalsothankMichael RemillardandJerodWeinmanfor implementingportionsof thekiosk.

References[1] C. M. Bishop,NeuralNetworks for PatternRecogni-

tion, OxfordUniversityPress,1994

[2] Chitra Dorai andAnil Jain,”COSMOS- A represen-tation schemefor free form surfaces”,ICCV 95, pp.1024-1029,1995.

[3] L M J Florack,The SyntacticStructureof ScalarIm-ages,PhDDissertation,Universityof Utrecht,1993

[4] W. T. FreemanandE. H. Adelson,Thedesignanduseof steerablefilters, IEEE Trans.Patt.Anal. andMach.Intel., 13(9):891-906,1991

[5] J. J. Koenderink,TheStructureof Images,BiologicalCybernetics,50:363-396,1984.

[6] GostaGranlundandHansKnutsson,SignalProcess-ing for ComputerVision, Kluwer AcademicPublish-ers,1995

[7] J. J. KoenderinkandA. J. vanDoorn,Representationof Local Geometryin the Visual System,BiologicalCybernetics,55:367-375,1987

[8] J. J. KoenderinkandA. J. Van Doorn,SurfaceShapeandCurvatureScales,ImageandVision Computing,10(8),1992

[9] M. Kirby andL. Sirovich,Applicationof theKruhnen-Loeve Procedurefor the Characterizationof HumanFaces,IEEETrans.Patt.Anal.andMach.Intel.,12(1):103-108,Jan.1990.

[10] S. Lawrence, C. Giles, A. Tsoi and A. Back,Face Recognition: A Hybrid Neural network ap-proach.Tech.ReportUMIACS-TR-96-16,Universityof Maryland,1996.

[11] T. Lindeberg, Scale-SpaceTheory in ComputerVi-sion,Kluwer AcademicPublishers,1994

[12] W. Y. Ma andB. S.Manjunath,Texture-BasedPatternRetrieval from Image Databases,Multimedia ToolsandApplications,2(1):35-51,Jan.1996

[13] P. C. Mahalanobis,On the GeneralizedDistanceinStatistics,Proceedingsof theNationalInstituteof Sci-ence,India,12:49-55,1936.

[14] B. Moghaddam,C.Nastar, andA. Pentland,Bayesianfacerecognitionusingdeformableintensity surfaces,In Proc.Comp.Vision andPatt. Recognition96, pp.638-645,1996.

[15] C. Nastar, The imageshapespectrumfor imagere-trieval, TechnicalReport3206,INRIA, June1997.

[16] S.K. NayarandH. MuraseandS.A. Nene,Paramet-ric AppearanceRepresentation,EarlyVisualLearning,Oxford UniversityPress,Feb. 1996.

[17] Olivetti ResearchLabs, Face Dataset, ��M�M�� ��.�1�R�R� � � KNx�!4���1� � � � � �W� �.� K � � g}KNMOK&�:K&� � � ��M�x��[18] A . Pentland,B. Moghaddam,andT. Starner, View-

basedModularEigenspacesfor facerecognition,Proc.Comp.VisionandPatt.Recognition,1994.

[19] P. JonathonPhillips, H. Moon, S. A. Rizvi andP. J.Rauss,TheFERETEvaluationMethodologyfor Face-RecognitionAlgorithms, Proc.ComputerVision andPatt.Recognition,1997.

[20] S. Ravela, The TeaCrowd Dataset”,2000,Contact:ravela@cs.umass.edu

[21] RajeshRaoandDanaBallard, Object Indexing Us-ing anIconic SparseDistributedMemory, Proc.Inter-nationalConferenceon ComputerVision, pp. 24-31,1995.

[22] S. Ravela, S. andC. Luo, Appearance-basedGlobalSimilarity Retrieval of Images.In Advancesin Infor-mationRetrieval, W. BruceCroft (Ed), Kluwer Aca-demicPublishers2000.

[23] S. Ravela, R. Manmatha,GaussianFiltered Repre-sentationsof Images,Encyclopediaof ElectricalandElectronicEngineering,JohnWebster(Editor), JohnWiley, 1999

[24] S.Ravela,S.andR.Manmatha,Retrieving ImagesbyAppearance,Proc.of theInternationalConf. on Com-puterVision, (ICCV), Bombay, India,Jan1998.

[25] S. Ravela, R. Manmatha, and E. M. Riseman,Retrieval from Image Databasesusing Scale SpaceMatching. Proc. of the EuropeanConf. on Comput-er Vision ECCV ’96, Cambridge,U.K., pages273–282,Springer, April 1996

[26] B. M. terHarRomeny, GeometryDrivenDiffusioninComputerVision,Kluwer AcademicPublishers,1994

[27] F. Samariaand A. Harter, Parameterizationof a s-tochasticmodelfor humanfaceidentification.In Proc.of Workshop on Applications of ComputerVision,1994.

[28] Bernt SchieleandJamesL. Crowley, ObjectRecog-nition Using MultidimensionalReceptive Field His-tograms,Proc.4th EuropeanConf. ComputerVision,Cambridge,U.K., April 96.

[29] Schmid,R. Mohr, LocalGrayvalueInvariantsfor Im-ageRetrieval, PAMI (19), No. 5, pp. 530-535,May1997.

[30] T. Sim, R. Sukthankar, M. Mullin, and S. Balu-ja, High-PerformanceMemory-basedFace Recog-nition for Visitor Identification, 1999 (see ��M�M�����.�1�R�R� � ��� � � x ��� � g � � � � �0� � � � �-� �-�.� � � ��M�x�� )

[31] D. L. SwetsandJ. Weng,UsingDiscriminantEigenFeaturesfor Retrieval, IEEE Trans.Patt. Anal. AndMach.Intel. 18(8): 831-836,1996.30.

[32] M. Turk andA. Pentland,EigenFacesfor Recogni-tion, Jrnl.CognitiveNeuroscience,3:71-86,1991.

[33] J. Wilder, FaceRecognitionusing transformcodingof grayscaleprojectionprojectionsandtheneuraltreenetwork, In Artificial Neural Networks with Appli-cations in Speechand Vision, R. J. Mammone(ed),pp.520-536,ChapmanHall, 1994

[34] L. Wiskott, J.-M. Fellous,N. Kruger andC. von derMalsburg, Face recognition by elastic bunch graphmatching,IEEE Trans Patt. Anal. and Mach. Intell.17(7):775-779,1997.

[35] A. P. Witkin, Scale-SpaceFiltering, Proc.Intl. JointConf.Art. Intell., pp.1019-1023,1983

[36] W. Zhao, A. Krishnaswamy, R. Chellapa,D. Swet-s, and J. Weng, Discriminant analysisof principalcomponentsfor face recognition, In Face Recogni-tion: FromTheoryto Applications,H. Weschler, P. J.Phillips et.al. (ed),pp 73-85,Springer-Verlag,Berlin,1998.