rotation-invariant image recognition at low light levels

954 J. Opt. Soc. Am. AVol. 3, No. 7/July 1986

Rotation-invariant image recognition at low light levels

Thomas A. Isberg and G. Michael Morris

The Institute of Optics, University of Rochester, Rochester, New York 14627

Received December 2, 1985; accepted March 8, 1986

Rotation-invariant filtering at low light levels is investigated. Low-light-level input scenes are cross correlated witha complex circular-harmonic component of a reference image stored in computer memory. Approximate expres-sions are given for the probability-density function of the correlation signal and for the squared modulus of thecorrelation signal when the number of detected photoevents is Poisson distributed as well as when the number ofdetected photoevents is fixed. Experimental measurements of the squared modulus of the correlation signal,obtained by using a two-dimensional, photon-counting detector and position-computing electronics, are found to beinvariant with respect to rotation of the input image. In addition, when the reference object is input, its orientationcan be determined from the real and imaginary parts of the correlation signal. Good discrimination between thereference image and other test images is observed, and the experimental results are in excellent agreement withtheoretical predictions.

1. INTRODUCTION

At low light levels, the spatial coordinates of detected photo-events can be used to calculate an estimate of the correlationfunction between a reference function (i.e., system impulseresponse) stored in computer memory and the classical in-tensity image associated with the photon-limited input im-age. Recent workl-3 has demonstrated that only a smallnumber of detected photoevents are needed to discriminatebetween a reference image and various test images. Becauseonly a few detected photoevents are needed for recognition,the time required to detect, process, and make a decision canbe quite short. When available photon-counting detectorsand position-computing electronics are used, the total com-putation time can be tens of milliseconds for many images.

When the image of the reference object is taken as thereference function, the correlation output corresponds tothat of a matched filter and is thus sensitive to changes inrotation, scale, and shift of the input image. An importantproblem in machine vision research is to devise methods forrecognition that are invariant with respect to rotation, shift,and change in scale of the input image. An elegant solutionto this problem is the method of invariant filtering. Mellintransforms, 4 '5 which have scale-invariant properties, andinvariant momentst68 which have scale-, position-, and rota-tion-invariant properties, have been shown to be useful forinvariant image recognition at high light levels. The use ofcircular-harmonic filters, initially suggested by Hsu, Arse-nault, and co-workers,9-1" and discussed by Wu andStark12"13 and Schils and Sweeney,' 4 has been demonstratedto be useful for coherent, rotation-invariant pattern recogni-tion; both optical and digital implementations have beeninvestigated.

Rotation-invariant filtering at low light levels is consid-ered in this paper. The method of circular-harmonic filter-ing is reviewed in Section 2. Circular-harmonic filtering ofphoton-limited images is considered in Section 3. Approxi-mate expressions are given for the probability-density func-tions of both the correlation signal and the squared modulusof the correlation signal when the number of detectedphotoevents is Poisson distributed as well as when the num-

ber of detected photoevents is fixed. In Section 4, laborato-ry experiments are described in which photon-limited inputscenes are cross correlated with a circular-harmonic compo-nent of a classical intensity reference image stored in com-puter memory. The experiments demonstrate that reliableimage recognition, regardless of the orientation of the inputimage, can be achieved with approximately only three timesthe number of detected photoevents required for thematched filter (registered image) case. In addition, whenthe reference image is input, its orientation angle can bedetermined from the real and imaginary parts of the correla-tion signal.

2. ROTATION-INVARIANT FILTERING BYUSING CIRCULAR HARMONICS

Any two-dimensional function f(r, 6) can be represented interms of its circular-harmonic components as follows:

f(r, 0) = 3 Fm(r)exp(im0), (1)m= -I

where

Fm(r) =- 2J f(r, 6)exp(-im6)dO.2x fo

(2)

In Eq. (1), Fm (r, ) = Fm(r)exp(imO) is said to be the mthcircular-harmonic component of the function f(r, 6). Otherinvestigations have shown9-14 that rotation-invariant filter-ing can be achieved by taking the reference function R(r, 0)to be the complex conjugate of a single (or multiple) circular-harmonic component(s) of the reference object, e.g.,

R(r, 0) = F*(r)exp(-im0). (3)

The cross correlation of an input function g(r, 0) with R(r, 0)is given by

2Cr OCQr, 0) = ' F.*(r + r', f)g(r', )rodr'dO.

Jo 10(4)

Using the expansion in Eq. (1) for g(r', ) and substituting for g(r', 0) in Eq. (4) yields

0740-3232/86/070954-10$02.00 © 1986 Optical Society of America

T. A. Isberg and G. M. Morris

Vol. 3, No. 7/July 1986/J. Opt. Soc. Am. A 955

C(r, 0) = 2r j F.(r + r')Gm(r')r'dr', (5)

which attains its maximum value when Gm(r) = Fm(r). Notethat only the mth circular-harmonic component of the inputcontributes to the correlation owing to the orthogonality ofthe theta integration. If the input function is rotated by anangle a with respect to the reference, the correlation signal is

C(r, a) = C(r, 0)exp(ima), (6)

which makes the squared modulus of the correlation signal

IC(r, a)21 = IC(r, 0)12, (7)

independent of the rotation angle of the input image. If thereference image is input, its orientation can be obtainedfrom the ratio of the real and imaginary parts of the correla-tion signal. The rotation angle a is given by

density function of the squared modulus of the correlationsignal when a complex circular harmonic of a classical inten-sity reference image is used as the reference function. Also,an expression is given for the probability-density function ofthe rotation angle when the reference image is input.

A low-light-level input image V(x', y') can be representedas a collection of Dirac delta functions,

N

V(x', y') = 3 e(x' - xi, y' - (

i=l

where (xi, yi) represent the coordinate location of the ithdetected photoevent and N is the total number of detectedphotoevents in the reference scene window. The cross cor-relation of V(x', y') with R(x', y') is then

N

C(x,y) = 3R(x + xi,y +yi),i=l

(10)

a = [1/m]tan'[ImC(r, a)}/Re{C(r, a)], (8)

which can be easily calculated when the correlation is imple-mented digitally. The rotation angle a is far more difficultto obtain in an optical implementation because most detec-tion schemes are intensity based.

The magnitude of the correlation peak in Eq. (7) willdepend on which order harmonic is chosen as the referencefunction. The optimum circular-harmonic reference func-tion can be determined by using the Hotelling trace criteri-on.15 Also, it is evident from Eq. (7) that the magnitude ofthe correlation signal, while being invariant with respect torotation, is dependent on the offset position, r. Moreover,the magnitude of C(r, a) is dependent on the location of thepoint on the reference image about which the reference cir-cular harmonic was expanded; this location is referred to asthe expansion center. The magnitude of C(r, a) will be amaximum only when the proper center is chosen as theexpansion center.9 The procedure for finding the propercenter involves a recursive algorithm that does not alwaysconverge for complicated objects.12 In addition, the propercenter may be different for different order harmonics, mak-ing the use of multiple harmonics less straightforward thanmight be expected. Several methods using the center ofmass of the object (rather than the proper center) as anexpansion center for multiple harmonics have been demon-strated to be effective in rotation-invariant recognitionproblems."1- 2 When the proper center is easily found, amethod involving decision making in multidimensional fea-ture spacel was shown to be useful. In Ref. 11 multipleharmonics, expanded about their proper centers, are used asreference functions.

3. IMAGE CORRELATION AT LOW LIGHTLEVELS

A low-light-level correlation signal can be obtained by crosscorrelating a photon-limited input scene with a referencefunction stored in computer memory. The statistics of thephoton-limited correlation signal are reviewed here for thecases of both real and complex reference functions. Ap-proximate expressions are given for the probability-densityfunctions of the correlation signal when the number of de-tected photoevents is Poisson distributed and when it isfixed. In addition, expressions are given for the probability-

where (x, y) represent the offset of the reference scene win-dow within the input scene. Note that the correlation signalis simply the summation of the values of the reference func-tion sampled by using the spatial coordinates of the detectedphotoevents.

A. Real Reference Function

Poisson-Distributed Number of Detected PhotoeventsIn Eq. (10), the photoevent coordinates (xi, yi) are indepen-dent random variables. The probability density function ofthe photoevent coordinates is given by

P(xi, Y) = V(xi, y,) IA V(x', y')dx'dy', (11)

where V(x', y') is the classical intensity of the input sceneand A denotes the area of the reference scene window. If thecorrelation signal is realized by detecting photoevents for aconstant integration time r, then the number of detectedphotoevents is also a random variable. When the scene isilluminated by a polarized thermal source (with the systemintegration time long compared with the coherence time ofthe source) or by a well-stabilized, single mode laser, then Nis Poisson distributed' 6 with Poisson parameter N, providedthat detection dead-time effects17 are negligible, i.e.,

p(N) = (N)Nexp-ftf/N!, (12)

where

N = I I V (x V, '), dx'dy' (13)

is the mean number of detected photoevents. In Eq. (13), qis the quantum efficiency of the detector, r is the integrationtime, h is Planck's constant, and v is the mean frequency ofthe quasi-monochromatic radiation.

The photon statistics in Eqs. (11) and (12) are used tocompute the statistics of the correlation signal, C(x, y). IfN is large and R(x', y') is real valued, then C(x, y) is real andapproximately Gaussian distributed, with mean value (C(x,y)), given by

(C(x, y)) = N J p(x', y')R(x + x', y + y')dx'dy', (14)

and variance


(9)

956 J. Opt. Soc. Am. A/Vol. 3, No. 7/July 1986

= NJ I p(x', y')R2 (x + x', y + y')dx'dy'. (15)

Note that (C(x, y) is proportional to the cross-correlationfunction of the classical-intensity input function V(x', y')with the reference function R(x', y').

Our detection criterion is based on the theory of hypothe-sis testing. Given a set of classical intensity input imagesand the reference image to be recognized, theoretical predic-tions for the mean value and variance of the correlationsignals are made by using Eqs. (14) and (15), in which thevarious input images are cross correlated with the referencefunction. By observing the area of overlap of the probabili-ty-density functions, one can determine the number of de-tected photoevents required to attain the desired probabili-ties of detection and false alarm. One then sets a thresholdcorrelation, CT, and uses the following decision criterion. Ifthe correlation signal (realized by collecting N photons in anintegration time r) is greater than the threshold CT, thereference image is said to be present in the input image. Ifthe correlation signal is less than CT, the reference image issaid to be absent from the input image. The probability ofdetection is proportional to the area under the probability-density function curve for the reference that is greater thanCT. Similarly, the probability of false alarm is proportionalto the area under any nonreference probability curve that isgreater than CT. Thus, with a single realization of the corre-lation signal, the presence or absence of a registered refer-ence image can be determined within known probabilities ofdetection and false alarm.

Fixed Number of Detected PhotoeventsIn the previous section, it is assumed that the correlationsignal is realized by detecting photoevents for a fixed timeinterval r. Hence, provided that the assumptions concern-ing the illuminating radiation are correct, the number ofdetected photoevents is a Poisson-distributed random vari-able. Mandel18 has suggested that an idealized light sourcethat emits a definite number of photons per message symbolcould provide the maximum information-carrying capacityfor an optical communication channel. We achieve a some-what analogous effect here by making the number of de-tected photoevents fixed rather than Poisson distributed.If the correlation signal is realized by detecting a fixed num-ber of photoevents N, the number of detected photoevents isno longer a random variable.

If the cross correlation of a photon-limited input scenewith a reference function is performed by detecting a fixednumber of photoevents N [see Eq. (10)], then, by the centrallimit theorem,' 9 the probability-density function of the cor-relation will approach a Gaussian distribution, assumingthat the number of detected photoevents is large, since thephotoevent coordinates are statistically independent. Themean value and variance of the correlation signal can becomputed directly. The mean value of the correlation signalis given by

/NC\

( C(x, y) = (, R(x +xi, y + Y i=l

(16)

where b... ) denote an ensemble average. Using Eq. (11) forthe probability-density function of the photoevent coordi-

nates, and performing the ensemble average term by term,one obtains

(C(x, y)) = N J f p(x', y')R(x + x', y + y')dx'dy'. (17)

Therefore the mean value of the correlation signal is thesame as in the case when N is Poisson distributed, if N isreplaced by N. The variance, u-2 is calculated in a similarmanner. By definition, q2 is given by

a = (C2(X, ) - (C(X, Y))2. (18)

Computing (C2(x, y)) directly, using Eq. (11), one finds

(C2(x, y)) = N J p(x yR2(x + x', y + y')dx'dy'

+ (N2 - N) [ IA p(x',y')R(x + x', y + Y')dx'dy'J.

Hence, from Eqs. (17)-(19), one finds

a2= N f p(x', y')R2 (x + x', y + y')dx'dy'

- N[J IA p(x', y')R(x + x', y + Y')dx'dy'J

(19)

(20)

Noting that the first term in Eq. (20) is simply the variancewhen the number of detected photoevents is Poisson distrib-uted and the second term is (1/N) (C(x, y))2, Eq. (20) can bewritten as

a2 = 2 (C(X, Y)) 2

5 N (21)

where 2 is defined to be the variance given in Eq. (15) withN = N. Therefore the variance is smaller when the numberof detected photoevents is fixed.

B. Complex-Valued Reference Function

N Poisson DistributedNow consider the case in which the reference function iscomplex (e.g. a circular-harmonic component of the refer-ence object) and the number of detected photoevents isPoisson distributed. The correlation signal C(x, y) is nowcomplex. For convenience, define

C(x, y) = C'(x, y) + ic(x, y), (22)

where C'(x, y) and C"(x, y) are the real and imaginary partsof the correlation signal. Both the joint probability-densityfunction and the joint moments of the real and imaginaryparts of the correlation signal can be derived by using thecharacteristic function, in a manner similar to that used inthe real case.'

The joint characteristic function for the real and imagi-nary parts of the correlation signal is given by

4(o', w") = (expji[w'C'(x, y) + w"C'(x, y)]}). (23)

When the counting distribution in Eq. (12) is valid, thecharacteristic function can be derived in a manner similar tothat of the case of the real reference function. 20 One findsthat



t(co', W) = exp [fN J dx'dy'p(x', y')(exp[i[c' RefR(x', y')j

+ w"ImR(x',y'))]-1)] (24)

C(x, y) = C'(x, y) + i"(xy),

where

Q=C'-(C")O.'

where p(x', y') is given in Eq. (11), and Re{R(x', y')J andIm{R(x', y')j denote the real and imaginary parts, respective-ly, of the reference function. The joint moments of thecorrelation signal are computed using the relation

(C'm x, y)C"(x, y)) = (-i)O+nm+n'( ' . (25)do)mco"O n | '=wo=O

Using Eqs. (24) and (25), the mean values of the real andimaginary parts of the correlation signal, (C') and (C"), arefound to be

(C'(x, y)) = N J J p(x', y')ReR(x + x', y + y')Jdx'dy' (26)

and

(C"(x,y)) = N J A pax', y')Im{R(x + x', y + y')jdx'dy',

(27)

respectively. The variances of the real and imaginary partsof the correlation signal, cr'2 and cr"2, are given by

O=N JI J p(x', y') [RefR(x + x, y + y')f]2dx'dy' (28)

and

U/=N 2 f J p(x', y') [ImJR(x + x', y + y')fl2 dx'dy', (29)

respectively. The correlation coefficient p, defined as

p(C'C") - (')(C"1)

/a" (30)

is found to be

The joint characteristic function of the real and imaginaryparts of (x, y) is given by

Y cc") = expi[7 + C;)J}Ixep( NJ Jx'iLdy'p- , y) {'

( A I I ( 0-1+ &o Im{R(x, y) I

'Ii)] ') (35)

In an analogous manner to the real case,1 if one expands theexponential in the integral in Eq. (35), integrates term byterm, and takes the limit as N approaches infinity, one ob-tains, using Eqs. (26)-(31),

'tq(c', ") = exp {-2 [cO/2 + WA"2 + 2Pcc'w"I}- (36)

Hence, when N is large, the real and imaginary parts of thecorrelation signal are approximately jointly normally dis-tributed, with the form

1 f -1 c (C- (c)) 2

(C, C ) 2rU'a"(1 - p2)1 /2 exp 2(1 - 2) L a2

- 2p (C' - (C'))(C"' - (C''))UA//

(37)

where (C'), (C"), a-', ", and p are given in Eqs. (26)-(31). Ifthe reference function is the complex conjugate of the mth

N J A p(x', y')ImR(x + x', y + y')ReJR(x + x', y + y')jdx'dy'

A'-

The joint probability-density function for the real andimaginary parts of the correlation signal is obtained by tak-ing the inverse Fourier transform of the characteristic func-tion, i.e.,

P(C', C") = -J J dc'dco'"(c', w)expJ-i[co'C'(x, y)

+ "C"(x, y)]}. (32)

Unfortunately, the Fourier inverse of Eq. (32) cannot beobtained in general. If the average number of detectedphotoevents N is large, it is straightforward to derive a limitform for the characteristic function and hence the probabili-ty-density function.

Note that in Eqs. (26)-(31), when N is large, the momentsof the correlation signal tend to infinity. This difficulty isavoided by making the following change of variables: Let

circular harmonic of a reference image f(r, 0), then (C) =(C') + i( C ) is given in polar coordinates by

27rN exp{im} J FP*(r + r')Vm(r')r'dr'

(38)(C(r, a)) =J J r'dr'dO'V(r', ')

Note that the squared modulus of the numerator in Eq. (38)attains its largest value when the input image is the same asthe reference image, just as in Eq. (5). Also note that themean value of the complex correlation signal is proportionalto the high-light-level correlation signal, in analogy with thereal case. Because the orientation angle again appears onlyas a phase term, the squared modulus of (C(r, a) ) is invari-ant with respect to rotation of the input image. The use ofmultiple circular harmonics as reference functions improvesthe reliability of the detection process.1 1 2 However, for

(33)

C, = C" - (C")a"

(34)

p - (31)


. 2+ (C- - (C-�)

O.//2


simplicity in illustrating the operating principle of the meth-od at low light levels, we use only one component.

Because it is the squared modulus of the correlation signalthat is rotation invariant, the probability-density functionof the squared modulus of the correlation signal is requiredto predict probabilities of detection and false alarm for agiven average number of detected photoevents N. To findthe probability density function for Cl2, the followingchange of variables is made in Eq. (37): Let

C' = IC cosy, C" = C sin y, (39)

where Cl 2 and -y are given by

ici 2 = C'2 + C/

2, = tan-' (C2).

The marginal density function for the squared modulus ofthe correlation signal P(l Cl 2) can then be written as

are the only random variables in the correlation signal. Inaddition, as N becomes large, the joint density function ofthe real and imaginary parts of the correlation signal againapproaches a joint normal distribution by the central limittheorem. However, since the real and imaginary parts of thecorrelation signal may be correlated, the joint density func-tion will have a correlation coefficient, just as in the casewhen Nwas Poisson distributed. Note that this correlationdoes not affect the conditions for application of the centrallimit theorem because the coordinates of the detected pho-toevents are statistically independent. Therefore the jointdensity for the real and imaginary parts of the correlation isagain given by Eq. (37). The correlation coefficient and thevariances of the real and imaginary parts of the correlationsignal can be computed directly, using a procedure similar tothat given in Subsection 3.A for the case of a fixed number of

P(I 12) = 441 J, dyexp-1 [I C 12 cosN(y) -{ 2(1 - p2) L

21 C I (C') cos(y) + (C) 2

-2 ICl 2 coS(y) sin(-y) -ICI[(C")cos(-y) + (C')sin(y)] + (C)(C")

I I C I2 sin 2(-r) -2 (C")| C I sin(y) + (+ ~~~~~~f/2

Unfortunately, a closed-form solution to the integration inEq. (41) does not exist in general.21 However, the integra-tion is readily performed numerically. Equation (41) can beused to generate theoretical predictions of the probabilitydistributions for the squared modulus of the correlationsignal when the various input objects are correlated with areference circular harmonic.

If the input image is the reference image, then one cancompute the probability-density function for the rotationangle a by noting that -y = ma [see Eqs. (8) and (40)]. Themarginal density function for the rotation angle of the input,P(a), is obtained using Eqs. (37), (39), and (40):

P 4r(1 -p 2)1/2*'cT" A dl C 2 exp{ 2 [lC 2cos2(m)

detected photoevents. The mean values of the real andimaginary parts of the correlation signal are

(C'(x, y)) = N f I p(x', y')Re$R(x + x', y + y')jdx'dy'

(43)and

(C"(x, y)) = N f A p(x', y')ImjR(x + x', y + y')jdx'dy',

(44)

-2 IC (C') cos(mcv) + (C') 2

- 2 ICI 2 cos(ma) sin(ma) -IC [(C"1) cos(mae) + (C') sin(ma)] + (C')(C")

+ lCI2 sin2 (ma) - 2(C"')ICIsin(ma) + -, )OX2 J

By using Eq. (42) it is possible to predict the accuracy in thedetermination of the rotation angle a for a given averagenumber of detected photoevents.

N FixedAs in the case of the real reference function, the statistics ofthe photon-limited correlation signal C(x, y) change whenthe number of detected photoevents N is fixed for eachrealization of the correlation signal. The mean and variancecan be calculated in a manner similar to that in Subsection 3.A.

The correlation signal C(x, y) is again defined by Eqs. (10)and (22). If the number of detected photoevents N is fixed,the spatial coordinates (xi, yi) of the detected photoevents

(42)

respectively. The variances of the real and imaginary partsof the correlation signal are given by

i2 = N f f p(x', y')[ReIR(x + x', y + y')}2 dx'dy' - (C')

(45)

and

an2 = N JJ (x', y')[Im{R(x + x', y+ y-)1]2dxdy_ (Ca)2

(46)

(41)



The correlation coefficient p is

p = [N/'"] IA p(x', y')ImIR(x + x', y + y'))RejR(x + x', y + y')dx'dy'- (47)f A Nocr'

where u'2 and tr"2 are given in Eqs. (45) and (46). The signal and the marginal density function for the rotationmarginal density of the squared modulus of the correlation angle a are given by Eqs. (41) and (42), respectively, using

the moments of the correlation signal given in Eqs.

Digital (43>(47).Computer

Posputer 4. EXPERIMENTAL RESULTS

,,,. - 1 Experiments were performed to test the low-light-level per-

- - / J formance of the complex circular-harmonic filter for both

111111 o / 21 'a' o |/:Eimage discrimination and determination of the rotation an-gle of a reference object. The system configuration is shownschematically in Fig. 1. In the experiments, 35-mm formatinput scenes illuminated by an incoherent light source were

imaged onto a two-dimensional, photon-counting detector.A neutral density of 10 was inserted between the input scene

Den tya /and the detector to obtain an acceptable count rate; in this

case the count rate was approximately 30,000 cycles per

second (cps). The detector is capable of detecting photo-

rhozon-countig events at rates up to 105 cps. A dark count of 60 cps was

Pheotoncounting observed at room temperature.The detector2 2 consists of a bi-alkali photocathode in cas-

cade with a stack of microchannel plates followed by a resis-Imaging Lens tive anode. The microchannel plates provided an electron

INCOHERENT gain of approximately 107. The output from the resistive

ILLUMINATION anode was sent to a position computer23 that computed the

Fig. 1. System diagram for low-light-level image recognition. spatial coordinates of each detected photoevent. Image res-

OBJECT

Fig. 2. Objects and asso-IF ^ S ^ f ciated second-order circu-F2 (re ~ ~f lar harmonics. First row:

classical intensity images;second row: squared mod-ulus of respective secondcircular-harmonic compo-

- nents; third row: real-_ -==_- X _ ->; parts of second circular-

>__ _____ harmonic components;____ ~~~~~~~~~~fourth row: imaginary

_ _ __-l_ ___ parts of second circular-

Re| F2 (rse)j harmonic components.Re) F (r~)air

Im F2 (r3e))



Table 1. Expected Values and Standard Deviations of the Squared Modulus of the Correlation SignalaInput Image Theory Experiment

V(x', y') (ICky a (IC2)a

Vise grips: a = 00 1.60E + 09 7.85E + 07 1.51E + 09 7.47E + 07Vise grips: a = 900 1.60E + 09 7.85E + 07 1.58E + 09 7.67E + 07Vise grips: a = 180° 1.60E + 09 7.85E + 07 1.58E + 09 7.68E + 07Vise grips: a = 2700 1.60E + 09 7.85E + 07 1.58E + 09 7.55E + 07

Wrench: a = 00 9.71E + 08 6.07E + 07 9.69E + 08 6.39E + 07Pliers: a = 00 7.06E + 08 5.36E + 07 7.03E + 08 4.55E + 07

a The number of detected photoevents is N = 3000.

II.

tion signal obtained by using each input scene for N = 3000detected photoevents are shown in Table 1. Note that themean value of the squared modulus of the correlation signalobtained with the vise grips at each orientation is invariantwith respect to rotation of the input object.

A histogram of the correlation values obtained using 3000detected photoevents is shown in Fig. 3. In the figure, Iindicates the range of correlation values observed with thevise-grips input rotated by 90 deg with respect to the refer-

1.231C12[X 109]I

Fig. 3. Histogram of squared modulus of correlation signal whenthe input image is I, vise grips, rotated by 90 deg with respect to thereference; II, crescent wrench; and III, pliers. The number of de-tected photoevents is N = 3000.

olution has been reported as high as 500 X 500 with the ITTdetector.2 4 In our experiments the spatial coordinates ofeach photoevent were digitized to 8-bit accuracy and sent toa digital computer for processing.

Figure 2 shows the input images and their associated cir-cular-harmonic components used in the correlation experi-ments. The vise grips were taken as the reference objects,with a pliers and a crescent wrench as test objects.

In all the experiments performed, the complex conjugateof the second circular harmonic of the vise grips (computedabout the centroid of the object) was chosen as the referencefunction [see Eq. (1) and Fig. 2]. The correlation signalswere realized by detecting a fixed number of photoevents,as discussed in Subsection 3.B. The correlation signalwas computed by using Eq. (10), and the rotation angle ofthe input with respect to the reference was determined byusing Eq. (8).

To demonstrate the performance of the circular-harmonicfilter at low light levels, the complex correlation signal wascomputed with the reference image input rotated about itscentroid through angles of 0, 90, 180, and 270 deg withrespect to the reference; the test images were input orienta-ted as shown in Fig. 2. One thousand measurements of thecorrelation signal were madd for each input scene by usingdifferent values of N (N = 500, 1000,2000, and 3000 detectedphotoevents).

The mean values and standard deviations of the probabili-ty-density function for the squared modulus of the correla-

0.

= 1000

= 500

Fig. 4. Histogram of values for the rotation angle a when the visegrips are input rotated by 90 deg with respect to the reference. Thenumber of detected photoevents is N = 3000.

__

N=3000

0.6

-0.4

-0.2

i585 B7. 92.5 95C

Fig. 5 ROC curves for the vise grips and the crescent wrench. Thevise grips are rotated by 90 deg with respect to the reference.

r1.0

N=3000

go

NU2

a.

0.75

0.50

0.25


I A0.613 1.a4 2.45


5.0

-N =3000

N=3000

.0

.5

1.25 2

0.8 1.2 1.6 2.0 2A

IC12[.101

Fig. 6. Comparison of theoretical probability-density functions of the squared modulus of correlation signal when the number of detectedphotoevents is fixed (N = 3000) and when the number of detected photoevents is Poisson distributed (N = 3000). The input image is the visegrips rotated by 90 deg with respect to the reference.

ence, II shows the range of correlation values with thewrench input, and III shows the range of correlation valueswith the pliers input. The solid curves are theoretical pre-dictions of the probability density function for the squaredmodulus of the correlation signal I C12 = I C(0, 0)12. Thesecurves are obtained by performing the integration in Eq.(41) numerically, using expressions for the mean values andvariances given in Eqs. (43)-(47).

Figure 4 shows a histogram of values of the orientationangle a obtained from 1000 measurements of the complexcorrelation signal realized by using 3000 detected photo-events with the vise grips rotated by 90 deg with respect tothe reference. The rotation angle a is obtained from thecorrelation signal by using Eq. (8). The solid curve is thetheoretical prediction of the probability-density functionP(ca) for the rotation angle a. The solid curve is obtained byusing Eqs. (42) and (43)-(47). In the experiment, the meanof P(a) was 89.6 deg and the standard deviation was 0.705deg.

In Fig. 5, the probability of false alarm and the probabili-ty of detection (ROC curves) are plotted for the vise grips(input rotated by 90 deg) and for the crescent wrench fordifferent values of N. With a fixed number of detectedphotoevents, the standard deviation of the squared modulusof the correlation signal is less than for the case when N isPoisson distributed [see Eqs. (44) and (45)], which results ina greater separation of the density functions. The differ-ence in the variances of the squared modulus of the correla-tion signal when N is Poisson distributed and when N isfixed is illustrated in Fig. 6. The two density functions inFig. 6 are theoretical predictions for the squared modulus of

the correlation signal when the vise grips are input rotatedby 90 deg with respect to the reference.

5. DISCUSSION

The experimental results in Section 4 show that rotation-invariant image recognition can be achieved with photon-limited images. Rotation-invariant image recognition ofphoton-limited images has obvious applications in standardlow-light-level situations, such as night vision or low-doseradiological imaging. In addition, the photon-countingtechniques reported here may be useful in standard high-light-level machine vision applications as well. In manycases it may be faster for a computer to recognize an imageusing photon-counting techniques as opposed to digitizingand processing the entire high-light-level scene directly. Byusing this photon-counting technique, the computer pro-cesses the minimum amount of information required tomake an accurate recognition decision. Since only a fewthousand detected photoevents are required to recognizemany images, the time required to recognize an image maybe of the order of tens of milliseconds, given the presentmaximum detection rate, which is -105 cps. An additionaladvantage to phpton-counting methods is the small amountof computer hardware necessary to perform the photon-limited correlations. Computing the correlation signal sim-ply involves looking up the value of the reference function atthe detected photoevent coordinates and summing the re-sult in an accumulator. Mapy microcomputers have suffi-dient power to compute the correlation signal at rates 105cps.



One advantage in the use of circular-harmonic compo-nents of a reference image for rotation-invariant image rec-ognition is that the rotation angle of the reference object canbe obtained from the phase of the correlation signal. How-ever, depending on the order of the harmonic used as thereference function, there are certain ambiguities in the rota-tion angles that are determined from the correlation signal.For example, let m = 2 in Eq. (6), and assume that thereference objects is input, which makes C(r, 0) real. Equa-tion (6) can then be rewritten as

C(r, a) = C(r, 0)[cos(2a) + isin(2a)]. (48)

Note that the phase of the correlation signal will have thesame value if a' = a + 7r. Hence the orientation of thereference is not uniquely determined. (This result can alsobe obtained by observing the 180-deg rotational symmetryof the real and imaginary parts of the second-harmonic com-ponents in Fig. 2.) When the correlations are performed atlow light levels, this problem is easily solved. One simplyrotates the coordinates of a small number of detected pho-toevents by vr/4 rad and observes the change (if any) in thesign of the phase of the correlation signal. By using this signinformation the rotation angle can be uniquely determined.It should be stressed that the coordinates of only a few tensof detected photoevents need be rotated and that the anglethrough which they are rotated depends on the order of theharmonic that is used as the reference.

There is no difficulty in using multiple circular-harmoniccomponents of a reference image as reference functions forrotation-invariant image recognition at low light levels.One simply keeps multiple harmonics stored in computermemory and computes separate correlation signals arisingfrom each reference function. The recognition decision canthen be made following either of the methods given in Refs.11 and 12.

In practice, when realizing the photon-limited correlationsignal, a small number of dark counts are present. This canbe easily accounted for in the theoretical predictions for theprobability-density functions of the correlation signal.3

However, in our experiments the number of dark counts wassmall and had no significant effect on the correlation signal.

Finally, regarding changes in image position or scale, notethat a second reference function that is scale invariantand/or position invariant (see Refs. 4-8) may be used inconjunction with single (or multiple) circular-harmoniccomponent(s) of a reference image. Alternatively, one mayconsider some type of search algorithm, in which a globalsearch of the input scene is made by shifting the referencewindow in computer memory and tracking in the direction ofmaximum correlation signal.

For example, if one considers the input image to be com-posed of m independent resolution cells and shifts the centerof the reference window to each of the m locations in theinput image, then the probabilities of detection Pdm and falsealarm Pfam of making one detection of a reference objectwithin the entire input image become25' 26

1-PPd. = 1 1 - Pd, a (1- Pfr >(49)

fal

Pfa. = 1 - (1 - Pfd)m , (50)respectively, where Pd, are the probabilities of detection and

false alarm at a particular location. If one specifies therequired probabilities of detection and false alarm for a setof input images with m independent resolution cells, Eqs.(49) and (50) can be used to solve for Pd, and Pfm. Thenumber of detected photoevents required to produce Pd, andPfa1 can be computed as specified in Subsection 3.A.

6. SUMMARY

Rotation-invariant image recognition at low light levels wasinvestigated. Photon-limited input scenes from a two-di-mensional photon-counting detector are cross correlatedwith a reference function in computer memory. The theoryof rotation-invariant filtering by using a complex circular-harmonic component of a classical intensity reference imageis reviewed in Subsection 2A. The correlation output isgiven in Eq. (5). In Subsection 3.A the statistics of thephoton-limited correlation signal are reviewed for the case ofa real reference function when the number of detected pho-toevents is Poisson distributed. The mean value and vari-ance are given by Eqs. (14) and (15), respectively. Also inSubsection 3.A, expressions are given for the mean and vari-ance when the number of detected photoevents is fixed;these results are shown in Eqs. (17) and (20). In Subsection3.B, approximate expressions for the density function forthe photon-limited correlation signal using a complex refer-ence function are given for the cases in which the number ofdetected photoevents is Poisson distributed and fixed. Ex-perimental results are discussed in Section 4. Histograms ofthe correlation signal are shown in Fig. 4, and mean valuesand standard deviations of the squared modulus of the cor-relation signal for a particular light level are summarized inTable 1.

ACKNOWLEDGMENTS

This research was supported in part by the New York StateCenter for Advanced Optical Technology and the Joint Ser-vices Optics Program.

REFERENCES

1. G. M. Morris, "Scene matching using photon-limited images,"J. Opt. Soc. Am. A 1, 482-488 (1984).

2. G. M. Morris, "Image correlation at low light levels: a computersimulation," Appl. Opt. 23, 3152-3159 (1984).

3. G. M. Morris, M. N. Wernick, and T. A. Isberg, "Image correla-tion at low light levels," Opt. Lett. 10, 315-317 (1985).

4. D. Casasent and D. Psaltis, "Position, rotation, and scale invari-ant optical correlation," Appl. Opt. 15, 1795-1799 (1976).

5. D. Casasent and D. Psaltis, "Scale invariant optical transform,"Opt. Eng. 15, 258-261 (1976).

6. M. K. Hu, "Visual pattern recognition by moment invariants,"IRE Trans. Inf. Theory IT-8, 179-187 (1962).

7. R. Y. Wong and E. L. Hall, "Scene matching with invariantmoments," Comput. Graphics Image Process. 8, 16-21 (1978).

8. R. C. Gonzalez and P. Wintz, Digital Image Processing (Addison-Wesley, London, 1977), Chap. 7.

9. Y.-N. Hsu, H. H. Arsenault, and G. April, "Rotation-invariantdigital pattern recognition using circular harmonic expansion,"Appl. Opt. 21, 4012-4015 (1982).

10. Y.-N. Hsu and H. H. Arsenault, "Optical pattern recognitionusing circular harmonic expansion," Appl. Opt. 21, 4016-4019(1982).

11. Y.-N. Hsu and H. H. Arsenault, "Pattern discrimination bymultiple circular harmonic components," Appl. Opt. 23,841-844 (1984).



12. R. Wu and H. Stark, "Rotation-invariant pattern recognitionusing a vector reference," Appl. Opt. 23, 838-840 (1984).

13. R. Wu and H. Stark, "Rotation-invariant pattern recognitionusing optimum feature extraction," Appl. Opt. 24, 179-184(1985).

14. G. F. Schils and D. W. Sweeney, "Rotationally invariant correla-tion filtering," J. Opt. Soc. Am. A 2, 1411-1418 (1985).

15. K. Fukanaga, Introduction to Statistical Pattern Recognition(Academic, New York, 1972), p. 261.

16. M. Bertolotti, in Photon Correlation and Light Beating Spec-troscopy, H. Z. Cummins and E. R. Pike, eds. (Plenum, NewYork, 1974), Chap. 2.

17. G. Bedard, "Dead-time corrections to the statistical distribu-tions of photoelectrons," Proc. Phys. Soc. 90, 131-141 (1967).

18. L. Mandel, "Fundamental limits on information capacity of anoptical communication channel," Kinam Rev. Fis. Ser. C 5,213-232 (1983).

19. B. V. Gnendenko, Theory of Probability (Chelsea, New York,1963), Chap. 8.

20. A. Papoulis, Probability, Random Variables, and StochasticProcesses (McGraw-Hill, New York, 1984) p. 382.

21. K. S. Miller, Multivariate Distributions (Krieger, New York,1975), p. 28.

22. Detector model F4146M, ITT Electro-Optical Products Divi-sion, Tube and Sensor Laboratories, Fort Wayne, Ind.

23. Model 2401 Position Computer, Surface Science Laboratories,Inc., Mountain View, Calif.

24. C. Firmani, E. Ruiz, C. W. Carlson, M. Lampton, and F. Paresce,"High-resolution imaging with a two-dimensional resistive an-ode photon counter," Rev. Sci. Instrum. 54, 570-574 (1982).

25. D. J. Goodenough, "Objective measures related to ROC curves,"Proc. Soc. Photo-Opt. Instrum. Eng. 47, 199-204 (1975).

26. H. H. Barrett and W. Swindell, Radiological Imaging (Academ-ic, New York, 1981), Vol. 2, pp. 626-628.


rotation-invariant image recognition at low light levels

Documents