Voronoi Projection-Based Fast Nearest-NeighborSearch Algorithms: Box-Search and MappingTable-Based Search TechniquesV. Ramasubramanian* and K. K. Paliwal†*Computer Systems and Communications, Tata Institute of Fundamental Research,Homi Bhabha Road, Bombay—400 005, India; and †School of MicroelectronicEngineering, Griffith University, Brisbane, Queensland 4111, Australia

Ramasubramanian, V., and Paliwal, K. K., VoronoiProjection-Based Fast Nearest-Neighbor Search Algo-rithms: Box-Search and Mapping Table-Based SearchTechniques, Digital Signal Processing 7 (1997), 260–277.

In this paper we consider fast nearest-neighbor searchtechniques based on the projections of Voronoi regions.The Voronoi diagram of a given set of points provides animplicit geometric interpretation of nearest-neighborsearch and serves as an important basis for severalproximity search algorithms in computational geometryand in developing structure-based fast vector quantizationtechniques. The Voronoi projections provide an approxi-mate characterization of the Voronoi regions with respectto their locus property of localizing the search to a smallsubset of codevectors. This can be viewed as a simplifiedand practically viable equivalent of point location usingthe Voronoi diagram while circumventing the complexityof the full Voronoi diagram. In this paper, we provide acomprehensive study of two fast search techniques usingthe Voronoi projections, namely, the box-search and map-ping table-based search in the context of vector quantiza-tion encoding. We also propose and study the effect andadvantage of using the principal component axes for datawith high degree of correlation across their components, inreducing the complexity of the search based on Voronoiprojections. r 1997 Academic Press

1. INTRODUCTION

1.1. Fast Nearest-Neighbor SearchThe aim of nearest-neighbor search is to deter-

mine the closest point to a query point among Npoints in K-dimensional space. The problem of find-ing the nearest neighbor in multidimensional space

arises in several areas such as pattern classification,nonparametric estimation, information retrieval frommultikey data bases, and image and speech datacompression using vector quantization. The computa-tional complexity of nearest-neighbor search is amajor problem in these areas, when the size N of thepoint set to be searched becomes very high. As aresult, the problem of developing algorithms for fastnearest-neighbor search has attracted significantattention in these areas. In this paper we considerfast nearest-neighbor search in the context of vectorquantization encoding.

Vector quantization is a powerful data compres-sion technique used in speech coding, image coding,and speech recognition [1, 3, 18–20, 25, 28]. Vectorquantization has the potential to achieve codingperformance close to rate-distortion limit with in-creasing vector dimension. However, the utilizationof vector quantizers is severely limited by its encod-ing complexity which increases exponentially withdimension K. Vector quantization encoding is theminimum-distortion quantization of a vector x 5 (x1,. . . , xK) (referred to as the test vector), using a givenset of N K-dimensional codevectors (called the code-book C 5 5ci6i51,...,N, of size N ), under some distancemeasure d(x, y). This involves finding the nearest-neighbor of x in C, given by q(x) 5 cj : d(x, cj) # d(x,ci), i 5 1, . . . , N, which requires N vector distancecomputations d(x, ci) using the exhaustive full-search for a codebook of size N. The codebook size Nis related to the dimension K and bit-rate r (bits/sample or bits/coordinate) as N 5 2Kr and the com-plexity of encoding x increases exponentially with Kand r. The encoding complexity also constitutes themost computation intensive step in iterative vector

DIGITAL SIGNAL PROCESSING 7, 260–277 (1997)ARTICLE NO. SP970300

1051-2004/97 $25.00Copyright r 1997 by Academic PressAll rights of reproduction in any form reserved.

260

quantizer design [27] and the problem of reducingthe computational complexity of vector quantizationencoding has received considerable attention for realiz-ing the full potential of vector quantization and forrendering it practically useful for real-time applications.

The fast algorithms reported so far for fast nearest-neighbor search can be broadly categorized undertwo search constraint paradigms: (i) fast searchbased on elimination rules and (ii) Voronoi partition-based structured search.

1.1.1. Elimination-Based Fast SearchElimination-based fast search exploits some prop-

erties of the distance measure used in the nearest-neighbor definition. Here, fast search is achieved bya ‘‘quick-elimination’’ or ‘‘throwaway’’principle, wherecodevectors which cannot be nearer to the given testvector than the current nearest-neighbor are elimi-nated without incurring the cost of a distance compu-tation. These algorithms typically employ efficientelimination schemes such as the partial distanceelimination [4, 10, 31] elimination based on thehypercube [10, 42, 49] and absolute-error-inequalityapproximations of the Euclidean distance [41], andtriangle-inequality based elimination [8, 9, 16, 17,21–23, 30, 33–36, 39, 40, 44, 45].

1.1.2. Voronoi Structure-Based Fast SearchThis paradigm is based on the Voronoi partition,

which is a geometric construct describing the nearest-neighbor mapping of the input space for a given setof codevectors. The Voronoi partition permits thesearch to be viewed as a point location problem ofdetermining which Voronoi region contains the testvector with the corresponding codevector being thedesired nearest-neighbor. The Voronoi diagram servesas an important basis for several proximity searchtechniques in computational geometry ([15, 26, 32,48] give an extensive overview of these problems).

The techniques based on the Voronoi diagram forvarious search problems is a classic example of ageneral problem solving paradigm called the ‘‘locus’’method in computational geometry. In this method,the search space is partitioned into a number ofequivalence classes—a set of nonoverlapping regionssuch that the answer is invariant for all query pointsthat fall in the same region. This method is particu-larly applicable for search in the so-called ‘‘repetitivemode’’ where arbitrarily long sequence of queries areencountered. This is typically the situation in vectorquantization or pattern classification applications,where it is required to encode or classify test (orfeature) vectors arriving as long sequences in timeusing a given codebook or classifier.

Various solutions based on the Voronoi diagramhave been reported in the computational geometryliterature for two (planar) and higher dimensions.While several optimal solutions which are also prac-tically realizable exist for the planar nearest-neighbor problem, all currently known solutions forlarge dimensions pose major practical difficulties interms of high preprocessing, storage, and overheadcosts in constructing and using the Voronoi diagramdespite their excellent asymptotic search time re-sults obtained theoretically. Consequently, emphasisin fast search rests on finding efficient geometricstructures and space constraints that characterizethe Voronoi partition to reduce the point locationproblem to a very small Voronoi region subset bycomputationally inexpensive operations.

1.1.3. Scope and Organization of the PaperIn this paper, we investigate fast nearest-neighbor

search techniques based on the projections of theVoronoi regions. The projections of the Voronoi re-gions were proposed and used for fast search in anearlier work [10]. However, this basic paradigm hasnot received adequate attention with respect to thecomplexity of the higher dimensional Voronoi dia-gram and algorithms which use the full Voronoidiagram. The Voronoi projections provide an approxi-mate characterization of the Voronoi regions withrespect to their locus property in localizing thesearch to a small subset of codevectors; this resultsin a simplified and practically viable equivalent ofpoint location which circumvents the complexity ofthe complete Voronoi diagram.

In this paper, our objective is to highlight the basicefficiency of using the Voronoi projections and toshow that fast search based on the Voronoi projec-tions, using appropriate data structures, can resultin very practical and highly efficient fast searchalgorithms. In this respect, this paper provides acomprehensive study of two methods for using theVoronoi projections for fast search, namely, box-search and mapping table-based search. We considertheir average and worst-case complexity perfor-mance, and in addition provide a detailed study oftheir overall performance which includes the averageand worst-case overhead computations, storage andpreprocessing cost. (A condensed version of this paper isreported elsewhere as a correspondence item [50].)

The paper is organized as follows. In Section 2, wedescribe the basic structure of Voronoi diagram-based fast search and the complexity of algorithmsbased on the full Voronoi diagram. In Section 3, weintroduce the basic search structure based on theprojections of the Voronoi regions emphasizing the

261

main computational step involved in finding thereduced subset of codevectors whose Voronoi projec-tions contain the test vector. In Section 4, we de-scribe a simple search scheme termed ‘‘box-search’’(BS) which treats the projections directly as hypercu-boid approximations of the Voronoi regions. In Sec-tion 5, we discuss in detail the projection methodproposed by Cheng et al. [10] which was the firstwork to consider the use of the Voronoi projections.In Section 6, we propose and study the use of theprojections on the principal component directions forreducing the complexity of search for the box-searchand mapping table-based search procedures. In Sec-tion 7, we give simulation results comparing theperformances of the box-search and the mappingtable-based search in the context of vector quantiza-tion encoding of speech waveform.

2. FAST NEAREST-NEIGHBOR SEARCH USINGTHE VORONOI DIAGRAM

Given a set of N codevectors C 5 5ci6i51,...,N, of sizeN, along with a distance measure d(x, y), x, y [ RK,the RK space is partitioned into N disjoint regions,known as Voronoi regions, with each codevectorassociated with one region. The Voronoi region Vj

associated with a codevector cj contains all points in RK

nearer to cj than any other codevector and is thenearest-neighbor locus region of cj. If the nearest-neighbor of x in C is q(x) 5 cj : d(x, cj) # d(x, ci), i 5 1,. . . , N, the Voronoi region Vj is defined as Vj 5 5x [ RK :q(x) 5 cj65 5x [ RK : d(x, cj) # d(x, ci), i 5 1, . . . , N6.

The Voronoi region Vj is thus a convex regionformed by the intersection of the half-spaces 5H(cj,ci), i 5 1, . . . , N, j Þ i6, given by

Vj 5 >jÞi

H(cj, ci),

where H(cj, ci) is the set of points closer to cj than ci, i.e.,

H (cj, ci) 5 5x [ RK : d (x, cj) # d (x, ci)6.

For the Euclidean distance, H(cj, ci) is the half-space containing cj formed by the perpendicularbisector plane of cjci, the line connecting cj and ci.Figure 1a illustrates the idea of the Voronoi regionassociated with a point in the plane (R2) for theEuclidean distance and Fig. 1b shows the Voronoipartition for the given set of points.

By the definition of the Voronoi diagram, giventhat a test vector x is contained in a Voronoi regionVj, the associated codevector cj will be the nearest-neighbor of x. Fast nearest-neighbor search using

the Voronoi diagram can therefore be viewed as a‘‘locus method’’ which involves use of fast methodsfor searching the Voronoi subdivisions of the Voronoidiagram by means of a point location algorithm todetermine the Voronoi region containing the testvector. Fast search using the Voronoi diagram thusconsists of two steps:

1. Constructing the Voronoi diagram: Apreprocess-ing phase of constructing the Voronoi diagram, repre-senting it in the form of a graph data structure andassociating with each facet (for instance, vertex,edge and region, in the 2-dimensional case) of thediagram, the set of closest codevectors.

2. Point location: Given the Voronoi subdivisions,determine the facet of the subdivision which con-tains the test vector and the associated codevector isthe desired nearest-neighbor.

2.1. Complexity of Algorithms Based on FullVoronoi Diagram

Though the use of the Voronoi diagram provides anefficient paradigm for nearest-neighbor search to becarried out as a point location search for lowerdimensions (such as for the 1-dimensional and pla-nar cases), the construction and usage of the Voronoi

FIG. 1. Example of Voronoi partition.

262

diagram for higher dimensions is beset with imple-mentational difficulties, mainly due to the fact thatfor a given set of points, the number of itemsrequired to describe the Voronoi diagram growsexponentially with dimension.1 Thus, the main com-plexity of algorithms based on the full Voronoidiagram arises from the preprocessing stage inconstructing the Voronoi diagram of N input pointswhich has an V(NK/2) worst case time and storage, inaddition to the cost required in organizing the Voro-noi diagram for point location. While the basiccomputation of the Voronoi diagram invariably facesthis intrinsic exponential complexity, the use of thediagram for fast nearest-neighbor search has furtherpractical difficulties since the known search (point-location) algorithms which use the Voronoi diagramin higher-dimensional subdivisions have similar ex-ponential dependence on dimension in their averageand worst-case query time performance bounds andpreprocessing and storage costs which grow exponen-tially with dimension [11–13, 47].

3. VORONOI PROJECTION-BASED FAST SEARCH

In this section, we consider in detail, search tech-niques based on the projections of the Voronoi re-gions on the coordinate axes. The Voronoi projectionsprovide an approximate characterization of the Voro-noi regions and allow for a simplified, but practicallyviable equivalent of point location while circumvent-ing the complexity of the complete Voronoi diagram.Though the projections constitute only a very partial

information about the Voronoi regions they retaintheir locus property of localizing the search to asmall subset of codevectors and we see from empiri-cal results for large dimensions and typical codebooksizes that the Voronoi projection-based search has anexcellent complexity reduction potential.

Let Vi be the Voronoi region associated with thecodevector ci. Then the projection of Vi on thecoordinate axis j is a projection interval Pi

j, withlower and upper boundary points (Pi,L

j , Pi,Uj ) given by

Pi,Lj 5 min

x[Vixj

and

Pi,Uj 5 max

x[Vixj.

This represents the two hyperplanes x : xj 5 Pi,Lj

and x : xj 5 Pi,Uj , normal to the jth coordinate axis.

The K projections (Pi,Lj , Pi,U

j ), j 5 1, . . . , K define a setof 2K hyperplanes which bounds the Voronoi regionVi by the smallest hypercuboid region

Bi 5 5x : (Pi,Lj # xj #Pi,U

j ), j 5 1, . . . , K 6.

Each codevector has such a hypercuboid approxima-tion of the Voronoi region associated with it and werefer to this as the Voronoi-box (or ‘‘box’’) of thecodevector. The hypercuboid box has sides parallel tothe coordinate axes and is a simple geometric approxi-mation of the Voronoi region.

3.1. Basic Structure of Search UsingVoronoi Projections

Clearly, Vi # Bi and hence, if x [ Vi, then x [ Bi.Conversely, if x Ó Bi, then x Ó Vi. Thus anycodevector whose hypercuboid box does not containthe test vector cannot be the nearest-neighbor of thetest vector since its Voronoi region does not containthe test vector. However, if x [ Bi, then it indicatesthat the corresponding Voronoi region may containx. Since the boxes are not disjoint, several boxes cancontain a test vector and all such codevectors need tobe considered as candidates for the nearest-neighborsearch. Identifying the boxes which contain the testvector results in generating a candidate set of code-vectors, only one of which will correspond to theVoronoi region that actually contains the test vector.The actual nearest-neighbor can be determined by afull-search in this candidate set.

The fast search using the Voronoi projections canbe viewed as consisting of two steps:

1. Determine C8(x) 5 5ci: x [ Bi6: the candidate set ofcodevectors whose boxes Bi contain the test vector x.

1 A Voronoi diagram in K-dimensions is a cell complex made upof convex polytopes which consist of faces of dimensions varyingfrom 0 to K, such as vertex (0-dimension), edge (1-dimension),region (2-dimension), facet ((K 2 1)-dimension), and cell (K-dimension). In general, a face in the cell complex is called a k-face,(0 # k # K), if it is of dimension k. Thus, vertex, edge, region, facetand cell are 0-face, 1-face, 2-face, (K 2 1)-face, and K-face, respec-tively. The Voronoi diagram, being a cell complex formed byhyperplane arrangements, is ideally represented and stored inthe form of an incidence graph [15]. Here, each face of the diagramis associated with a node of the incidence graph and two nodes areconnected if the corresponding faces are incident. In order toprovide the actual spatial correspondence, each node in the graphcorresponding to a cell stores the coordinates of the point corre-sponding to the cell. Klee [24] showed that the maximum numberof faces grows exponentially in K. Tight upper bounds obtained forthe number of k-faces of the Voronoi diagram of N-points is inO(Nmin5K112k,K/26), for 0 # k # K [15, 38]. Thus, the Voronoi diagramof N points in 3-dimensions may have O(N2) edges [32] and forhigher dimensions (K . 3), the maximum number of edges andvertices of the Voronoi diagram of N points is O(NK/2). Algorithmswhich construct the Voronoi diagram of N points in K-dimensionalspace require O(N(K11)/2) time in the worst-case and O(NK/2)storage [2, 5–7, 37, 46]. A detailed formal treatment of higherdimensional Voronoi diagram and its application to variousproximity search problems can be found in [15, 32].

263

2. Perform a full-search in the candidate set C8(x)to obtain the actual nearest-neighbor of x.

This approach can be viewed as a simplification ofthe problem of point location directly in the Voronoiregion to a problem of point location within thesmallest hypercuboid box enclosing the Voronoi re-gion. However, point location within the boxes doesnot yield a unique solution for the Voronoi regioncontaining the test vector, thus necessitating a searchamong the candidate set of codevectors generated bypoint location among the boxes. If the size of thiscandidate set of codevectors C8(x) is small in compari-son to the full codebook size N, the search willachieve significant complexity reduction.2

The basic structure of search using the Voronoi projec-tions is illustrated by an example in 2-dimension in Fig.2. This figure shows the Voronoi diagram of 16 points,Vi, i 5 1, . . . , 16 within a range of interest ABCD. Theprojections of a Voronoi region forming the box enclosingit are illustrated for V14. Consider a test vector x 5 (x1,x2) ([ V10). The test vector is located inside the boxes ofV2, V4, and V10, marked as abcd, efgh, and ijkl, respec-tively. These are the only boxes containing the testvector and hence form the final candidate set C8(x),which has to be searched to find the actual nearest-neighbor c10 with a reduced complexity of 3 distancecomputations as against 16 for the full-search.

The main complexity of the fast search using theVoronoi projections, in terms of the number of dis-tances computed, is the size of the candidate set C8 to

2 We note here that a related algorithm which is also based onthe idea of using a simple geometric approximation of the Voronoiregion was proposed in [43]. Here, the smallest hypersphereenclosing the Voronoi region and centered at the codevector isdetermined first for each of the codevectors. Each codevector isthus implicitly associated with the smallest hypercube containingthis sphere. Given a test vector, only those codevectors whose

hypercubes contain the test vector are considered for a full-search.However, such a hypercube is invariably larger than the boundinghypercuboid boxes considered here, and the spherical Voronoi approxi-mation algorithm [43] therefore has intrinsically poorer complexityreduction efficiency than Voronoi projection-based search.

FIG. 2. Example of projection method.

264

be searched in step 2. This is determined by theaverage number of boxes that contain a test vectorfor a given test data distribution. However, the totalcomplexity of the algorithm is highly dependent onthe cost of step 1 in determining the candidate set ofcodevectors for a given test vector, i.e., the complex-ity of finding which boxes contain the test vector. Themain complexity of step 2 is determined only by thedistribution of the codevectors and the test vectorsand is invariant to the method employed to performstep 1. Step 1 essentially contributes to the overheadcomputation of the fast search and it is important toobtain efficient procedures for carrying out this step inorder to reduce the overall complexity of the algorithm.

In the following sections, we consider in detail twosearch techniques for using the projection informa-tion to carry out step 1. These are namely, (i)box-search and (ii) mapping table search.

4. BOX-SEARCH

The hypercuboid box Bi is a simple geometricobject for point location and the final set of codevec-tors can be determined by directly checking if thetest vector lies within the box of each of the codevec-tors. This requires checking whether the jth compo-nent of the test vector is contained within theprojection boundaries (Pi,L

j , Pi,Uj ) of the Voronoi re-

gion Vi on the jth coordinate axis requiring twocomparisons per coordinate. A codevector can berejected when any of the test vector’s coordinatesfalls outside the corresponding Voronoi projection,i.e., xj , Pi,L

j or xj . Pi,Uj for any j. If a test vector

passes the test on all the coordinates (when all theprojections of a Voronoi region contain the correspond-ing coordinates of the test vector), then the test vectorlies inside the hypercuboid box approximation of theVoronoi region and the corresponding codevector has tobe considered for full-search. This search is as follows:

Box-searchdmin 5 `do i 5 1, . . . , N

if x_in_box(i) then Box-testd 5 d(x, ci)if d , dmin then nn 5 i; dmin 5 d

endifenddoprocedure x_in_box(i)do j 5 1, . . . , K

if xj , P ij ,L then x_in_box(i) 5 .false.; return

if xj . P ij ,U then x_in_box(i) 5 .false.; return

enddox_in_box(i) 5 .true.return

4.1. Storage and Computational Overheads ofBox-Search

The boxes are stored as a codevector indexed table;i.e., the 2K projection values are stored along withthe corresponding codevector, requiring 2KN realwords of additional memory to store the projectionvalues of the N codevectors.

The search described above performs a simpleelimination test (box-test) for each codevector in afull-search structure before computing its distanceto the test vector x. The distance between the testvector x and codevector ci is computed only if thecodevector is not rejected by the box-test x_in_box(i)which checks whether x [ Bi. A codevector can berejected after c comparisons, where 1 # c # 2K, andany codevector which passes the test for distancecomputation incurs a ‘‘box-test’’ cost of 2K compari-sons. For a given test vector, if ci is the number ofcomparisons for codevector ci, the total cost of box-search for the test vector is c(x) 5 oi51

N ci. The cost pervector component (or coordinate) is c(x)/K (withbounds N/K # c(x) # 2N) and the average of thisover a large set of test vectors characterizes theactual overhead comparison cost for the box-search.The quantity c(x)/N, on the other hand, gives theaverage number of comparison tests done per codevec-tor. This can have a value of 1 to 2K and whenobtained as an average over a large set of test vectorsindicates how quickly a codevector is eliminated onan average within the 2K comparisons required toreject a codevector in the worst case. These quanti-ties characterize the actual overhead computationsand the relative efficiency of box-search.

5. THE MAPPING TABLE-BASED SEARCH

In the projection method [10], the projections ofthe Voronoi regions are reorganized to form a ‘‘map-ping table’’ in a preprocessing stage which is thenused to generate the candidate set of codevectors fora given test vector. In this method, the space ispartitioned into a rectangular cell partition usingthe projections. Given a test vector, a rectangular cellthat contains the test vector is first identified usingonly scalar comparison operations on the structuredprojections. The candidate codevectors associatedwith the cell containing the test vector are thenformed by intersection operations on precomputedtables corresponding to each coordinate axes. Afull-search within this small set of candidate codevec-tors yields the nearest-neighbor of the test vector.

265

The N overlapping projection intervals Pij 5 (Pi,L

j ,Pi,U

j ) i 5 1, . . . , N generate 2N projection boundaries(y1

j , y2j , . . . , y2N

j ) with a natural ordering (y1j #

y2j # · · · # y2N

j ). This partitions the jth coordinateaxis into 2N 2 1 contiguous intervals 5I1

j , I2j , . . . ,

I2N21j 6, where Im

j 5 5(ymj , ym11

j )6. Each of these inter-vals Im

j is associated with a set, Smj , of indices of the

Voronoi region whose projection intervals Pij par-

tially or totally overlap with Imj , i.e., Sm

j 5 5i : Pi,Lj #

ymj and Pi,U

j $ ym11j 6.

Given a test vector x 5 (x1, . . . , xK), let Imj 5 5(ym

j ,ym11

j )6 be the interval containing xj, i.e., ymj # xj #

ym11j . Denoting this interval as Ij(x), let Sj(x) be the

index set associated with this interval. Any testvector whose jth component xj lies in the intervalIj(x) can lie within any one of the Voronoi regionswhose projection overlaps with Ij(x). The set ofindices Sj(x) associated with Ij(x) thus correspondsto the candidate codevectors which could be thenearest-neighbor of the given test vector x. For agiven test vector x, each coordinate axis j generatessuch a set of candidate codevectors Sj(x), whoseVoronoi region may contain the test vector withrespect to the jth coordinate axis. The Voronoiregion actually containing the test vector shouldhave all its K projection intervals contain the corre-sponding test vector component and the nearest-neighbor codevector index will be contained in theintersection of the K sets Sj(x), j 5 1, . . . , K. Thisintersection set is the set of candidate indices whichneed to be examined by a full-search to obtain theactual nearest-neighbor.

The (2N 2 1) contiguous intervals 5I1j , I2

j , . . . ,I2N21

j 6, on each of the K coordinate axes can be seen aspartitioning the space into (2N 2 1)K rectangularcells. Identification of the K intervals Ij(x), j 5 1,. . . , K containing the test vector locates the testvector in a hyperrectangular cell H(x) 5 Pj51

K Ij(x).Since all scalar points in the interval Ij(x) areassociated with a common set of candidate indicesSj(x), all vector points inside the cell H(x) are alsoassociated with a single set of final candidate codevec-tors given by the intersection of Sj(x), j 5 1, . . . , K.(Note that the cell containing the test vector H(x) isactually contained within the volume of intersectionof the Voronoi boxes containing the test vector x andthat the set of final candidate codevectors obtainedas the intersection of Sj(x), j 5 1, . . . , K is identicalto that obtained by box-search.)

This represents an organization of the projectionsinto a large number ((2N 2 1)K) of disjoint rectangu-lar cells, each of which have an invariant set ofcandidate codevectors which can be seen to be on thelines of the ‘‘locus’’ method described in Section 1.1.2.

The set associated with each of the (2N 2 1)K hyper-rectangular cells can in principle be precomputedand stored for each cell directly. By this, a very smallset of candidate codevectors will be available forfull-search subsequent to K binary searches (each ofcost log (2N)) to find the K intervals Ij(x), j 5 1, . . . ,K containing the test vector projections. This, how-ever, will be at the cost of an exhorbitant storage of(2N 2 1)Kg, where g is the average size of thecandidate set associated with the rectangular cells.

Alternately, this is obtained at lesser storage bycomputing the intersections of individual intervalsubsets Sj(x) for every test vector x. This, however,introduces the additional computational overheadcost in forming the intersection of K subsets.

We illustrate this search for the example shown inFig. 2. This shows the Voronoi diagram of 16 points,Vi, i 5 1, . . . , 16, and their corresponding projectionsPi

j, i 5 1, . . . , N, j 5 1, 2 within a range of interestABCD. The projections are shown explicitly only forsome of the Voronoi regions to maintain clarity whilethe complete set of projection boundaries formingthe contiguous intervals are shown on each axis.Given the test vector x 5 (x1, x2) ([ V10), the inter-vals I1(x) and I2(x) containing x1 and x2 are identifiedfirst. These intervals are associated with the indexsets S1(x) 5 5c2, c3, c4, c10, c136 and S2(x) 5 5c1, c2, c4,c6, c7, c9, c10, c116, respectively. The intersection ofthese two sets gives the final candidate set 5c2, c4,c106, which has to be searched for the actual nearest-neighbor c10.

5.1. Storage and Computational Overheads ofMapping Table Search5.1.1. Organization, Preprocessing, and Storage

OverheadsCheng et al. [10] have used the following data

structures to perform fast search using the mappingtable at reduced storage complexity. A mapping tablefor coordinate axis j consists of the ordered bound-ary points 5ym

j 6m51,...,2N describing the 2N 2 1 inter-vals 5Im

j 6m51,...,2N21 and the associated index sets5Sm

j 6m51,...,2N21. These are obtained in a preprocessingstep for each coordinate axis and K such tables, onefor each coordinate axis, comprise the completemapping table. The mapping table entries can beorganized in two ways: (i) index map (IM) and (ii) bitmap (BM).

Index map. In the IM, the indices are stored asnumerical values. If am

j is the size of the index set Smj ,

the average number of indices per interval for thejth axis is aj 5 [1/(2N 2 1] om51

2N21 amj and the overall

average number of indices per interval for the entiremapping table is a 5 (1/K) oj51

K aj. The storage of the

266

mapping table in the index mode requires 2NK realwords for the ordered projections, (2N 2 1)aK and(2N 2 1)K integer words for the index sets and theindex set size respectively. The total storage is then(3 1 a)NK real words. This is referred to as the fulltable (FT) organization in [10].

Cheng et al. also considered a reduced table (RT)organization which reduces the storage to 5KN words.This makes use of the fact that an interval boundaryis either a lower or an upper projection of a Voronoiregion, and consequently the index sets on adjacentintervals differ by only one element; i.e., if aninterval boundary ym is a lower projection Pi,L

j ofsome Voronoi region Vi, then the index set Sm

j

corresponding to the interval Imj 5 5(ym

j , ym11j )6 is

Smj 5 Sm21

j 1 5i6 and if ym is an upper projection Pi,Uj ,

then Smj 5 Sm21

j 2 5i6. Therefore, only the differingelement i has to be stored as a signed index sm 5 1ior sm 5 2i for each interval Im

j 5 5(ymj , ym11

j )6, 1/2representing ym to be a lower/upper Voronoi projec-tion. Then, the index set Sm

j of any interval Imj can be

formed recursively starting from the index s1 on thefirst interval, i.e., Sm

j 5 Sm21j 1 5sm6 with S1

j 5 5s16.This form of storage, therefore, represents a signifi-cant reduction in storage in comparison to the full-table organization, since only one signed index isstored per interval. This, however, contributes toincreased overhead operations to form the indexcandidate sets Sm

j , j 5 1, . . . , K recursively afterdetermining the interval Im

j containing xj. The em-pirically estimated cost for forming the K index setsfor a test vector is KN logical and comparisonoperations.

Bit map. In the BM organization, the indices areposition coded in a N bit word with the nth bit as oneif the nth codevector is a candidate and zero other-wise. The BM storage requires (3 1 N/b)NK words,where b is the processor word length. These storagerequirements can be very high for large dimensionsand codebook sizes.

5.1.2. Computational Overheads of MappingTable SearchIn order to consider the computational overheads

of the fast search based on the mapping table, wegive here a search procedure for finding the nearest-neighbor of a test vector x:

(a) Find the interval Imj 5 5(ym

j , ym11y )6 in each

coordinate j containing the component xj of the testvector, i.e., ym

j # xj # ym11j .

(b) Find C8 5 >j51K Sm

j , the final set of candidatecodevectors C8 from intersection of the K indexcandidate sets Sm

j , j 5 1, . . . , K. Let Smj 5 5i1

j , i2j , . . . ,

ibjj 6, where bj 5 0Sm

j 0. This can be obtained as follows:

C8 5 5 6C[i] 5 0, i 5 1, . . . , Ndo j 5 1, . . . , K

do k 5 1, . . . , b jC[ik

j ] 5 C[ikj ] 1 1

if C [ ikj ] 5 K then C8 5 C8 1 5ık

j 6enddo

enddo

(c) Carry out a full-search within this candidateset C8 to obtain the actual nearest-neighbor of x.

The first step (a) of interval location is carried outby a binary search in the mapping table of each of thecoordinate axis, incurring a total cost of K log (2N)comparisons. In the case when the mapping table isorganized as an IM/RT, an additional step is neces-sary to form the index candidate sets Sj(x), j 5 1,. . . , K by the recursion Sm

j 5 Sm21j 1 5sm6, with S1

j 55s16 after determining the interval Im

j containing xj.In step (b), we have given a simple, but efficient,

procedure for finding the intersection of subsets.This is a variant of the shift method employed byYunck [49] and will be referred to henceforth as theintersection count (IC) procedure (introduced in arelated data structure [35]). This procedure is basedon the observation that the candidate set required bythe intersection of the K index sets consists of indicescommon to all the K index sets and hence will appearexactly K times in the collection of the K sets. Allother indices which do not belong to the K-setintersection will occur less than K times. Therefore,it suffices to keep count of the number of times aparticular index has occurred in all the K sets andthe intersection set is formed by those indices whichhave a count of K. This intersection count procedurerequires an additional memory of N scalar array ofresolution (log aKb) bits. (In the method used byYunck [49], the above procedure would correspond tousing a K-bit register in which the count is main-tained by shifting the contents of the register to theleft with the full-count value after K shifts corre-sponding to 2K.)

The intersection complexity of the IM storage canbe quantified in the following manner. For a testvector x, let Ij(x) be the interval containing xj and letSj(x) be the index set associated with this interval.Denoting the size of the set Sj(x) by bj (with bounds1 # bj # N), the intersection complexity is propor-tional to b 5 oj51

K bj, irrespective of the nature of theintersection procedure carried out on the K setsSj(x), j 5 1, . . . , K. The average b, b, obtained over alarge set of test vectors characterizes the basic

267

intersection complexity in the IM mode and hasbounds, K # b # KN. The complexity of the intersec-tion-count procedure given above is, therefore, binteger increment and scalar comparison operationsin performing the counting and in checking for countvalue K to form the final intersection indices. Theaverage intersection complexity is therefore d 5(b 1 N) scalar operations per vector, including thecost of initializing the count array of size N to zeroand has bounds (K 1 N) # d # (K 1 1)N.

Based on empirical observation, Cheng et al. esti-mated the intersection complexity to be about KNcomparisons for the IM case and KN/3 logical opera-tions for the BM case. The BM case was noted tohave a smaller intersection complexity as it allowsthe intersection computation using logical AND op-erations on the position encoded bit representationof the candidate indices.

The total overhead computational complexity ofthemappingtable-based intersectionprocedure (asgivenby steps (a) and (b)) is, therefore, (K log 2N 1 b 1 N)scalar operations per vector, and that of box-search isc(x) scalar comparisons per vector, with bounds N #c(x) # 2KN. From simulation results (Section 7.2),we see that the average value of this cost forbox-search asymptotically saturates to a value of2N/K per vector component, which decreases lin-early with dimension. This is significantly less thanthe corresponding bounds of [log 2N 1 (K 1 N)/K,(log 2N 1 (K 1 1)N/K] of the mapping table-basedintersection procedure, which has an average com-plexity bound of O(N) which increases linearly withdimension. Moreover, we also see that (b 1 N) scalaroperations/vector of the intersection count proceduregiven above represents a overhead complexity whoseaverage bound is lower than (and worst-case boundcomparable to) the intersection complexity of KN forthe IM mode observed by Cheng et al. [10].

PRINCIPAL COMPONENT ROTATION

In this section, we propose and study the influenceof principal component rotation of the codevectors inreducing the complexity of different search algo-rithms using the Voronoi projections. The complexityof the projection-based search depends mainly on theextent of intersection between the Voronoi boxes.This in turn reflects in the extent of overlap of theprojections of the Voronoi regions on the coordinateaxes. We show that the projections of the Voronoiregions obtained using the principal component direc-tions as the new ‘‘rotated’’ coordinate system reduce

the relative overlap between the Voronoi boxes signifi-cantly and improve the efficiency of the projection-based search in decreasing the main search complex-ity of number of distance computations, i.e., byreducing the size of the final set of candidate codevec-tors. We also show that the box-search schemederives advantage by the principal component rota-tion in reducing both the main search complexityand the overhead computational complexity andthat, in comparison, the mapping table-based inter-section search suffers increased storage and compu-tational overheads.

The K principal component directions of a givenset of vector points are the directions of the eigenvec-tors of the covariance matrix of the points [14]. If li,i 5 1, . . . , K (with l1 $ l2

. . . $ lK) are the eigen-values of the covariance matrix of the points, thenthe principal component directions ei, i 5 1, . . . , Krepresent the orthogonal directions of maximumvariance in the data, the variance of the projectionsof the points along the direction ei being proportionalto the corresponding eigenvalue li. e1 is referred toas the first principal component direction (or coordi-nate axis), e2 the second principal component direc-tion, and so on. e1 is the direction of maximumvariance of the projections of the given set of pointson it, and e2 is the direction perpendicular to e1 withthe largest variance of the point projections on it. Ingeneral, ei is the direction perpendicular to e1, . . . ,ei21 having the largest variance of the projections ofthe points on it. ei is also the direction which is thesolution for the best perpendicular least-square er-ror (or eigenvector) fit perpendicular to e1, . . . , ei21,with minimum cross-sectional volume orthogonal toit in the subspace ei11, . . . , eK. The principal compo-nent rotation is particularly applicable in the case ofspeech waveform vector quantization where the vec-tors are highly correlated across their components.Figure 3a shows the high degree of correlationbetween the components of speech waveform vectorsfor the 2-dimensional case.

6.1. Effect of Principal Component Rotation onExtent of Projection Overlap

In view of the basic property of the principalcomponent directions, the projections of the Voronoiregions on the first principal component axis willhave the maximum spread (more than on any otherdirection) and the projection intervals will be maxi-mally separated from each other. This implies mini-mum overlap between the various projection inter-vals. (This can be easily visualized by consideringthe extreme case where the codevectors are aligned

268

on a straight line along some direction. This willthen be the direction of the first principal componentwith some variance and other directions having zerovariance. This actually degenerates to a 1-dimen-sional case with the Voronoi projections correspond-ing to contiguous and disjoint intervals.) This will inturn reflect in the smallest a j, the average size of theindex sets of the contiguous intervals formed by theoverlapping projections. The extent of overlap on thesecond principal component axis will be more thanon the first, and in general, the overlap will increasefrom principal coordinate axis 1 to K. To illustratethis, Fig. 3 shows the Voronoi boxes of the codevec-tors of a codebook of size 32 in 2-dimensions for therotated and the unrotated case alongwith 150,000speech waveform vectors used for obtaining theprojection estimates. This also shows how the datadetermine the box estimates and the extent of inter-

section between the various Voronoi boxes; thespreading out of the projections along the firstcoordinate axis in the rotated space and the reduc-tion in the extent of overlap due to rotation can benoted here.

In Fig. 4, we show the effect of rotation on theextent of overlap across the coordinates by plottinga j and a j as a function of coordinate axis j 5 1, . . . ,K with and without rotation for dimension K 5 8 andcodebook size N 5 1024. a j is the average index setsize and a j is the maximum index set sizemax1#m#2N21 am

j . It can be seen that for the unrotatedcase, aj and a j are rather unchanging across thecoordinates, implying the codevector orientation tobe along a direction which generates similar projec-tion overlap on all the coordinate axis. For therotated case, a j and a j values are much smaller than

FIG. 3. Voronoi boxes of codevectors for codebook size N 5 32 and dimension K 5 2 for (a) without rotation along with 150,000vectors of speech waveform vectors used for obtaining the projection estimates and (b) with rotation.

269

for the unrotated case in the first and second princi-pal coordinate axes. However, for the higher coordi-nate directions in the rotated space, a j and a j

increase to values larger than for the unrotated case.This is due to the fact that the variances of thecodevector projections along these directions aresmaller than the corresponding coordinates in theunrotated case. These directions belong to the sub-space orthogonal to the first and second rotated axesand therefore have a smaller projected cross-sec-tional volume than for any other direction. Hence theprojections on the directions in this subspace areconfined within smaller cross sections with a resul-tant increase in their overlap.

In general, the number of principal componentdirections which have smaller a j and aj values thanthe corresponding unrotated coordinates and thedifferences in magnitudes of the index set sizesbetween the rotated and unrotated cases depend onthe correlation between the vector components andthe relative magnitudes of the variances along theprincipal coordinate directions. A highly favorablesituation is one where a large number of principalcomponent coordinates have significantly higher vari-ances (resulting in lower a j and a j values) than thecorresponding unrotated axes.

7. SIMULATION RESULTS

In this section, we present simulation resultsobtained in the context of vector quantization ofspeech waveform to characterize the complexity ofthe box-search and the mapping table-based searchmethods. First we describe the means of obtainingthe Voronoi projections of a given set of codevectors.The projection of each Voronoi region can in principlebe obtained either by analytical procedures such aslinear programming techniques or by constructingthe Voronoi diagram, from which only the vertexinformation of each Voronoi polytope is used furtherfor determining the projections of each Voronoi re-gion. But, as seen earlier, these can involve consider-able computational cost. Alternately, the projectionboundary points can be determined from the ‘‘ex-treme’’ vectors in the respective Voronoi region withrespect to each coordinate axis; this is done by firstgenerating a Voronoi ‘‘cluster’’ around each codevec-tor using a Monte Carlo approach of encoding a largeamount of training data (or uniformly distributedpoints) and then finding the extreme vectors of eachcluster with respect to each coordinate axis. Theprojection estimates thus obtained by encoding thepoints that fall within the region are then

Pi ,Lj 5 min

x[Rxj : q(x) 5 ci

and

Pi,Uj 5 max

x[Rxj : q(x) 5 ci ,

where R represents the training set (or the domain ofinterest in the RK space in the case of uniformlydistributed points).

This constitutes the main preprocessing step whichinvolves encoding of a large training data set and theuse of scalar comparison operations to update andmaintain the extreme projections of vectors on thecoordinate axes for each Voronoi region. The esti-mates so obtained are approximate, but have beenfound to be quite acceptable when obtained usingsufficiently large data. If the size of the training set Ris n, then the cost in obtaining the Voronoi projectionestimates is nN distance computations (to find q(x),;x [ R) and 2Kn scalar comparisons to update theprojections as Pi,L

j 5 min 5xj : q(x) 5 ci6 and Pi,Uj 5

max 5xj : q(x) 5 ci6, j 5 1, . . . , K, ;x [ R. This is aneffective method to obtain the projection estimatessince it can be performed as a part of the codebook

FIG. 4. Average and maximum index set sizes (aj, aj) forcoordinate axis j 5 1, . . . , K for dimension K 5 8 and code-book size N 5 1024. No Rotation: without principal componentrotation. Rotation: with principal component rotation.

270

design at the end of the LBG algorithm [27]. Here,the nearest-neighbor of the vectors in the trainingset is available at the end of the algorithm. It isnecessary only to maintain the updates of the projec-tions, the preprocessing cost then being only 2Knscalar comparisons. In the present simulation, theprojections were obtained using 150,000 vectors ofspeech waveform for dimensions K 5 2, 4, 6, 8, and10 and codebook sizes N 5 32, 64, 128, 256, 512, and1024. All results shown here were obtained using50,000 vectors from inside the data used for estimat-ing the Voronoi projections.

7.1. Main Complexity of Box-Search and MappingTable Methods

Here, we give the main complexity of the fastsearch using the projections in terms of the numberof distances computed, which is the size of thecandidate set of codevectors obtained by box-searchor the mapping table-based intersection method.Figure 5 shows the average and maximum numberof codevectors searched by the projection methods fordimension K 5 8 and codebook sizes N 5 32, 64, 128,256, 512, and 1024. Figure 6 shows the average andmaximum number of codevectors searched for dimen-sions K 5 2, 4, 6, and 8 and codebook size N 5 1024.This is the average and maximum complexity of thefast search by projection method in terms of vectordistances computed per vector. The excellent complex-ity reduction of the projection method with respect tofull-search can be noted in all these cases. (Thecurves marked as ‘‘Rotation’’ correspond to the projec-

tions obtained using the principal component rota-tion which is discussed in a later section; here, weconcern ourselves only with the results using theprojections obtained on the regular ‘‘unrotated’’ spaceas in Cheng et al. [10] and we refer only to the curvesmarked ‘‘No rotation’’ in Figs. 5 and 6 and in subse-quent figures.)

7.2. Overhead Complexities of Box-Search andMapping Table Methods

Here, we compare the computational overheadcost of the intersection count procedure using themapping table and that of the box-search. As seenearlier (in Section 5.1.2), the intersection countincurs a total of (K log 2N 1 b 1 N) scalar opera-tions (such as comparisons and increment) per vec-tor and the box-search incurs c(x) scalar compari-sons per vector. These are also numericallycomparable directly since the constituent scalaroperations are same and of similar cost. Figures 7aand 7b respectively show the average and maximumcosts per vector-component incurred by the IC andthe BS procedures for dimension K 5 8 and differentcodebook sizes N 5 32, 64, 128, 256, 512, and 1024.The box-search can be seen to have a smaller (butcomparable) overhead cost than the intersectioncount procedure using the mapping table.

In Figs. 8a and 8b, we show the above costs forcodebook size N 5 1024 and different dimensionsK 5 2, 4, 6, 8, and 10. Here, it is important to observethat while the cost of IC increases with dimension,that of BS shows a marked decrease. This is due to

FIG. 5. Average and maximum number of codevectorssearched (NC, NC) by projection method (intersection count/box-search procedures) for dimension K 5 8 and codebooksizes N 5 32, 64, 128, 256, 512 and 1024. No Rotation: withoutprincipal component rotation. Rotation: with principal compo-nent rotation.

FIG. 6. Average and maximum number of codevectorssearched (NC, NC) by projection method (intersection count/box-search procedures) for dimensions K 5 2, 4, 6, 8 andcodebook size N 5 1024. No Rotation: without principalcomponent rotation. Rotation: with principal component rota-tion.

271

the fact that in the case of the intersection proce-dure, the cost is proportional to the number of indexsets and their sizes. An increase in dimension in-creases the number of index sets and their sizes,thereby resulting in an increase in the computa-tional overhead cost with increase in dimension. Onthe other hand, in the case of box-search, as thedimension increases, there is an increase in thedegree of freedom for a box to not contain a given testvector with a consequent reduction in the averagenumber of comparisons required per coordinate inthe box-test before a codevector can be rejected. Inaddition, we have shown in Figs. 7 and 8 the cost of N

operations per vector component (plot marked asIM/FT), reported to be required in [10] for thiscomputation using the image-map/full-table organi-zation of the mapping table. It can be noted that, forthe various codebook sizes and dimensions shown inFigs. 7 and 8, the overhead cost of the IC procedureproposed here is considerably less than the N opera-tions per vector component observed in [10].

In Figs. 9 and 10, we show the average number ofcomparisons required per codevector in the box-search. As mentioned earlier, this measures howquickly a codevector is eliminated within the maxi-mum 2K comparisons required in the worst case todetermine if a point lies within a box. Figure 9 showsthis for dimension K 5 8 and codebook sizes N 5 32,

FIG. 7. Average (a) and maximum (b) overhead computa-tions/vector-component of intersection count (IC) procedureand box-search (BS) procedure for dimension K 5 8 andcodebook sizes N 5 32, 64, 128, 256, 512, and 1024. NoRotation: without principal component rotation. Rotation:with principal component rotation.

FIG. 8. Average (a) and maximum (b) overhead computa-tions/vector component of intersection count (IC) procedureand box-search (BS) procedure for dimension K 5 2, 4, 6, 8, 10and codebook size N 5 1024. No Rotation: without principalcomponent rotation. Rotation: with principal component rota-tion.

272

64, 128, 256, 512, and 1024, while Fig. 10 shows thisfor codebook size N 5 1024 and dimensions K 5 2, 4,6, 8, and 10. In both cases, the average box-test costcan be seen to be considerably less than the worst-case 2K cost. It is important to note here that theaverage value for the various dimension and code-book sizes asymptotically saturates to a value of 2,placing the empirical cost of box-search per vectorcomponent at 2N/K, which is significantly less thanthe cost of N per vector component of the mappingtable-based intersection procedure for large dimen-sions.

In Fig. 11, we show the histogram of the number ofcomparisons per codevector for dimension K 5 8 andcodebook size N 5 1024. As noted earlier, this has arange of 1 to 2K. It can be noted that the histogramhas a sharp high probability mode at the lower endfor values 1 and 2, falling drastically beyond this,indicating that most of the codevectors, for a largerepresentative test data, tend to be eliminated within2 comparisons; this fraction is about 70% in thehistogram shown. This histogram clearly demon-strates the excellent efficiency of the box-searchelimination in having a low overhead computationalcost. It can also be noted from this figure that theprobability (number of comparisons/codevector 5 16)corresponds to the number of codevectors for which2K comparisons were computed and incur the fullbox-test cost and is the upper bound of the number ofcodevectors whose distances are computed; thisworst-case probability of 2K comparisons is about0.02 and corresponds to approximately 10 codevec-tors (also shown as the worst-case complexity ofbox-search in Table 1).

Figure 12 compares the total storage required bythe mapping table with that of box-search for dimen-sion K 5 8 and codebook sizes N 5 32, 64, 128, 256,512, and 1024. The box-search storage of 2KN can beobserved to be significantly less than that of theIM/FT ((3 1 a)NK) and BM/FT ((3 1 N/b)NK withb 5 32).

7.3. Effect of Principal Component Rotation on theComplexity of Box-Search and MappingTable Methods

Here, we show the number of codevectors exam-ined (final candidate set size) for various dimensions

FIG. 9. Average number of overhead comparisons per code-vector in box-search for dimension K 5 8 and codebook sizesN 5 32, 64, 128, 256, 512, and 1024. No Rotation: withoutprincipal component rotation. Rotation: with principal compo-nent rotation.

FIG. 10. Average number of overhead comparisons percodevector in box-search dimension K 5 2, 4, 6, 8, 10 andcodebook size N 5 1024. No Rotation: without principalcomponent rotation. Rotation: with principal component rota-tion.

FIG. 11. Histogram of the number of overhead computationsper codevector in box-search for dimension K 5 8 and code-book size N 5 1024.

273

and codebook sizes in Figs. 5 and 6 with and withoutprincipal component rotation. The rotated case canbe seen to have a consistently lower complexity thanthe unrotated case. This is so, despite the increasedoverlap in the higher principal coordinate directions,because the reduced overlap in the lower principalcoordinate directions produces a resultant reductionin the overlap between the actual Voronoi boxes,which in turn reflects in the final candidate set size(i.e., the intersection set of the individual coordinateindex sets is smaller for the rotated case since thelower (main) principal coordinate directions give asmaller candidate set).

In Fig. 13, we show the histogram of the number ofcodevectors searched by the projection method withand without the principal component rotation fordimensions K 5 8 and codebook size N 5 1024. It canbe noted that there is a significant reduction in therange of the histogram with a shift in the high

probability mode toward the lower range clearlyindicating the extent of reduction in both the aver-age and maximum complexity of the search.

The increase in the average index set size withrotation results in an increase in both storage of themapping table based intersection procedure andcomputational overheads of the intersection proce-dure. In Figs. 7 and 8, the increase in the computa-tional overhead of the intersection procedure dueto rotation over the unrotated case can be seen.However, the box-search overhead cost has reduceddue to rotation. This can also be noted in Figs. 9 and10, which show the box-test comparisons per code-vector.

In this context, it is possible to consider the effectof using only the main principal coordinate direc-tions which have more highly reduced index set sizesthan the corresponding unrotated case. By this, the

TABLE 1Comparison of the Intersection Count Procedure and Box-Search with Full-Search

Algorithm

Main complexity Overhead computation

NC NC Ave Max Total storage m a c

IC (NR) 21.1 46 399.3 436.0 1596447 21.1 39.6 401.8IC (R) 5.6 14 359.4 411.1 3120534 13.6 10.5 824.6BS (NR) 21.1 46 822.1 855.0 24576 21.1 39.6 360.1BS (R) 5.6 14 237.2 265.6 24576 13.6 10.5 237.9Full-search 1024 1024 — — 8192 1024 1920 127.9

Note. Dimension K 5 8. Codebook Size N 5 1024. Data: 50,000 vectors of speech waveform. NR, no rotation. R, rotation. Main complexity:[NC, NC], [average, maximum] number of codevectors searched (distance computations). Overhead computation: Integer increment andcomparisons for intersection count procedure (IC); Real comparisons for box-search (BS); [m, a, c], average number of [multiplications,additions, comparisons] per vector-component.

FIG. 12. Total storage required by mapping table (IM/FTand BM/FT) and box-search (BS) for dimension K 5 8 andcodebook sizes N 5 32, 64, 128, 256, 512, and 1024.

FIG. 13. Histogram of number of codevectors searched (NC)for dimension K 5 8 and codebook size N 5 1024. No Rotation:without principal component rotation. Rotation: with princi-pal component rotation.

274

mapping table incurs reduced storage and computa-tional overheads. However, since fewer index setsare used in finding the final candidate set, this willresult in an increase in the final candidate set sizeand the search complexity in comparison to the fullK-axes intersection in the rotated case. The sameapplies to the box-search also, where the use of lessthan K axes results in decreasing the box-test over-heads. However, the reduction in the box constraintsin using only some of the projections results in use ofboxes ‘‘unbounded’’ in the higher coordinate direc-tions, consequently increasing the final candidateset size and the final search complexity. The trade-offbetween the reduction in the overhead computationand the increase in the final search complexity inboth the mapping table-based intersection procedureand the box-search can be studied only empirically,since the number of principal component axes usedand their relative index set sizes with respect to theunrotated case depend on the correlation in the dataand the size and dimensionality of the problem.

An interesting effect of the rotation in reducing theextent of overlap between the Voronoi boxes andconsequently in the final candidate set size can beobserved from the average error incurred (or equiva-lently the signal-to-noise ratio (SNR)) when the testdata are quantized by selecting the final codevectorrandomly from the final candidate set. Figure 14compares the full-search SNR and the ‘‘randomselection’’ SNR (denoted by SNRr) for the rotated andunrotated case, respectively, for dimensions K 5 8and codebook sizes N 5 32, 64, 128, 256, 512, and1024. Rotation can be seen to improve the SNRr very

significantly closer to the full-search SNR for all thecodebook sizes.3

7.4. Overall Complexity of DifferentProjection-Based Search Algorithms

In Table 1, we give a comparison of the mappingtable-based IC procedure and the BS procedure(with and without the use of rotation) with thefull-search complexity for dimensions K 5 8 andcodebook size N 5 1024 in terms of computationalcomplexity, storage and overhead computational cost.Here, the following can be noted:

1. IC and BS procedures have the same complex-ity in terms of the average and maximum number ofcodevectors examined, NC and NC.

2. The mapping table-based intersection proce-dure has an exhorbitant storage complexity in com-parison to the box-search method.

3. The overhead costs of the IC procedure isroughly comparable to that of BS for the unrotatedcase, but, for the rotated case, while the overheadcost of IC procedure almost doubles, that of box-search shows a significant decrease. The storagerequirement of the IC procedure also nearly doubleswith rotation.

4. The overall computational complexity of thevarious algorithms, including the overhead costs,can also be compared in terms of the standard (macs)measure of average number of multiplications, addi-tions, and comparisons per vector component [10];the box-search with rotation can be seen to have thelowest complexity and offers excellent reduction overthe full-search complexity with only the cost of scalarcomparisons being double that of full-search.

3 We note here that [10] mentions such a random selection to bevery efficient with a difference of only 0.18 dB between thefull-search and random selection SNR in a simulation for dimen-sion K 5 8 and codebook size N 5 1024 using 14,720 vectors ofdata. However, from the simulation results shown in Fig. 14, weobserve that this difference can be much larger—as much as 9 dBfor codebook size N 5 32 and about 4 dB for codebook size N 51024. The small difference of 0.18 dB observed by Cheng et al. canpossibly be attributed to insufficiency of the amount of data used(about one-tenth that used here). This will result in underestima-tion of the Voronoi projections and will produce only a limitedextent of overlap. Consequently, the final candidate sets will besmall and will mainly consist of codevectors which are close toeach other to permit even their underestimated Voronoi boxes tooverlap and contain a test vector. As a result, the contention in arandom selection from the candidate set will result in only smallquantization errors accounting for the small reduction in theSNR. Due to the large SNR difference we have observed, contraryto [10], we believe that it may not be possible to make use ofrandom encoding in the Voronoi projection based search forpractical purposes.

FIG. 14. SNR for full-search and for encoding by randomselection (SNR) from final candidate set in projection methodfor dimension K 5 8 and codebook size N 5 1024. No Rotation:without principal component rotation. Rotation: with princi-pal component rotation.

275

8. CONCLUSION

In this paper, we have considered fast searchtechniques based on the implicit geometric interpre-tation of the nearest-neighbor search in terms of ageometric construct called the Voronoi diagram ofthe given set of points. This paper is mainly con-cerned with fast search techniques based on theprojections of the Voronoi regions; it highlights thebasic efficiency of using the Voronoi projections andshows that fast search based on the Voronoi projec-tion, using appropriate data structures, can result invery practical and highly efficient fast search algo-rithms.

In this respect, this paper provides a detailedstudy of two fast search techniques using the Voronoiprojections, namely, the box-search and the mappingtable-based search. We have investigated the box-search as an alternate and simpler interpretation ofthe projection information, with significantly lessstorage than the mapping table-based search (whichhas a very high storage complexity). In the mappingtable-based method, we have considered an intersec-tion count procedure for obtaining a small set ofcandidate codevectors to be searched using the Voro-noi projections. This procedure, while being simpleto implement, has been shown to have less complex-ity than the procedure employed by Cheng et al. [10]to use the Voronoi projections.

The main observations in this study can be summa-rized as follows:

1. The box-search requires a significantly lowstorage of 2KN words in comparison to the muchhigher storage of (3 1 a)K N of the mapping table.

2. The overhead complexity of box-search de-creases with increasing dimension. In contrast, theintersection complexity and storage increase withdimension in the mapping table-based search. Theaverage value of the overhead computational costfor box-search asymptotically saturates to a value of2N/K per vector component, which decreaseslinearly for increase in dimension. This is signifi-cantly less than the corresponding bounds of[log 2N 1 (K 1 N)/K, (log 2N 1 (K 1 1)N/K] of themapping table-based intersection procedure, whichhas an average complexity bound of O(N) whichincreases linearly with dimension.

3. Box-search derives benefit from principal com-ponent rotation in reducing both the search complex-ity and overhead computation. However, in the caseof the mapping table, principal component rotationreduces the main search complexity only at the

expense of significant increase in storage and over-head computations.

REFERENCES

1. Abut, H. Vector Quantization. IEEE, Piscataway, NJ, 1990.2. Avis, D., and Bhattacharya, B. K. Algorithms for computing

d-dimensional Voronoi diagrams and their duals. In Advancesin Computing Research, Vol. I, pp. 159–180. JAI Press,London, 1983.

3. Barlaud, M., et al. IEEE Trans. Image Process. 5(2) (1996).[Special issue on vector quantization]

4. Bei, C. D., and Gray, R. M. An improvement of the minimumdistortion encoding algorithm for vector quantization. IEEETrans. Commun. COM-33 (1985), 1132–1133.

5. Bowyer, A. Computing dirichlet tesselations. Comput. J. 24(2)(1981), 162–172.

6. Brown, K. Q. Geometric Transformations for Fast GeometricAlgorithms. Ph.D. thesis, Department of Computer Science,Carnegie-Mellon University, Pittsburgh, Dec. 1979.

7. Browstow, W., Dussault, J.-P., and Fox, B. L. Construction ofVoronoi polyhedra. J. Comput. Phys. 29 (1978), 81–92.

8. Burkhard, W. A., and Keller, R. M. Some approaches tobest-match file searching. Commun. ACM 16(4) (1973), 230–236.

9. Chen, S. H., and Pan, J. S. Fast search algorithm forVQ-based recognition of isolated words, IEE Proc. 136(6)(1989), 391–396.

10. Cheng, D. Y., Gersho, A., Ramamurthi, B., and Shoham, Y.Fast search algorithms for vector quantization and patternmatching. In Proceedings of IEEE International Conferenceon Acoustics, Speech, and Signal Processing, Mar. 1984, Vol.1, pp. 9.11.1–9.11.4.

11. Clarkson, K. L. A probabilistic algorithm for the post officeproblem. In Proc. 17th Ann. ACM Symp. Theory Comput.,1985, pp. 175–184.

12. Clarkson, K. L. A randomized algorithm for closest-pointqueries. SIAM J. Comput. 17(4) (1988), 830–847.

13. Dobkin, D. P., and Lipton, R. J. Multidimensional searchingproblems. SIAM J. Comput. 5(2) (1976), 181–186.

14. Duda, R. O., and Hart, P. E. Pattern Classification and SceneAnalysis. Wiley, New York, 1973.

15. Edelsbruner, H. Algorithms in Combinatorial Geometry.Springer-Verlag, Berlin/New York, 1987.

16. Fischer, F. P., and Patrick, E. A. A preprocessing algorithm fornearest neighbour decision rules. In Proc. Nat. ElectronicsConf., 1970, Vol. 26, pp. 481–485.

17. Fukanaga, K., and Narendra, P. M. A branch and boundalgorithm for computing k-nearest neighbours. IEEE Trans.Comput. 24 (1975), 750–753.

18. Gersho, A., and Cuperman, V. Vector quantization: A patternmatching technique for speech coding. IEEE Commun. Mag.21 (1983), 15–21.

19. Gersho, A., and Gray, R. M. Vector Quantization and SignalCompression. Kluwer, Boston, 1992.

20. Gray, R. M. Vector quantization. IEEE ASSP Mag. 1 (1984),4–29.

21. Huang, S. H., and Chen, S. H. Fast encoding algorithm forVQ-based image coding. Electron. Lett. 26(19) (1990), 1618–1619.

276

22. Kalantari, I., and McDonald, G. A data structure and analgorithm for the nearest point problem. IEEE Trans. Soft-ware Eng. SE-9(5) (1983), 631–634.

23. Kamgar-Parsi, B., and Kanal, L. N. An improved branch andbound algorithm for computing k-nearest neighbours. PatternRecognition Lett. 3 (1985), 7–12.

24. Klee, V. On the complexity of d-dimensional Voronoi dia-grams. Arch. Math. 34 (1980), 75–80.

25. Kleijn, W. B., and Paliwal, K. K. Speech Coding and Synthe-sis. Elsevier, Amsterdam, 1995.

26. Lee, D. T., and Preparata, F. P. Computational geometry—Asurvey. IEEE Trans. Comput. C-33(12) (1984), 1072–1101.

27. Linde, Y., Buzo, A., and Gray, R. M. An algorithm for vectorquantization design. IEEE Trans. Commun. COM-28 (1980),84–95.

28. Makhoul, J., Roucos, S., and Gish, H. Vector quantization inspeech coding. Proc. IEEE 73 (1985), 1555–1588.

29. Niemann, H., and Goppert, R. An efficient branch-and-boundnearest-neighbour classifier. Pattern Recognition Lett. 7 (1988),67–72.

30. Orchard, M. T. A fast nearest-neighbor search algorithm. InProceedings of IEEE International Conference on Acoustics,Speech, and Signal Processing, Toronto, Canada, May 1991,pp. 2297–2300.

31. Paliwal, K. K., and Ramasubramanian, V. Effect of orderingthe codebook on the efficiency of the partial distance searchalgorithm for vector quantization. IEEE Trans. Commun.COM-37 (1989), 538–540.

32. Preparata, F. P., and Shamos, M. I. Computational Geometry.Springer-Verlag, Berlin/New York, 1985.

33. Ramasubramanian, V., and Paliwal, K. K.An efficient approxi-mation-elimination algorithm for fast nearest-neighboursearch based on a spherical distance coordinate formulation.In Signal Processing V: Theories and Applications, Proc. of theEUSIPCO ’90, Barcelona, Spain (L. Torres-Urgell and M. A.Lagunas-Hernandez, Eds.), pp. 1323–1326, North-Holland,Amsterdam, 1990.

34. Ramasubramanian, V., and Paliwal, K. K.An efficient approxi-mation-elimination algorithm for fast nearest-neighboursearch based on a spherical distance coordinate formulation.Pattern Recognition Lett. 13(7) (1992), 471–480.

35. Ramasubramanian, V., and Paliwal, K. K.An efficient approxi-mation-elimination algorithm for fast nearest-neighbor search.In Proceedings of IEEE International Conference on Acous-tics, Speech, and Signal Processing, San Francisco, pp. I89–I92, Mar. 1992.

36. Van Rijsbergen, C. J. The best-match problem in documentretrieval. Commun. ACM 17(11) (1974), 648–649.

37. Seidel, R. A convex hull algorithm for points in even dimen-sions. Technical Report Comp. Sci. Tech. Rep. 81-14, Univer-sity of British Columbia, Vancouver, 1981.

38. Seidel, R. The complexity of Voronoi diagrams in higherdimensions. In Proc. 20th Ann. Allerton Conf. Commun.Control and Comput., pp. 94–95, 1982.

39. Shapiro, B. A. The choice of reference points in best-match filesearching. Commun. ACM 20(5) (1977), 339–343.

40. Shasha, D., and Wang, Tsong-Li. New techniques for best-match retrieval. ACM Trans. Inform. Syst. 8(2) (1990), 140–158.

41. Soleymani, M. R., and Morgera, S. D. An efficient nearest-neighbour search method. IEEE Trans. Comput. COM-35(6)(1987), 677–679.

42. Soleymani, M. R., and Morgera, S. D. A high speed searchalgorithm for vector quantization. In Proceedings of IEEEInternational Conference on Acoustics, Speech, and SignalProcessing, 1987, pp. 1946–1948.

43. Soleymani, M. R., and Morgera, S. D. A fast mmse encodingtechnique for vector quantization. IEEE Trans. Commun.COM-37(6) (1989), 656–659.

44. Vidal, E. An algorithm for finding nearest neighbours in(approximately) constant average time complexity. Inform.Process. Lett. 4(3) (1986), 145–157.

45. Vidal, E., Rulot, H. M., Casacuberta, F., and Benedi, J. M. Onthe use of a metric-space search algorithm (AESA) for fastdtw-based recognition of isolated words. IEEE Trans. Acoust.Speech Signal Process. 36(5) (1988), 651–660.

46. Watson, D. F. Computing the n-dimensional delaunay tessel-lation with applications to Voronoi polytopes. Comput. J.24(2) (1981), 167–172.

47. Yao, A. C. On constructing minimum spanning trees inK-dimensional spaces and related problems. SIAM J. Com-put., 11(4) (1982), 721–736.

48. Yao, F. F. Computational geometry. In Handbook of Theoreti-cal Computer Science (J. van Leeuwen, Ed.), pp. 344–389.Elsevier, Amsterdam, 1990.

49. Yunck, T. P. A technique to identify nearest-neighbours. IEEETrans. Syst. Man Cybern. SMC-6(10) (1976), 678–683.

50. Ramasubramanian, V., and Paliwal, K. K. Fast nearest-neighbor search based on Voronoi projections and its applica-tion to vector quantization encoding. IEEE Trans. Speech andAudio Processing, 1997 (under review).

277