Algorithms for Querying by Spatial Structure

Dimitris Papadias’, Nikos Mamoulis’ and Vasilis Delis2

‘Department of Computer Science Hong Kong University of Science and Technology

Clear Water Bay, Hong Kong { dimitris, mamoulis} @cs.ust.hk

Abstract: Structural queries constitute a special form of content-based retrieval where the user specifies a set of spatial constraints among query variables and asks for all configurations of actual objects that (totally or partially) match these constraints. Processing such queries can be thought of as a general form of spatial joins, i.e., instead of pairs, the result consists of n-tuples of objects, where n is the number of query variables. In this paper we describe a flexible framework which permits the representation of configurations in different resolution levels and supports the automatic derivation of similarity measures. We subsequently propose three algorithms for structural query processing which integrate constraint satisfaction with spatial indexing (R-trees). For each algorithm we apply several optimization techniques and experimentally evaluate performance using real data.

1. Introduction Several types of spatial queries have been the focus of active research in the database community: window queries [G84], nearest neighbors [RKV95], relation-based queries [PTSE95] etc. The above types retrieve all objects in the database that satisfy some spatial property with respect to a fixed reference object or window. Recently the focus has shifted towards spatial joins [R91] [G93] [BKS93], which involve the retrieval of pairs of objects that satisfy some spatial predicate (most often overlap).

This work examines an alternative form of spatial information processing, namely, queries involving the retrieval of n-tuples (n>2) of objects that satisfy some spatial structure. Structure is described as a set of spatial constraints between query variables which can be expressed either by a “verbal” (e.g., select X, Y, Z, from Roadmap, where overlaps(X,Y) and north(Y,Z)) or pictorial language (e.g., by drawing a prototype configuration on a sketch-board). This type of queries can be thought of as the generalization of spatial joins (if the relation between variables is overlap, it corresponds to common multi-way spatial join). From a different

Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the VLDB copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Very Large Data Base Endowment. To copy otherwise, or to republish, requires a fee an&or special permission from the Endowment. Proceedings of the 24th VLDB Conference, New York, USA, 1998

‘Computer Engineering and Informatics Department and Computer Technology Institute

University of Patras, Greece delis@cti.gr

perspective, structural queries constitute a special class of image similarity retrieval, where the query specifies an input configuration to be matched with stored images. Similarity is based on relative locations and not on visual characteristics (e.g., colour, shape).

Let n be the query size (number of variables) and N be the data size (number of image objects): in the worst case (exhaustive search), all n-permutations of N objects have to be searched in order to find solutions (i.e., N!/(N-n)!). In real DBMSs where N>>n, this number is O(N”), meaning that the retrieval of structural queries can be exponential to the query size. Query processing becomes more expensive if inexact matches are to be retrieved, a situation which arises very often in practical applications.

In order to avoid this problem, most related previous techniques (e.g., [GR95] [NNS96]) have focused on a specific instance where images consist of known (labelled) objects and queries express spatial constraints among a subset of these objects. [PF97] employ R-trees to solve structural queries for images that contain a constant number of labelled objects (e.g., lungs) and a small number of unlabelled ones (e.g., tumours). Although their method is efficient for domains involving numerous small images with few unlabelled objects (e.g., medical databases of X-rays) it is not applicable to large images of unlabelled objects.

In this paper we deal with the general problem where large images contain arbitrary numbers of unlabelled objects. In order to provide a general solution, we present a unified framework for structural similarity, which can represent various resolution levels and automates the derivation of similarity measures. We then propose algorithms that can solve the problem for considerable data and query sizes. These algorithms utilize ideas from related work in spatial databases (spatial join processing) and AI (constraint satisfaction algorithms).

Although the problem of querying by structure is not a new one (it has been around since the early stages of computer vision [BB84]), to the best of our knowledge this is the first approach to provide a solution which combines search algorithms with spatial indexing and can be applied for secondary memory retrieval. Our techniques have a wide range of potential applications in various areas (e.g., GIS, Multimedia Databases, VLSI).

546

The rest of the paper is organized as follows: Section 2 describes a binary string encoding for the representation of structure in multiple resolutions and dimensions. Sections 3 outlines the problem and provides examples of spatial queries and their processing. Sections 4, 5 and 6 describe three algorithms for structural query processing: the first one extends traditional spatial join methods for R- trees to multi-way (nested) joins. The second algorithm uses a search heuristic to prune the windows where query variables can be instantiated from, while the third one combines ideas from the first two algorithms. Section 7 compares the performance of the algorithms under several conditions. Finally, Section 8 concludes with future research directions.

2. A Framework for Structural Similarity We will initially confine our discussion to one dimension and address the most common types of relations proposed and mathematically defined so far in the spatial domain [PS94], namely topological (e.g., inside, overlaps), directional (e.g., north, northeast) and qualitative distance relations (e.g., near, far). Figure 1 illustrates the three types of ID relations assuming that the lower interval is the reference object and the upper interval is the primary one.

Our goal is to provide a unified and adjustable framework which permits the definition of any type of spatial relation and the automatic generation of similarities between them. Assume that in a particular application the only relations of interest are the ones in Figure 1, and we are given a (reference) interval [a,b]. Then we identify nine potential regions of interest: I.(-co,a-6) 2.[a-&a-6] 3.(a-b,a) 4.[a,a] 5.(a,b) 6.[b,b] 7.(b,

b+6) S.[b+d, b+6] 9.(b+d,+oo) For each of the above regions we associate a binary

variable, r, s, t, U, v, w, x, y, z, respectively (see Figure 2). Given a primary interval [cd], the value of every variable indicates the result of the intersection between [cd] and the variable’s associated region (“0” corresponds to an empty intersection while “1” corresponds to a non-empty one). Thus, we can define ID relations to be 9-tuples

topological relations meets overlaps covers equal

IIiI

directional relations

- left-near ‘s

Figure 1 Categorization of 1D relations

(Rrsruww : c s, t, u, v, w, x, y, z E {O,l 1). Such a consecutive partitioning of space constitutes a

resolution scheme. There are several possible schemes; the particular choice is affected by the users’ expectations or the application’s requirements, as every scheme can refine or generalize a particular relation class. For example, when distance relations are not needed we can apply a scheme with only five bits (two corresponding to the points of the reference interval) which defines the 13 relations between intervals proposed by Allen [A83].

The feasible relations at a particular resolution scheme are called primitive relations. In general, the less the binary variables, the coarser the resolution, and vice versa. If b is the number of bits used by the resolution scheme, the number of primitive relations in 1D is b(b+1)/2 - k, where k is the number of point variables, i.e. intervals of the form [a,a]. For b=9, k=4 we get 41 relations (see Figure 2), while for b=5, k=2, there exist 13 (Allen’s) relations.

Each spatial relation is mapped onto a set of primitive relations. For instance, lest can be mapped onto ~R~oooooooo, RII~OOO~O~, R,IIOOOOOO, ROIIOOOOOO, Roo~oooooo~ and near onto (ROO1OOOOOO, ROOOOOO1OO). Disjunction of spatial relations (e.g., left or near) are represented by the unions of the corresponding sets, and conjunctions by their intersection (e.g., {ROO1OOOOOO} corresponds to left-near).

The next step is to provide a mechanism for representing similarities among relations independently of the resolution scheme. [F92] defined the concept of conceptual neighborhood as a cognitively plausible way to measure similarity among Allen’s interval relations. A neighborhood is represented as a graph whose nodes denote relations that are linked through an edge, if they can be directly transformed to each other by continuous interval deformations. Depending on the allowed deformation (e.g., movement, enlargement), several graphs may be obtained.

Figure 2 represents the neighborhood graph for a distance-enhanced resolution scheme, assuming that a minimal deformation is a movement of a single interval endpoint. Starting from relation R,OOOOOOOO and extending

covered by contains contained by disjoint

I 1 I -I

distance relations far

547

bit: 0 I 2 3 4 5678 r stu Y wiry.2

T T n

the upper interval to the right, we derive relation R1lOOOOOOO. With a similar extension we can get the transition from RIIOO~~~OO to RII~~~~OOO and so on. RIOO~~OOOO and RIIIOO~OOO are called 1” degree neighbors of Rllooooooo. The distance d between two relations is equa1 to the length of the shortest path connecting them in the neighborhood graph.

The binary string representation enables automatic calculation of distances using the pseudo-code of Figure 3, which counts the minimal number of “0”s that have to be replaced with “1”s in order to make the two strings identical. For example d(Rooo,loooo, RO1OOOOOOO) = 5 and d(R~lloooo, Rllooooooo) = 6 (the underlined% are the ones counted during the calculation of distance). The distance between a relation R and a relation set {Rt,. . . ,Ri} equals the minimum distance between R and any of RI,. . .,Ri (e.g., 4&~~~0000, WOIWOOOO, RIMLOOOOO))= 5). INT distance(relation Rl, relation R2) R = RI OR R2; /*bitwise OR */ d= 0;

FOR i:= R.leftmost-I to Rxightmost-I DO IF Rl[i]=O THEN d++; IF R2[i]=O THEN d ++;

RETURN (~5); Figure 3 Distance calculation

The encoding and distance calculation can be extended accordingly to multi-dimensional spaces. A D- dimensional relation is defined as a D-tuple of 1D projections, e.g. RIIOOOOOOO-IHOOOOOO = RIOOOOOOO, RIIIOOOOOO).

In order to derive a neighboring relation we have to replace one of the constituent 1D projections with its

Figure 2 1D Conceptual neighborhood including distances neighbors. As a result, computing D-relation distances is reduced to the already solved problem of computing 1D distances. In this paper we calculate the distance between two multi-dimensional relations by summing up the distances in each dimension (other metrics, such as Euclidean [NNS96], can also be applied).

The advantages of the proposed framework are i) the expressiveness of the encoding in the sense that given a binary string, the corresponding spatial configuration can be easily inferred, and vice versa, ii) efficient automatic calculation of neighborhoods and relation distance, and iii) the uniform representation of all three types of relations (topological, directional, distance) in various resolution levels. For the sake of clarity, in the rest of the paper we use the distance enhanced resolution scheme of Figure 2. However for more realistic applications, sufficiently fine schemes (large encoding strings) can be used, while retaining the model’s properties. The algorithms of the following sections are independent of the resolution and can be applied with any set of spatio- temporal relations. For a number of alternative resolution schemes and a more detailed description of the framework see [DPM98].

3. Structural Queries The projection-based definitions of relations and similarity measures of Section 2 are particularly suitable for structural similarity retrieval, because spatial databases often utilize minimum bounding rectangles (i.e., projection-based approximations) as a fast jZter step to

548

(4 Query (b) Constraints Figure 4 Example query

_,

(c) Solution

exclude the objects that could not possibly satisfy a query [086]. Furthermore, structural queries do not always have exact matches and crisp results. Rather, the output should have an associated “score” to indicate its similarity to the query. By adoption, this score is inversely proportional to the degree of neighborhood.

A structural query can be formalised as a binary constraint satisfaction problem [N89] (CSP) which consists of: l A set of n variables, VO,V1,...,Vn-i that appear in the

wry. l For each variable Vi a finite domain Di ={uo,. . ., UN-l} of

N potential values which correspond to image objects. In this paper we assume that all domains are identical, i.e., each variable can be instantiated to any image object.

l For each pair of variables Vi,Vj a binary spatial constraint Cij which is a set of primitive relations.

Consider, for example, the query of Figure 4(a) which is a spatial arrangement of n=4 variables, expressed using a query-by-sketch language. Assuming the distance- enhanced resolution scheme of Figure 2, the set of query constraints between all variable pairs is illustrated in Figure 4(b). The domain of each variable is the set of objects in the image to be searched. Figure 4(c) illustrates a solution where variable V, is instantiated to object 143, V, to object 207 and so on. Although the particular language specifies relations between all pairs of variables, in some cases (e.g., verbal languages), queries may be incomplete (some C, may be left unspecified) or indefinite (C, may be disjunctions of relations).

A binary instantiation {Vituk, Vjtui} is consistent, if R(uk,ui) z Cu. For instance, the constraint between V0 and V3 is R1OOOOOOOO-I1lOOOOOO, which is also the relation between their corresponding instantiations (143,42) in Figure 4(c); therefore, {V&143, Vst42) is consistent. We define the binary degree of inconsistency z of {Vi+

uk, Vj tui] as the distance between Cij and R(UiJ.li). Although the constraint between V0 and V, is ROO1lllOOO- OolllllOO, the relation between objects 143 and 207 is Roollllooo-oollllQoo; hence, z =l for {Vo+143, Vl+2W. The degree of inconsistency T of a solution {V,t up, . . ., V,-, t u,} is the sum of all binary inconsistency degrees:

p xd(C,,R( uk, 2.0)) where {vi + Uk, Vj + UI>

VIJ, OSi,j<n

Degrees of inconsistency are used for the retrieval of configurations that match the input structure closely, but not perfectly. The maximum allowed T and z are submitted with a query in order to adjust the trade-off between the level of approximation and the cost of query processing. For instance, if T=6 and 2=2, only solutions that produce total relation difference I 6 and pair-wise difference I 2 will be retrieved. Obviously as T and z increase, so does the number of solutions, but also the cost of query processing.

3.1 Forward Checking with Dynamic Value Ordering

A number of algorithms have been proposed for solving CSPs [N89]. One of the most effective, is forward checking (FC) [HE801 [BG95] which has been shown to outperform the rest for a wide range of problems involving “crisp” constraints [BvR95]. FC must be modified for structural queries in order to handle soft constraint processing using T and z.

The adjusted version works as follows: when a variable Vi is assigned a value uk, the domain of each future @n-instantiated) variable Vj is pruned according to uk and the constraint Cij, for all j>i. That is, all values ui that produce a distance d(Cti,R(Uk,Ui))>% are removed from the domain of Vj. The same happens for values that produce global inconsistency degree > T, taking into account the constraints between Vj and all instantiated

549

variables’. Consequently, when we reach instantiation level i (variables up to Vi have been instantiated), the values of variables V,,,... ,Vi will constitute a partial solution, and the domains of Wure variables will contain only values that may lead to a (complete) solution given the instantiations so far.

The procedure of pruning the domains of the future variables is called check forward. If, after a check forward the whole domain of a future variable is eliminated, the algorithm un-assigns the current variable’s value, and restores the values of future variables, which were eliminated due to the current instantiation. When the domain of the current variable is exhausted the algorithm backtracks to the previous one and assigns a new value to it. FC ou$uts a solution whenever the last variable is given a value, and terminates when it backtracks from the first variable.

In order to keep track of the allowable values for each variable at every instantiation level, FC uses a nxnxN domain table. Each element of domain[i]bJ is an array of N values that Vj can take at different levels. Before FC starts, domain[O]fiJ is initialized to D for all variables. When V0 is assigned a value up, domain[l]b] is computed for each remaining Vj, by including only values UI E domain[O]fi] such that d(Coj,R(up,ul))lZ. In general if uk is the current value of Vi, domain[i+I],ljJ is the subset of domain[iJli/ which is valid w.r.t. Cq and uk. In this way, at each instantiation level the domain[iJfiJ of V; continuously shrinks; when we reach level j, Vj gets instantiated from domainfi]fi] which contains only values compatible with the instantiations of previous variables. If a value of Vi results in the domain of some Vj to become empty, a new value is chosen and domain[i+I]/j] is re- initialized to domain[i]b].

Dynamic Variable Ordering (DVO) [BvR95] is a technique employed by several CSP algorithms to improve efficiency. The key idea behind FC-DVO is to reorder the future variables according to their domain size after “checking forward” at the current instantiation level. The variable with the minimum domain size becomes the next variable to be tested. In this way the number of search paths is minimized, because the variable with the smallest domain is the most likely to be pruned out; the algorithm will backtrack faster in the case that there is no valid assignment after the current partial solution. DVO is responsible for changing the order of VI and V2 in Figure 4(c).

The pseudo-code of a non-recursive version of FC with DVO which can be applied for structural query processing is given in Figure 5. FC-DVO has two drawbacks for the current application. First it is inapplicable for large spatial databases, because the 3D

’ The inverse constraints Cji are also considered but, for the sake of simplicity, we omit these tests in the rest of the paper.

FC-DVO(Query q, int r, T) FOR j = 0 TO n-f DO domain[Oj] = D /*inikdize all domains to D */ i = 0; /* index to the current variable */ WHILE (TRUE) (

new-value := chooseNextValue(domain[rjl[ifi; IF new-value = NULL THEN /* end ofdomoin */

IF i=O THEN RETURN; ELSE i:=i-1; CONTINUE; /*Buckrruck*/

ELSE instantiations[i] := new-value; /*store instantiation*/ IF i = n-l THEN /*last variable instantiated*/

output-solution(instantiations); ELSE /* intermediate variable instantiated */

IF check-forward(i) THEN /* successfil instantiation*/ DVO(i+l,n-I); Pvur. with the smallesl domain as next*/ i := i+ I; /* successful insfantiation: go forward */

1

BOOLEAN check-forward(int i) FOR j = i+l TO n-l DO /*for all uninstanriated variables*/

domain[i+l]b]= domain[i]b]; FOR all values UI E domain[i+l],fJ

IF d(Cg,R(instantiations[i],ul)) > z OR Texceeded THEN domain[i+l],Q]= domain[i+l],Q]-{u,);

IF domain[i+ I]i/=0 THEN RETURN FALSE; RETURN TRUE; Figure 5 Soft forward checking with dynamic value ordering

domain table cannot fit in main memory. The second drawback is the fact that it does not utilize the existing spatial indices which may exist for spatial relations. The incorporation of R-trees [G84] and appropriate query processing techniques can solve both these problems.

3.2 Multi-Relation Spatial Join

Structural queries can be viewed as multi-way spatial self- joins, where structural constraints correspond to join predicates. For example, a pair-wise spatial join is equivalent to a structural query with two variables related by a spatial constraint. The most influential technique for efficiently computing pair-wise, intersection joins using R-trees is presented in [BKS93]. It is based on the enclosure property: if two intermediate R-tree nodes do not intersect, there can be no MBRs below them that intersect. The algorithm first joins the high level nodes and then follows the links in order to find qualifying pairs below them (Figure 6). SpatialJoin(Rtree-Node N[i], Nfi]) FOR all NI E Nu] DO FOR all Nk E N[i] with Nk n NI # 0 DO

IF N[i] is a leaf page THEN Output (Nk, W

ELSE ReadPage(Nk.ref); ReadPage(Nl.ref); SpatialJoin(N[k], N[I])

Figure 6 R-tree SpatialJoin

550

In the pseudo-code of Figure 6, as well as in the rest of the paper, we make the distinction between an R-tree node N[i] and its entries Nk, which correspond to MBRs included in N[i]. Nk.ref points to the corresponding node N[k] at the next (lower) level. Although SpatialJoin assumes that the nodes to be joined are of equal height, the extension to different heights is straightforward.

Two local optimization techniques are used to improve the CPU speed of the above algorithm. The first, search space restriction, reduces the quadratic number of pairs to be evaluated when two nodes N[i], Nlj] are joined. If an entry Nk E N[i] does not intersect the MBR of Nu] (that is the MBR of all entries contained in Nu]), then there can be no entry N, E Nu], such that Nk and Ni overlap. Using this observation, space restriction performs two linear scans in the entries of both nodes before starting the SpatialJoin procedure, and prunes out from each node the entries that do not intersect the MBR of the other node. The second technique, based on the plane sweep paradigm [PS88], applies sorting in one dimension in order to reduce the computation time of the overlapping pairs between the nodes to be joined.

In addition, [BKS93] apply a technique that uses pinning (or page fiing), a well known I/O buffer management method, to force page fetching according to the optimal order. In [HJR97], SpatialJoin was extended by introducing an on-the-fly indexing mechanism to optimize the execution order of matchings at intermediate levels. [BKSS94] study the multi-step processing of spatial joins using several approximations, while [BKS96] employ parallel execution.

In order to use an arbitrary relation as the join condition in SpatialJoin, we need a mapping from relations, to bounding conditions between intermediate node entries that should be recursively joined. Table 1 shows the bounding condition BCij for Ni given NJ. This condition is based solely on the positions of the leftmost and rightmost l’s in C+ In particular, the leftmost 1, determines the position of Ni.l with respect to Nj.U, while the rightmost 1 of Ni.u with respect to Nj./ (I and u represent the lower and upper node-MBR points respectively). Entries that do not satisfy these conditions can be excluded during search.

(a) leftmost bit (a) rightmost bit Table 1 Bounding condition BCij for Ni

Assume, for instance, the query “find all pairs (V2, V,) related by RooOOOOOO1-OO1lOOOOO”. An entry N2 is bounded with respect to N3 by the following conditions: (N2.u > N3.1 + 6 ) on the x dimension, and by (I$.1 < N3.u), (NZ.u 2 N3.1) on the y dimension. Figure 4.3 illustrates an example for axis x: if N3 is the intermediate node entry containing an object assigned to V3, then the upper point of candidate entries for N2 (Nz.u) should lie in the grey area. Entries, like N2, not satisfying this constraint, cannot contain consistent instantiations of VZ. For approximate retrieval, bounding conditions are adapted to include z.

N’,

I N, taunding mndium for F$ Y

Figure 7 Example of bounding condition for intermediate nodes Using the above transformation, SpatialJoin is

extended to handle multiple relations. Figure 8 illustrates the code for multi-relation spatial join (MJS). In this case, the desired relation Cij, as well as z, are passed as parameters. Each BCG is computed using CG, z and Table 1 (inverse conditions are also computed, but omitted for clarity). Leaf nodes constitute solutions, if they are related by a relation whose distance from Cti is I z. Intermediate nodes are recursively searched if they satisfy BCij.

MSJ(Rtree-Node N[i], Nu], RelationSet C,,, int z) BCg = computeNodeBC(C,j,r); FOR all NI E Nb] DO FOR all Nk E N[i] DO

IF N[i] is a leaf page THEN IF d(C,,R(Nk, NI)) 5 z THEN OUtpUt (Nk, Nk, d)

ELSE IF BC&, NI) THEN

ReadPage(Nk.ref); ReadPage(Nl.ref);

MSJ(NIkl, WI, C+ 2) Figure 8 Multi-relation spatial join

Structural queries could be processed by executing MSJ for all pairs of variables and combining the binary solutions. The main problem with this approach is the large number of pair-wise joins (six for the query of Figure 4) and the complexity of combining their results (which may be too large to fit in main memory). In the rest of the paper we propose three algorithms that avoid calculating intermediate results by incorporating ideas from forward checking and traditional spatial join processing.

4. A Multilevel Forward Checking Algorithm The first algorithm, multilevel forward checking (MFC), extends MSJ to deal with n-tuples instead of pairs. MFC finds all n-combinations of intermediate nodes (at each level of the R-tree) that may contain some solution objects

551

and follows the references to the next level, until it reaches the leaves, where it outputs solutions. As an example consider the rectangles of Figure 9(a) which are organized in the R-tree of Figure 9(b) assuming a bucket size equal to three. The path to solution (de, 4) of the example query is: (1 ,l, 1,2) at the top, (B,B,A,D) at level 1 and (d,e,a,k) at level 0.

(a) Image &

Figure 9 Image and corresponding R-tree The calculation of combinations of the qualifying

nodes at each level (e.g., (l,l,l,l), (1,1,1,2), . . . . . (2,2,2,2) for the top) is expensive, as their number can be as high as C?, where C is the capacity of an R-tree bucket. Although the search space is not prohibitively large (usually nH0 and &200), the computational burden is due to numerous appearances of the problem during query processing. Finding the subset of node combinations which is consistent with the input query can be treated as a local CSP at each level. In particular the problem consists of l A set of n variables, V,,,Vi,. . .,V,,-i. l For each variable Vi a domain Di={No,. . .,Ni.i} of I (15

C) potential values which correspond to entries in R-tree node N[i].

l For each pair of variables V,Vj a binary constraint which: i] for intermediate nodes is a bounding condition BCG derived from Table 1 using the corresponding Cij and z, ii] for leaf nodes is a constraint CQ (disjunction of primitive relations).

The CSP ti the case of the top level of the tree in Figure 9 has four variables V0,V1,V2,V3, which can be instantiated to entries 1 or 2 of the root. As we saw in the example of Figure 7, BCz3 is: @&,u > Ns.1 + 6) on the x dimension, and (I$.1 < N3. u ) A (N2. u 2 Ns.1) on the y dimension. The binary instantiation {V,t2, V3tl} cannot lead to a solution at the lower levels because (1.~ < 2.1 + 6). Therefore, all combinations (x,x,2,1) can be pruned out during search.

MFC (Figure 10) applies forward checking to solve the CSP at each R-tree level: every time a variable Vi is instantiated to an entry Nk, the algorithm eliminates all Ni that do not satisfy BC@k,Ni) from the domains of each un-instantiated variable Vj. Initially N[j is set to an n- tuple that points to the tree root for all variables, i.e. N[i]=root, for i=O.. .n-1. A solution for the current tree level is found when the last variable is instantiated. The algorithm is then recursively invoked for the lower level, taking as parameter the n-tuple of the solution’s references. Solutions are output if they refer to actual objects. MFC returns to the previous tree level when it backtracks from the first variable at the current level.

In the example of Figure 9, when the first valid combination (l,l,l,l) is found at the top, MFC will be called for the next level, trying to find a combination of entries inside node 1 that satisfy all BCti (the domain of all variables is now D={A,B}). If such a combination does not exist, as is the case here, it will backtrack to the top level and attempt to find another solution - assume

MFC(Query q, Rtree-Nodes N[], int r, T) FORj=OTOn-ID0

domain[O]u] = {NI( NI E Nu]) /*NI is an entry ofNj*/ i = 0; /* index to the current variable */ WHILE (TRUE) (

new-value := chooseNextValue(domain[i][iJ); IF new-value = NULL THEN /* end of domain */

IF i=O THEN RETURN; ELSE i:=i-I; CONTINUE; /*Backtrack*/

ELSE instantiations[i] := new-value; /*store instantiation*/ IF i = n-1 THEN /Vast variable instantiated*/

IF (N[i] is a leaf page) THEN output-solution(instantiations);

ELSE MFC(q, instantiations.ref, 1, T) /*go to lower tree level */

ELSE /* intermediate variable instantiated */ IF check-forward(N[i].level,i) THEN /*valid instantiation*/

DVO(i+l,n-I); /*var. with the smallest domain as next*/ i := i+l; /*go to the next variable */

1

BOOLEAN check-forward(int level, int i) FOR j = i+l TO n-l DO /*for all uninstantiated variables*/

domain[ii+l],$J= domain[i]l]; FOR all values III E domain[i+l]/J]

IF (level = 0) /*leafnodes*/ IF d(Cij,R(instantiations[i],ui)) > I OR Texceeded

THEN domain[i+l]b]= domain[i+ I]fi]-{III}; ELSE /*intermediate nodes*/

IF NOT (BCij(instantiations[i],ui)) THEN domain[ii+l]/j]= domain[i+l]/j]-{uI};

IF domain[i+l]~]=@ THEN RETURN FALSE; RETURN TRUE;

Figure 10 Multilevel FC

552

(l,l, 1,2). The new domains for the next call of MFC become: D0=Di=D2={A,B} and D,={C,D}. A solution at this level is {V&B, VicB, V+A, VitC}. At the next call of MFC for level 0, the domains become D,,=D,={c,d,e}, Dz={a,b,f), D,={l,kj} and the solution (de, a, k) is found.

In order to enhance the performance of MFC we have implemented a variation of the space restriction heuristic. Assume the qualifying 4-tuple (1,2,2,2) for the top level of the tree. Although, candidate values for V0 are {A,B}, due to the relative positions of B and intermediate node 2 (disjoint), there can’t be any instantiations of Vi below node 2 that lead to solutions for {V&B} (valid instantiations for V0 and Vi should be inside intersecting nodes). Therefore, we can safely prune value B from VO’s domain and avoid useless instantiations. The following Space-Restriction routine takes the entries (e.g., A, B) of a node (e.g., 1) one by one and tests them against the rest of the nodes (e.g., 2) eliminating the ones that do not satisfy the corresponding bounding conditions.

Space-Restriction(Query q. Rtree-Nodes N[])( FOR i=O TO n-l DO

FOR ail Nk E N[i] DO FOR j=O TO n-l, i#j DO

IF N[i] is a leaf page THEN IF NOT (LBCzjN, Nljl))

THEN domain[O][i]= domain[O][i]-(Nk}; ELSE /* N[i] is a intermediate node */

IF NOT (BCt@k, Nbl)) THEN domain[O][i]= domain[O][i]-(Nk};

Figure 11 Multi-relation space restriction

The bounding conditions of Table 1 are used when N[i] is at an intermediate level. On the other hand, when N[i] is a leaf node (its entries are object MBRs) a more restrictive bounding condition can be applied. Consider that in Figure 12, we want to join objects in N[2] with all objects in N[3] w.r.t. R,,OOOooool (in Figure 7 we showed that N[2] satisfies the corresponding BC). Once we know the locations of each MBR in N[2] we can determine that some objects, such as N12, can be excluded. N’* cannot be related by ~OOoooOl with any MBR in N[3] because N12.1 < N[3].1+6. If only the bounding conditions of Table 1 were used, N’2 would pass the space restriction test.

NPI

N31 baading condhon for N2 I

Figure 12 Example of leaf bounding conditions Table 2 illustrates the complete set of leaf bounding

conditions LBCij between object MBRs and intermediate nodes. The bounding condition for the previous example

is at the bottom row of the first table (the corresponding condition was unlimited in Table 1).

(a) leftmost bit (a) rightmost bit Table 2 LBC that Nk must satisfy to pass space restriction

5. A Window-Reduction Algorithm Usually, constraints between intermediate nodes are in general too loose even for tight queries. As a result, a large number of qualifying intermediate nodes are visited by MFC, and this does not pay off in most cases, where the number of solutions is small and a large percentage of qualifying intermediate nodes are false hits. An alternative approach that overcomes this problem is to use the data MBRs for the instantiation of query variables and employ forward checking with R-trees to efficiently prune the domains. As mentioned in Section 3.1, the problem with this method is that it cannot be applied for large images because of the domain table size (O(n2N)). In this section we propose another FC-based algorithm, window reduction (WR), which avoids this problem.

WR maintains a 2D domain window (instead of the 3D domain set used by FC) that encloses all potential values for each variable (and possibly some false hits). When Vi takes a new value Nk, a new window Wj is computed for every un-istantiated variable Vj taking into account Nk and Cji. The intersection of Wj with (existing) domain Window [i],ijJ is stored at domainwindow [i+l]b]. Figure 13(a) illustrates the domain windows for V2 and V3, assuming that the first two variables of the example query have been

(a) {Votd, VlteI (a) {Votd, VICe, V2Ca) Figure 13 Example of WR

When VZ is instantiated to a (Figure 13(b)), the constraint C32 specifies that valid instantiations for V3 should lie in W3. The new domain Window[3][3] for V3 is the intersection of domainWindow[2][3] and W3, i.e., it corresponds to the only area that may contain values consistent with both {V&d, Vice} and V+a. Table 3 illustrates bounding windows used for the computation of Wj, given Cji and {Vi+ Nk}

553

(a) leftmost bit

ml (a) rightmost bit

Table 3 Domain window bounds

If some domain window becomes null (empty intersection), the current instantiation is invalid and the algorithm proceeds to the next value for Vi. WR can be thought of as a “lazy” version of forward checking because the domain windows are calculated but no values are retrieved until the variable gets instantiated. A drawback of this method is the fact that a possibly empty domain of Vj cannot be detected until WR reaches instantiation level j and performs the window search. However, this disadvantage is counterbalanced by the smaller number of R-tree searches. WR is illustrated in Figure 14.

WR(Quety q, int r, T) FOR j=O TO n-l DO domain Window[O]fi] = U, /*Universal Space*/ i=O; /* index to the current variable */ WHILE (TRUE) {

new-value := getNextValue(domain Window[i][ifi; IF new-value = NULL THEN /* end ofdomain */

IF i=O THEN RETURN; ELSE i:=i-1 ; CONTINUE; /*Backtrack*/

IF d(C,,,R(instantiations[i],ui)) > z OR Texceeded THEN CONTINUE /* invalid value inside domain window */ ELSE instantiations[i] := new-value; /*store instantiation*/

IF i = n-1 THEN output-solution(instantiations); ELSE /+ intermediate variable instantiated */

IF window-reduction(i) THEN /* success-l instantiation*/ Window-DVO(i+l,n-I); /*var. with smallest window next*/ i := i+ 1; /* successful instantiation: go forward */

1

BOOLEAN window-reduction(int i) FOR j = i+l TO n-1 DO /*for all uninstantiated variables*/

W, = computeWindow(instantiations[i],Cj,,r); domain Window[i+ I]li/= domain Window[i/llinW,; IF domain[i+ I//j]=0 THEN RETURN FALSE;

RETURN TRUE; Figure 14 Window-reduction algorithm

The next value for a variable Vi is retrieved via getNext ValueO, which uses domain Window[il[il as the query window for Vi. GetNextValueO does not perform a window query every time it is invoked, but the whole search path for each variable is maintained in memory. The overhead for this path-holding technique is pinning na h pages - a small number for most applications. After a value is retrieved for Vi, the algorithm checks whether it is consistent with the previous instantiations since not all

values that fall inside the domain window of Vi are necessarily legal.

In addition to domain windows and path maintenance techniques, WR uses DVO: when the domain windows of the future variables are calculated after an instantiation, the variable with the smallest domain window becomes the next to be examined. This is lead by the intuition that a small window is more likely to contain the least number of instantiations and minimize redundant consistency checks.

WR can be seen as a special form of indexed nested loop join. All blocks of first variable are scanned and directed index search finds the qualifying instantiations of the rest of the variables. The difference of this approach, is that its input is a graph of relations instead of a chain, and that it applies FC to take advantage of all constraints.

6. A Join Window-Reduction Algorithm WR essentially searches the whole space in order to instantiate the first variable, but after doing so it performs only window queries which are cheap operations in R- trees. The disadvantage of blindly instantiating the first variable in the whole universe could be avoided by an algorithm that combines properties of multi-relation spatial join and window reduction. The third algorithm (JWR) first applies a pairwise spatial join to retrieve instantiations for the first pair of variables and then uses window reduction to instantiate the rest of the variables. The subsequent variables are instantiated in the same way as WR:

JWR(Quety q, int 2, T ) FOR j=O TO n-l DO domainWindow[l]lj] = U; /*Universal Space*/ i=l; /* index to the current variable. Initially 1 (means both 0 and 1) */ WHILE (TRUE) (

IF i=l THEN /* valuesforfirst pair of variables (0, I) V IF getNextPair(instantiations,q)=NULL THEN RETURN;

ELSE /* values ofsubsequent variables */ new-value := getNextValue(domain Window[i][iJ); IF new-value = NULL THEN /* end of domain */

i:=i-1 ; CONTINUE; /*Backtrack*/ IF d(C,j,R(instantiations[i],ui)) > r OR Texceeded

THEN CONTINUE /* invalid value inside domain window */ ELSE instantiations[i] := new-value; /*store instantiation*/

IF i = n-l THEN output-solution(instantiations); ELSE /+ intermediate variable instantiated */

IF window-reduction(i) THEN /* successfil instantiation*/ Window-DVO(i+l,n-I); /*var. with smallest window next*/

i := i+l; /* successful instantiafion: goforward */

1 Figure 15 Join window-reduction algorithm

Function getNextPair(l assigns the next pair that satisfies the relations between the first two variables using MU, search space restriction (like MFC) and plane sweep. We apply a multi-relation plane sweep (MPS), which can deal with the whole set of relations of the

554

current resolution scheme. MPS finds intersections of rectangles belonging to nodes N[i], Nb] in two steps: a] first transforms the x-projection of each rectangle N,

ENS] to a new one N’r, according to Cij. This transformation is done so that: if N’i does not intersect on the x-axis with some entry Nk E N[i], then the original rectangle Ni will not be consistent w.r.t. Nk and Cti.

b] then it applies spatial sorting and plane-sweep to find all pairs (NL.x, N’r) that intersect. For each such pair it checks whether the corresponding pair (Nk,Ni) is consistent according to Cij and Cji.

In order to perform the transformation, MPS chooses a bit whose value is 1 on the x projection of CQ. Bits that refer to points <i.e. odd bits), rather than intervals, are preferred, because they restrict the resulting intervals N’i into single points. For instance, consider that Cij= ~OOOoOO1l. We transform the reference interval Nr~Nlj] to N’i as shown in Figure 16. If N’i, (which is a single point) does not intersect some Nk then the original intervals cannot satisfy ~OODOOOl i.

R 00000001 I Nk -

Figure 16 An example transformation We call guide bit, the bit according to which the above

transformation is performed. For our resolution scheme, the preference order for guide bits is {3,5,1,7,4,2,6,0,8}. The transformation is then performed for intermediate and leaf-level entries as illustrated in Table 4, where the first column illustrates the guide bit. The transformation to leaf node entries corresponds to the binary variables presented in Figure 2.

N',.I=N,.I, N',u=N,.u

(a) Intermediate nodes (a) leaf nodes Table 4 Transformation of x-axis projections

For calculating the first pair of variables to be joined we use statistical information about the number of occurrences of each relation in the data files. Relations that occur rarely prune search space more effectively than frequent ones. For instance, the constraint Rr,ollllooO- oollllloO between VO and V, is more restrictive than the other relations, because only a few pairs of objects satisfy it in normal data distributions.

7. Experiments In order to compare the performance of the three algorithms presented above, we implemented and tested them under several conditions. For our experiments we used LB data-file [T94] which contains 53,145 rectangles representing road segments of Long Beach county. The maximum distance of the rectangles in each axis is 10000, and the data density 0.25. From the above tile we built several R*-trees [BKSS90] of different block sizes, i.e. 512 bytes, lK, 2K, and 4K. The LRU buffer size of the R*-trees during the experiments was set to 128. We constructed 5 artificial sets of 30 queries: the number of variables in the queries of each set was fixed to 3,4, . , . , 7. In order to avoid trivial queries, each variable was set to intersect with some other variable on at least one axis. The distance between two variables on each axis did not exceed 6, which was set to 100. The number of solutions ranged from 0 to 6,366. The implementation language was C++, and all experiments were run on a SUN UltraSparc2 (200MHz) workstation with 256 MB of RAM.

Figure 17(a) shows the mean CPU-time and 17(b) the I/O page accesses averaged over all query-sets on the R*- tree with 1KB block size. WR and JWR clearly outperform MFC by orders of magnitude in terms of CPU- time. The performance gap widens with the query size because the domain windows in WR and JWR are continuously decreasing as new variables are instantiated. Moreover, empty window domains of the latter variables are detected early using the window reduction policy. On the other hand, the relaxed constraints between intermediate nodes do not permit MFC to prune the search space at the higher levels of the tree; thus, MFC cannot avoid the combinatorial explosion of possible instantiations as the number of variables increases. It is interesting to notice that MFC is better than WR in terms of page faults and this is due to the fact that WR instantiates the first variable in the whole space.

Another important observation from our experiments (not obvious in these diagrams) was the expected behaviour of MFC for almost all queries; the CPU-time was at the same levels depending only on the query size. On the other hand, the performance of WR and JWR was unpredictable: for instance the CPU time of WR may differ an order of magnitude for two different queries of the same size. This unstable behaviour is due to the fact that the resolution scheme may facilitate large reduction of the domain windows for some queries (e.g. inside), and not for others (e.g. disjoint).

Although MFC is not an appropriate algorithm for the current resolution scheme, it is still useful in other applications; we found that it outperforms the other algorithms in some cases of multiway intersection joins involving high density data. Improving the CPU-time of MFC is an issue for future work, as now the algorithm

555

3om 400 EIMFC n WR q JWR

350. 2500 ,/- 300 ,A

,/' ,m.. . /'

zew 250

200 - 24W

~ ___._e ."

150 -

(a) CPU-time as a fonction of n (block=lK) (b) Page accesses as a function of n (block=lK) (c) CPU-time for WR - JWR as a function of n

512

(d) CPU-timt as a diction of ikck size (%I) (e) Accesses as a function of block size (n=4) Figure 17 Experimental evaluation

(0 Overall cost of I& and J6 (n=4)

applies plain FC at a specific level, without taking 7. Conclusion advantage of the spatial locations of the objects.

Figure 17(c) illustrates the relative CPU-time performance of WR and JWR (also for block size of 1K). JWR maintains a significant performance gain over WR. The performance gap is not affected by query size, because the only difference of the algorithms is the instantiation method for the first pair of variables.

In order to evaluate the algorithms for various block sizes we executed the 4-variable query set using R*-trees of 5 12, lK, 2K, and 4K bucket sizes. CPU-time and page accesses are shown in Figure 17(d) and (e), respectively. Figure 17(f) shows the overall cost for WR and JWR, which was estimated by charging 1Oms for each page access (a typical value [HJR97]). The algorithms perform better for page size of 2K, while for larger sizes (4K) the degeneration of the tree affects the speed of the search.

Finally, we tested the performance of JWR over queries with non-zero degrees of inconsistency. In all experiments the T was set to 10. Figure 18 illustrates the overall cost of JWR for the 2K page size R*-tree. Each line corresponds to a different value of local tolerance 7. Because approximate retrieval is equivalent to exact retrieval using a larger window, the domain windows of JWR get larger as z increases. Larger windows imply more potential legal values and more consistency checks.

OJ

3 4 5

Figure 18 Overall cost of JWR for partial retrieval

There has been significant progress recently on image and video content retrieval [M98]; research focused mainly on visual content, i.e. properties like colour, shape, texture, etc. Here, we shift our interest on a rather neglected type of content retrieval, namely structural retrieval. This paper addresses the issue of spatial structural queries, i.e., queries that ask for all n-tuples of objects that satisfy some spatial constraints.

We first described a framework for encoding 1D relations in a way that allows efficient generation of similarity measures. We subsequently extended the model in a uniform way to arbitrary dimensions and multiple resolution levels. Then we presented three algorithms for structural query processing: - MFC which applies hierarchical constraint

satisfaction to eliminate tuples of intermediate nodes that cannot lead to solutions.

- WR which gradually reduces the domain windows of uninstantiated variables based on the values of instantiated ones.

- JWR which performs a pairwise join to instantiate the first pair of variables and then applies the same window reduction technique as WR.

Finally we experimentally evaluated their performance and found that JWR clearly outperforms the rest for the current application. All algorithms are independent of the resolution scheme so they can be used to process any type of spatial predicates.

Currently we are working on optimizing the I/O time of JWR and MFC, by using several page fetching policies. Adapting and testing the algorithms for several query plans is also an interesting topic. Research can also be carried out on the integration of other search algorithms with spatial indexes.

556

Acknowledgements We would like to thank Dimitris Meretakis and Eleanna Kafeza for their insightful comments. This work was by DAG96197.EG36 from Hong Kong Research Grant Council.

References

L4831 Allen, J., “Maintaining Knowledge About Temporal Intervals”, CACM, 26( 1 l), 1983.

[BG95] Bacchus, F., Grove, A. “On the Forward Checking Algorithm”, International Conference on Principles and Practice of Constraint Programming, 1995.

[BB84] Ballard, D., Brown, C. “Computer Vision”. Prentice Hall, 1984.

[BKSS90]Beckmann, N., Kriegel, H.P., Schneider, R., Seeger, B. “The R*-tree: an Efficient and Robust Access Method for Points and Rectangles”. ACM SIGMOD, 1990.

[BKS93] Brinkhoff, T., Kriegel, H.-P., Seeger, B., “Efficient processing of spatial joins using R- trees”. ACM SIGMOD, 1993.

[BKSS94]Brinkhoff, T., H.-P. Kriegel, R. Schneider, and B. Seeger “Multi-step Processing of Spatial Joins”. ACM SIGMOD, 1994.

[BKS96] Brinkhoff, T., Kriegel, H.-P., Seeger B. “Parallel Processing of Spatial Joins Using R- trees”. Proceedings of 12th International Conference on Data Engineering, 1996.

[BvR95] Bacchus, F., van Run, P. “Dynamic Variable Ordering in CSPs”, International Conference on Principles and Practice of Constraint Programming, 1995.

[DPM98]Delis, V, Papadias, D., Mamoulis, N. “Assessing Multimedia Similarity: A Framework for Structure and Motion”, ACM SIGMM, 1998.

[~921

[GR95]

EG931

[G841

Freksa, C., “Temporal Reasoning based on Semi Intervals”, Artificial Intelligence, Vol 54, pp. 199-227, 1992.

Gudivada, V., Raghavan, V. “Design and evaluation of algorithms for image retrieval by spatial similarity”. ACM Transactions on Information Systems, 13(1):115-144, 1995. Guenther, 0. “Efficient computation of spatial joins”. IEEE International Conference on Data Engineering, 1993. Guttman, A. “R-trees: A Dynamic Index Structure for Spatial Searching”. ACM SIGMOD, 1984.

[HE801 Haralick, R.M., Elliott, G.L., “Increasing tree search efficiency for constraint satisfaction problems”. Artificial Intelligence Vol 14, pp 263-313, 1980.

[HJR97] Huang, Y-W, Jing, N, Rundensteiner, E. “Spatial Joins using R-trees: Breadth First Travesral with Global Optimizations”. VLDB, 1997.

[M98] Maybury M. (ed.), Intelligent Multimedia Information Retrieval, AAAI Press/MIT Press, 1998.

[NNS96] Nabil, M., Ngu, A., Shepherd, J., “Picture Similarity Retrieval using 2d Projection Interval Representation”, IEEE TKDE, 8(4), 1996.

w91 Nadel, B. “Constraint Satisfaction Algorithms”. Computational Intelligence, 5, pp. 188-224, 1989.

[086] Orenstein, J. A. “Spatial Query Processing in an Object-Oriented Database System. ACM SIGMOD, 1986.

[PS94] Papadias, D., Sellis, T., “Qualitative Representation of Spatial Knowledge in Two- Dimensional Space”, VLDB Journal, Vol. 3(4), pp. 479-516, 1994.

[PTSE95]Papadias, D., Theodoridis, Y., Sellis, T., Egenhofer, M. “Topological Relations in the World of Minimum Bounding Rectangles: A study with R-trees”. ACM SIGMOD, 1995.

[PF97] Pet&is, E., Faloutsos, C. “Similarity Searching in Medical Image Databases”. IEEE TKDE, 9 (3)435-447, 1997.

[PS88] Preparata F, Shamos, M. “Computational Geometry”. Springer, 1988.

P911 Rotem, D. “Spatial Join Indices”. IEEE International Conference on Data Engineering, 1991.

[RKV95] Roussopoulos, N., Kelley, F., Vincent, F., “Nearest Neighbor Queries”, ACM SIGMOD, 1995.

D941 TIGER/Line Files, 1994 Technical Documentation I prepared by the Bureau of the Census, Washington, DC, 1994.