An Ontological Framework for Large-Scale Grid Resource Discovery

Juan Li, Son VuongComputer Science Department, University ofBritish Columbia

tjuanli, vuong}@cs. ubc. ca

Abstract

This paper presents GOIDS (Grid OntologicalIntegration and Discovery System), a comprehensivearchitecture for resource sharing and discovery inlarge-scale grids, where nodes integrate localontologies to expand semantic knowledge of sharedgrid resources. A peer-to-peer based ontologicaldirectory service is used to facilitate the formation ofthe semantic small-world - ontological communities.Ontological communities help prune the searchingspace and reduce the cost of searching. Insidecommunities, resource knowledge is integratedbetween nodes, and a DHT overlay is used forknowledge dissemination and discovery. Thisframework architecture improves interoperabilityamong grid participants and aids efficient resourcediscovery through an expressive query language.

1. Introduction

A large-scale grid enables the sharing of a widevariety of resources, including hardware components,software packages, knowledge information, devices,and other grid services. However, the resourcediscovery problem in large-scale grids is challengingbecause of the abundance of resources and thedynamic, heterogeneous, and distributed nature ofresources. Equally challenging is the task of integratinginformation about different properties and usagepolicies of these resources.

In this paper, we describe appropriate enhancementsfor grid resource discovery. We focus on the design ofGOIDS, an effective resource discovery and integrationframework which reduces the complexity ofinformation sharing and discovery on Internet-scalegrids. The system relies on ontologies to describe thestructure and semantics of resource properties toincrease the system's expressiveness andinteroperability.

To improve the scalability of this approach, wereconfigure the network topology according to nodes'ontological interests, so that the network exhibits the"small-world" properties [1]. Specifically, we use anontology directory overlay to help nodes form virtualcommunities. The community discovery, construction,and maintenance are manipulated in a decentralizedand automatic manner. Communities helpdiscriminatively distribute queries to ontologicallyrelated nodes, thus reducing the search space andimproving the scalability.

Inside each community, nodes share similarinterests, but they may use different ontologies. Anontology integration strategy is presented. It permitsdifferent community members to establish mappingsamong the various ontologies. Resource knowledgewithin a community is indexed in a DHT overlay and isaccessible through an expressive ontology-based querylanguage. We proposed two different coarse-grainedindexing schemes; an application can choose onescheme according to its specific need.

2. Related work

In medium-sized grids, resource discovery is usuallyguided by centralized servers [2]. To discoverresources in more dynamic, large-scale, and distributedgrid environments, P2P techniques have been used(e.g., [14, 15]). Flooding is the predominant searchmethod in unstructured P2P networks. This method,though simple, does not scale well in terms of messageoverhead. An efficient approach is the use ofdistributed hash tables (DHTs) [3-6], which has beenshown to be scalable and efficient. However, a missingfeature is the inherent support for expressive queries.

Research on the Semantic Web project [7] hasrecently gained much attention for its knowledgeintegration vision. Its focus is to exploit the power ofsemantic technologies to aid in informationrepresentation, retrieval and aggregation over the web.Most of the Semantic Web projects use the standard

1-4244-1521-7/07/$25.00 ©2007 IEEE757

RDF language [12] to describe data. Ontologylanguages such as DAML+OIL [17] and OWL [18]build on top of RDF allow a user to describe relationsbetween resources, defining a more abstract andexpressive resource sharing environment.

In the last few years, several projects aimed tocombine the strengths of grid and semantic webtechnologies, particularly for the use of resourcesharing. For example, the UK myGrid [8] project usesontologies to describe and select web-based servicesused in the Life Sciences, while OntoGrid [9] mixes intechniques from agent computing and P2P fordistributed discovery of semantic knowledge.

The integration of Semantic Web and P2Ptechnologies can also serve to benefit each other. Forexample, in the InfoQuilt [10] system, nodes use theDAML+OIL ontological language to describe theirinformation. Ontology knowledge is registered to acentral server for efficient indexing, while queries areforwarded between nodes in a P2P manner. Helios [ 1]is another system that uses the P2P model for semanticknowledge sharing. Unlike InfoQuilt, both the ontologyknowledge and data instance discovery in Helios arebased on an unstructured P2P network.

Like the mentioned systems, our system takesadvantages of the semantic web and the P2Ptechnologies to enhance resource sharing andsearching. We focus on improving the interoperabilityand scalability of an Internet-scale grid.

3. System overview

3.1. Two-tier structure

To make the searching more focused, we clusternodes sharing similar ontological interests together; asa result, the queries can be forwarded to nodes that arelikely to contain relevant resources. However, in anopen global-scale grid, it is difficult to discover nodeswith similar interest. We solve this problem by using anontological directory to organize dispersed ontologies,and to allow nodes find others sharing similar interest.The directory works as a "rendezvous" for nodesmeeting others with similar ontological properties, thusfacilitating the construction of semantic clusters. Thedirectory does not need to be predefined; itspontaneously grows as the network ontology evolves.

The system is organized by employing two layers -the directory overlay and the community overlay; andthe former facilitates the formation of the latter. Bothoverlays adopt the P2P-based DHT structure. When anode joins the system, it first locates the interestedontological community by looking up the directory

overlay, from where it gets one or more contacts of theinterested community. It then joins the communitythrough those contacts. After it joins the community,the new node publishes its resource ontologyknowledge in the community overlay. It also maps itsontology with other related ontologies through thecommunity overlay. Based on the ontology knowledgeindexed in the community overlay, nodes can share anddiscover resource information efficiently.

3.3. Peer ontology

3.3.1. Knowledge repository. Nodes use ontologies toexplicitly describe details of their possessed resources,as well as their knowledge of concepts andrelationships regarding the resources. In our system theontology knowledge is separated in two parts: theterminological box (T-Box) and the assertion box (A-Box). The T-Box statements describe concepts andrelationships between concepts, for example a set ofclasses and properties. The A-Box represents theconcrete knowledge about individuals within thedomain, for example the instances of the classesdefined in the T-Box. A node also uses inferenceengines to derive additional facts from instance dataand class descriptions.

3.3.2. Ontology mapping. Nodes in the communitymay use different ontologies. To reconcile the ontologydifferences, a node maps its ontology with those ofother nodes. The ontology mapping is created bymapping related elements from different ontologies.We define the following inter-ontology mappings:

() MC, >C2 Mc Ct_r cr=r

Mc: class mapping between class C1 and C2_: subClass mapping

: superClass mapping: equivalentClass mapping: referentialClass mapping

(2) Pi Mp >P24Mp `-p, DP,=p, -p

Mp: property mapping between property P1 and P2c: subProperty mapping_p

, superProperty mapping_p

equivalentProperty mappingp

.,: reverseProperty mapping

The defined mappings between different ontologieseither refer to the same concept, relation, or one is aspecial (or general) case of the other. We also note thatvarious ontologies may contain different supplementaryinformation about the same real world individual; thus

758

we add a special referentialClass relation betweenconcepts. This allows individuals to be merged ifspecific properties match, creating an aggregatedentity. The ontology mapping information is indexed inthe community overlay. When a node joins thecommunity, it uploads its ontological concepts andproperties to the community overlay, where it alsolearns knowledge of other related concepts andproperties. The knowledge gives hints and suggestionsto the new node when it maps its ontology to those ofothers.

4. Ontological directory overlay

This section describes details of the directoryoverlay: the first layer of the two-layer architecture.

4.1. Introduction

The function of the directory overlay is to assist theconstruction of community overlays. The directory isrepresented by the ontology hierarchy which is shapedby the high-level concepts within the whole resourceontologies. Nodes in the directory overlay are also usernodes that are stable and have good Internetconnectivity compared with the rest nodes. Conceptsare organized into a hierarchical directory, and thedirectory is distributed among the overlay nodes. Adirectory path starting from the root is used to representthe ontology domain (e.g., Isciencelcomputer science/AI).A problem we need to solve here is how to index

and search the hierarchical directory data. Manyapplications use hierarchical DHTs like Canon [16] orHIERAS [2] to index hierarchical data, but they are notapplicable to our system: specifically, in Canon,information is only stored in real nodes (leaves nodes),while all internal nodes are purely virtual; however,directory information in our system is stored in real(non-virtual) internal nodes. In addition, it is difficult toimplement directory browsing inside the Canonnetwork. HIERAS constructs the multi-level hierarchyby creating multiple DHT overlays for each sub-domain. Maintaining multiple DHTs requires a bigoverhead; it is a huge waste if the sub-domains are notlarge enough. In this paper, we propose a simple flatDHT structure to index the hierarchical directory. Thisstructure enables economical, flexible, and balancedlookup services.

4.2. Directory indexing and lookup

The directory overlay provides three lookupinterfaces: (1) exact path lookup, (2) directory browser-

based lookup, and (3) keyword-based lookup. An exactpath lookup query contains the complete directory pathof the interested domain, for example"/computerlhardware/CPU". The directory path of thequery is hashed to a key and then a correspondinglookup of the hashed key on the DHT is executed.However we can not expect a user knows the exactdirectory path to locate an interested domain.Therefore, we provide users a more flexible interface:directory browser. Users can expand a directory treenode to browse its child branches until they find thedesired directory entries. A node can also specify oneor more key concepts in its ontology and use them askeys to lookup the directory overlay. The node that isin charge of the keyword keeps links to thecorresponding directory entries. If multiple entriesappear under the same keyword, they are returned tothe user for further specification. Keywords areextended with WordNet [13] vocabularies andontology knowledge obtained from the communityoverlay.

Index by directorypath

-5 NI 0 nex by keywords

N9619

nIdex b directory path/entertainment/movie/science/animal

...

Index by keywordsCPUauthor

Figure 1. Directory overlay structure

We use Chord [5] to implement the directoryoverlay. As shown in Figure 1, for each directory path,sub path, or keyword, a hash value is computed usingan SHA- 1 algorithm. Each key is assigned to itssuccessor node, which is the nearest node traveling thering clockwise. Each successor node of the directorykey maintains a LRU cache storing information ofpeers that are interested in this directory. To implementthe directory browser's functionality, an overlay nodethat is in charge of a directory entry also stores thatdirectory's children information. When the userchooses one directory, Chord routes to that directoryentry and retrieves children directory information, suchthat the directory can be extended dynamically whilebrowsing. An overlay node also stores keywords thatare hashed to it, and links the keywords with relateddirectory entries.

Based on its semantic interests, a new joining noderegisters to the directory overlay. The overlay node in

759

charge of that interest returns this new node one ormore peer contacts from its cache. Then the new nodecan join the community through those contacts. A nodewith multiple interests can register with multiplecommunities.

4.3. Directory overlay load balancing

Nodes storing very popular directories run the riskof being overloaded by large amount of directoryregister requests. In this scenario, redistribution viashedding some local directories to other nodes does notguarantee any significant improvement of the situationsince even a single directory, popular enough, couldinduce an overload. Our system uses a replication-based method to solve this problem. Each nodeperiodically checks its current load. If the load is abovean overloading threshold, the node will pick a light-loaded node to replicate its directory keys. Since morethan one node is now responsible for a populardirectory, algorithms can be applied that wouldencumber each responsible node with only a fraction ofthe total load. Nodes then periodically exchange theircached peer information to make sure the communitiescreated with their assistance are well-connected.

As we indicated, key replication can relieve theregister load of nodes in charge of popular directories.However, in a DHT overlay, another significant sourceof workload is from relaying messages among nodes. Anode may be overwhelmed simply by the trafficinduced by forwarding incoming queries, such as in thecase of intermediary nodes on the lookup path ofpopular directories, which can be overloaded by havingto forward requests to the directories. To address thisproblem, a node n can actively redistribute its routingload by replicating its routing table to another node m.Future queries can then be routed to either n or m, andsince m has n's routing table, m can keep forwardingqueries to the right destination, maintaining thecorrectness of the network.

In order for the previously outlined scheme to be ofany use, however, replica nodes should obviously beadvertised in the network so that other nodes know andcan subsequently take advantage of their existence. ForChord-based DHTs, the replicas should be entered intothe finger tables of related nodes. Similar to the nodejoining process, the algorithm starts with the ith fingerof the original node n and then continues to walkcounter-clock-wise on the identifier circle until itencounters a node whose ith finger precedes n. The totalcost for the replica update, in terms of the number ofmessages exchanged, is O(log2 N).

5. Ontological community overlay

This section explains how to efficiently manage anddiscover resource knowledge in a relatively largecommunity using the DHT approach. Sharing in asmall community can be achieved using a similargossip-based strategy applied to a Gnutella-likeoverlay.

5.1. Resource knowledge indexing

5.1.1. T-Box indexing. Like the database schema, anode's T-box knowledge is more abstract, describingthe node's high-level concepts and their relationships.Basically, the T-Box knowledge includes classelements and property elements. They are indexed tothe community overlay by hashing the subject and theobject of their RDF triples. For example, a classassertion: <os><superClasss> <unix>, is first indexedby subject, and sends the following message to theoverlay:STORE [key, [("subject", <os>),

('>predicate ", <superClass>),("object", <unix >)}}

where key=SHAJHash("<os> ')In the message, the first attribute-value pair

("subject", os) is the routing key pair, and key is theSHAI hash value of os. Similarly, the triple is indexedby object as well, i.e., use the attribute-value pair("object", unix ) as the routing key pair. The targetDHT node stores the assertion and possibly generatesnew assertions by applying the entailment rules. Thesenew assertions have to be sent out to other nodes. Thus,after finishing this process, the whole T-Boxknowledge is accessible in a well-defined way over thecommunity overlay.

5.1.2. A-Box indexing. T-Box indexing does notrequire creating and maintaining oversized indices ofindividual instances. The downside of keeping only theT-Box information in the community overlay, is thatquery still have to be forwarded to many unrelated(from the instance-level) nodes. To improve theaccuracy of the query answering, nodes can index theirA-Box instances in the community overlay. The A-Boxindexing is based on indexing the RDF triples as well.We store each triple three times, indexing by thesubject, predicate, and object respectively.

There is a tradeoff between query overhead andindexing overhead. When the system has a highrequirement for fast and efficient query answering, ithas to pay more for the knowledge indexing; on theother hand, if the system does not index the detailed

760

knowledge, it has to explore more nodes for searchingthe query results. Applications should determine theright indexing granularity according to their specificneed.

5.2. Semantic query processing

Once a node has joined a community, it can queryinformation within that community. Our current systemuses RDQL [19] as the query language, which is basedon matching {subject, predicate, object} triples.A query is generally constructed with the user's

local ontology, and only nodes that use the sameontology can understand the query. To retrieve relevantdata reachable through other ontologies, the initialquery should be extended and reformulated based onthe inter-ontology relationships stored in thecommunity overlay.

Because of the distributed nature of the resourcedata, the system should be capable of breaking queriesinto appropriate sub-queries to be executed at differentsites and then piecing together the results to computeanswers to the original queries. The query analysisengine must use the ontological knowledge toefficiently navigate the search space as to reduce thesize of the data that is aggregated at each inter-nodaljunction in query processing.

6. Experiment

Our first set of experiments evaluates theperformance of the load balancing algorithm of ourdirectory overlay. The initial overlay network size is5000. Then directory paths are advertised to theoverlay. The directory lookup queries are Zipfdistributed, regarding the fact that most of the peers areinterested in popular directories but only a few areinterested in rare directories. Each node is assigned avalue (from 1 to 5) representing its capacity. We variedthe directory query frequency from 5 to 50 per timeslice and conducted a separate experiment for eachfrequency.

Figure 2 plots the mean and the 10th and 90thpercentiles of the peer workloadlcapacity ratio. Theresult can be used to represent the workload varianceson the peers. The smaller the difference, the better theload balancing performs. From Figure 2, we can seethat our algorithm greatly balanced the load of eachpeer. We also observe that, as we increase the queryfrequency, the variance becomes larger. This is becausedirectory lookups are highly skewed, introducing morequery will result in more imbalanced distribution ofdirectory accesses.

500400

P. 3000o 200 -

7ct 100 __________7_____________

query freq 5 10 15 20 25 30 35 40 45 50

(a) Without load balancing

500-400____________________300-200-

10

query fteq 5 10 15 20 25 30 35 40 45 50

(b) With load balancing adjustment

Figure 2. Mean, 10th and 90th percentiles of the ratioof load/capacity.

Figure 3 compares the performance of the T-Boxand the A-Box indexing in terms of query overhead andindexing overhead. There are totally 30 differentontologies. In the simulation, each node can randomlypick one ontology. From the figures we can see that,the A-Box indexing overhead is high, while thesearching overhead based on it is low. On the contrary,the T-Box indexing overhead is low but the searchingoverhead could be high.

200

150 - T-Box indexa, 100 A-Boxindex

050

node# 1000 1500 2000 2500 3000 3500 4000 4500 5000

(a)lndexing overhead: triples per node

D 500

o: 400

a. 300

gm 200

W- 100

- T-Box indemingA-Box indemxng

0

node# 1000 1500 2000 2500 3000 3500 4000 4500 5000

(b) Query overhead: messages per query

Figure 3. Comparison of the indexing and queryoverhead between T-Box indexing and A-Box indexing

Figure 4 illustrates the effect of ontologyheterogeneity on the performance of T-Box indexing.We noticed that the searching performance based on T-Box indexing is related to the system's ontologyheterogeneity: the more ontologies in the network, the

761

K

better the searching performs. This is easy tounderstand: when nodes have homogeneous ontologies,most nodes have the same T-Box knowledge; thenindexing T-Box cannot effectively distinguish nodes,thus query forwarding performs poorly. When thesystem has highly heterogeneous ontologies, T-Boxindexing can distinguish nodes' ontologies well;therefore query routing is more efficient.

15

0ontology #5 10 15 20 25 30 35 40 45 50

(a) Number of triples per node

5000 0.

400m i300 \E~.200o 100

0ontology # 5 10 15 20 25 30 35 40 45 50

(b) Number of messages created per query

Figure 4. Effect of ontology heterogeneity on T-Boxindexing

7. Conclusion

As more and more resources appear in grids, there isa compelling need to find an effective and efficient wayto discover and query these resources. In this paper, wepresented GOIDS, an ontological framework forresource integration and discovery in large-scale grids.The system provides a distributed ontological directoryservice to help nodes form communities according totheir semantic properties. As a result, searching cost isreduced by sending discovery requests only toappropriate semantic communities. Inside acommunity, nodes may use different ontologies torepresent their resource knowledge, and a distributedintegration mechanism was proposed to cope with themediation between different ontologies. To efficientlylocate the desirable resources in the community, aflexible DHT indexing scheme is provided. The mainalgorithms of the system were evaluated withsimulation experiments.

8. Reference

[2] Zhiyong Xu, Rui Min, Yiming Hu. "HIERAS: A DHTBased Hierarchical P2P Routing Algorithm". ICPP 2003[3] B. Y. Zhao, J. D. Kubiatowicz, and A. D. Joseph."Tapestry: An Infrastructure for Fault-Tolerant Wide-AreaLocation and Routing," Technical Report, UCB/CSD-01-1141, April 2000.[4] A. Rowstron and P. Druschel. "Pastry: Scalable,Distributed Object Location and Routing for Large-ScalePeer-to-Peer Systems," Proceedings of the IFIP/ACMInternational Conference on Distributed Systems Platforms,Middleware, November 2001.[5] I. Stoica, R. Morris, D. Karger, M. F. Kaashoek, andH.Balakrishnan. "Chord: A Scalable Peer-to-Peer LookupService for Internet Applications," ACM SIGCOMM,August 2001, pp. 149-160.[6] S. Ratnasamy, P.Francis, M.Handley, R.Karp, and S.Shenker. "A Scalable Content-Addressable Network," ACMSIGCOMM, August 2001, pp. 16 1-172.[7] Berners-Lee T, Hendler J, Lassila 0 (2001) The semanticweb. Sci Am 284(5):34-43[8] Stevens,R.D., Robinson,A.J. and Goble,C.A. (2003)myGrid: personalized bioinformatics on the information grid.Bioinformatics.[9] OntoGrid project: http:Hwww.ontogrid.net/[10] M. Arumugam, A. Sheth, and I. B. Arpinar. Towardspeer-to-peer semantic web: A distribuited environment forsharing semantic knowledge on the web. In Proc. of theInternational World Wide Web Conference 2002.[11] S. Castano, A. Ferrara, S. Montanelli, and D. Zucchelli.Helios: a general framework for ontology-based knowledgesharing and evolution in P2P systems. In IEEE Proc. ofDEXA WEBS 2003 Workshop, Prague, Czech Republic.[12] Ora Lassila and Ralph R. Swick, "W3C ResourceDescription framework (RDF) Model and SyntaxSpecification".[13] G. Miller, "WordNet: A lexical database for English",Communications of the ACM, vol. 38, no. 11, 1995.[14] M. Cai, M. Frank, J. Chen and P. Szekely, "MAAN: AMulti-Attribute Addressable Network for Grid InformationServices." The 4th International Workshop on GridComputing, 2003.[15] lamnitchi A, Foster I, "On Fully Decentralized ResourceDiscovery in Grid Environments," Proc. The 2nd IEEE/ACMInternational Workshop on Grid Computing 2001, Denver.[16] P. Ganesan, K. Gummadi and H. Garcia-Molina,"Canon in G major designing DHTs with hierarchicalstructure", Proceedings of the 24th ICDCS, Tokyo,2004.[17] I. Horrocks, F. van Harmelen, and P. Patel-Schneider.DAML+OIL.http:Hwww.daml.org/2001/03/daml+oil-index.html[18] W3C. Web-ontology (webont) working group.http:Hwww.w3.org/2001/sw/WebOnt/.[19] A. Seaborne. "RDQL: A Data Oriented Query Languagefor RDF Models."www-uk.hpl.hp.com/people/afs/RDQL/, 2001.

[1] Watts, D.J., Strogztz,S,: Collective dynamics of 'small-world' networks. Nature 393 (1998).

762