development of a new indexing technique for xml document retrieval
TRANSCRIPT
Development Of a New Indexing Technique for XML Document Retrieval
by:
Amjad Ali Amjad
Agenda
Introduction
Background
Problem Statement
Proposed Solution
Results and discussions
Conclusion and future directions
Introduction
What is XML
XML is a markup language much like HTML
XML was designed to describe data
XML tags are not predefined. You must define your own tags
XML uses a Document Type Definition (DTD) or an XML Schema to describe the data
Introduction (continue)
Example Doc 1:<invoice>
<buyer>
<name>ABC Corp</name>
<address>1 Industrial Way</address>
</buyer>
<seller>
<name>Acme Inc</name>
<address>2 Acme Rd.</address>
</seller>
<item count=3>saw</item>
<item count=2>drill</item>
</invoice>
Introduction (continue)
Database management systems are increasingly being called upon to manage semi-structured data: data with an irregular or changing organization
Semi-structured data is often represented as a graph (tree structure)
Evaluating queries over semi-structured data involves navigating paths through this relationship structure
Introduction (continue)
Index Construction for efficient access
Traditional indexing Techniques are
applicable.
Parent child nesting relationship
Expensive querying for this representation even with indexes
Introduction (continue)
Building specialized data manager
For semi-structured data repository
Example projects LORE, Tamino,
XYZFind
Update causes a re-computation
How to deal with update problem
Introduction (continue)
Relative Region Co-ordinate
Knowledge of start and end position
Within the parent element
Only need to update the portion of the index file
Value indexes on attribute values
Introduction (continue)
Term-based inverted indices on element content when this is a large piece of text.
Index on tag name (i.e given a tag name we can return all the elements with the specified tag.
Background
Position based indexing
Queries are processed by manipulating the range of offsets of words, elements or attributes.
In path-based indexing, the location of words is expressed as structural elements and the paths in tree structures are used for the processing of query.
Background (continue)
Bitcube: A three dimensional indexing for XML Documents
According to this technique documents can be hierarchically represented by XML elements. XML documents are represented and indexed.
Background (continue)
Content and Structure in indexing and ranking XML
Index structures with a ranking support are therefore needed for fast access to relevant parts of large documents collections. An analysis reveals that ranking parameters related to both the content and structure of data are poorly supported by most known XML indexes.
Background (continue)
Ctree
It provides an indexing structure that is based on two levels: path summary and detailed element-level relationships. The first one, the path summary, is a tree that is
distracted from the original data
Background (continue)
Indexing for XML Siblings:
Given the importance of XPath based query access, Grust proposed R-tree index, we refer to as whole-tree indexes (WI). Such index, however, has a very high cost for the following-sibling and preceding-sibling axes. In this method they develop a family of index structures, which refer to as splittree indexes (SI), to address this problem, in which (i) XML data is horizontally split by a simple, yet efficient criteria, and (ii) the split value is associated with tree labeling.
Background (continue)
High-performance XML Storage/Retrieval System.
The basic idea of this technique is to allocate a field ID to each text data item of the XML element and to register it in the structure index and text index. The structure index manages the hierarchical structure of each field, and the text index manages the field ID and document ID in which query words appears. The structure index is one big data tree and represents
the overlapped structure of documents.
Background (continue)
Indexing documents for queries on structure,content and attributes
It Explains position-based indexing and path-based indexing to access XML document by content, structure, or attributes.
Background (continue)
Extensible index technique
An extensible index technique is proposed to express position information between nodes in a XML document. It is an efficient index technique that simplifies the comparative object applied to a search query and minimizes the reconstruction of index structure by update operation. In addition, they specially proposed extensible index technique with deferred update.
Problem Statement
Support of element addressing
Doc.ID should include NodeId (Xpath) + Offset
Index size becomes very large
Xpath are long
Support of typed data
Integer, float, simple types of XML schema
Requires classical indexes for certain elements
Problem Statement (continue)
Query processing
Structural joins
Text search
Exact search
Support of updates
Incremental updates would be a plus
Problem Statement (continue)
Evaluation criteria Identifiers
Per node or per document
Descendant/Ancestor Search By join algo. By graph traversal By OID comparison
Keyword Search By element scan By B-tree traversal
Update Incremental
Index size Entry number Entry size
Problem Statement (continue)
indexing structures use which the absolute address to pinpoint where data resides,
update causes a re-computation
If the update frequency is high the cost of reconstruction is unbearable
Support of updating the indexes is not considered in most of the indexing techniques.
Problem Statement (continue)
Updates are an issue in any such labeling scheme. It is conceivable that a complete re-labeling could be required for each update,
the existing techniques do not support the storage of multiple documents in a single time.
Proposed Technique
An XML document instance is a plain-text file that uses markup delimiters (tags) to define the logical structure of a document in a hierarchical fashion.
Robert Korfhage proposed three purposes of indexing in IR, which can best take advantage of structured documents.
To permit easy location of documents by topic;
To define topic areas and hence relate one document to another;
To predict relevance of a given document to given information need.
Proposed Technique (continue)
The current structured query and indexing models for XML have not fulfilled these requirements.
The ideal system seems to be one that will provide efficient and comprehensive indexing of document content and structure, and be able to support the predicted degree of relevance all matching documents have to a particular query
Proposed Technique (continue)
There is a node corresponding to each element, with child nodes for sub-elements. However, all attributes of an element node are clubbed together into a single node, which is then stored as a child node of that element node
The content of an element node, if any, is pulled out into a separate child node.
Proposed Technique (continue)
Ancestor–descendant relationship
a node(S1,E1,L1) is the ancestor of node (S2,E2,L2) Iff S1<S2 ^ E1>E2
Parent–child relationship
a node (S1,E1,L1)is the parent of node(S2,E2,L2) iff S1<S2^E1>E2 ^L1= L2-1
Proposed Technique (continue)
S1 and S2 are start labels, E1 and E2 are end labels, and L1 and L2 are levellabels in these formulae.
We address the update issue by leaving gaps between successive label values.
Results and discussions (continue)
System architecture
Data Parser
The Data Parser takes an XML document as input, and produces a parse tree as output.
Data manager takes each node of tree mark its indices and store it into database.
Results and discussions (continue)
If the node is of mixed type, with multiple content parts interspersed with sub-elements, each content part is pulled out into a separate child node.
All processing instructions, comments, and such are simply ignored
Conclusion and future directions
Reconstruction of index file due to a partial update is a problem that XML database applications inevitably have to face
We have developed the indexing system that is based on the two indexing techniques extensible index technique and the relative region coordinate based indexing of XML documents with our own proposed scheme which assigns the level numbers to each node of XML documents and document number to each document.
Conclusion & future directions (continue)
Update of the index structure which
increases the cost is successfully removed as the index structure remains unaffected after adding the new nodes.
Parent child and ancestor-descendent
relationship could be found easily for efficient retrieval.
Conclusion & future directions (continue)
all processing instructions, comments, and such which are simply ignored. In a future, it could be created yet another child node of the element node with all such data.
An index that is efficient for both update and retrieval may not available.
Conclusion & future directions (continue)
One of alternatives is building two separate indices such that one is suitable when update is frequent, the other is better at query processing.
In this case, a transformation mechanism between the indexing structures is needed to be developed.