achieving adaptivity for olap-xml federations torben bach pedersen aalborg university joint work...

Achieving Adaptivity for OLAP-XML Federations

Torben Bach PedersenAalborg University

Joint work with Dennis Pedersen, TARGIT

Torben Bach Pedersen · DOLAP 2003 · 04/18/232

Overview

• Background: OLAP-XML federations• New challenges

– XML data changes– Slow or unreliable XML sources– Schema changes in data sources– Other challenges

• Integration in TARGIT architecture• Other applications of the techniques• Conclusion and future work• Related work


Data Warehousing & OLAP

Multidimensional analysis: TARGIT Analysis


OLAP

• Good for complex ad hoc queries– Simple: natural, graphical queries– Fast: pre-aggregation

• A number of problems with physical integration– Short-term and varying data needs

• Population, product info, ...

– Dynamical data• Stock quotes, competitor pricing, ...

– Data with limited access• Competitor product info, public databases, ...


OLAP-XML Federations

OLAP-server

Client

Cube

Traditional OLAP architecture:



• Logical integration of XML data– External dimensions– External measures

• Data combined at query time

Federation

Client

XMLCube



• Logical integration of XML data– External dimensions– External measures

• Data combined at query time• Transparent for users• Flexible: many XML sources• Quick: running in a few mins• Data is always fresh• Performance often comparable

to physical integration

Federation

Client

XMLCube Cube


XPath Queries for Fetching XML<Books>

<Book>

<Title>1984</Title>

<Author>Orwell</Author>

</Book>

<Book>

<Title>Of Mice and Men</Title>

<Author>Steinbeck</Author>

</Book>

</Books>

/Books/Book[Author=”Steinbeck”]/Title

Federation

Client

XMLCube Cube

XPath

Dimension value


Old And New TARGIT Architecture


New Challenges

• Our previous work focused on basic aspects– Flexibility– General performance– Implementation

• New: what can go wrong? – need for adaptivity– XML data changes– XML sources slow or unreliable– Schema changes (XML, OLAP, federation)

• We often have no control over the XML sources• A solution has broad interest: views over XML sources


XML Data Changes

• Basic federation– XML data is integrated at query time => XML data

changes handled automatically• However, XML data is cached for performance

– Cache timeout value ensures fresh data (set manually or automatically)

– 0 cache timeout => always fetch from source• Only few current XML databases inform about changes

– Xyleme allows users to subscribe to changes– Only delta should be transferred


ICE: Information and Content Exchange

• Protocol proposed by W3C for automatically informing about and requesting changes– Supported by major vendors– Push: subscribe to changes and keep cache up-to-date– Pull: request changes from source at query time


Slow and Unreliable XML Sources

• Overload, maintenance, HW breakdown, attacks– Often we no influence on this

• Incremental presentation for user

– What if source is too slow or no reply at all? • Inform user that the system is not working…?• Specification of alternative sources

– Several queries per external dimension/measure– Increased fault tolerance, also better performance

Source Server Client



• Start several queries and use the fastest

– Always uses the fastest, but heavy load on sources– Use first response time as indicator for total time

• Start one query at a time

• Minimal load on sources, but slower

Fed Fed

Fed Fed

?Fed

1

2

3

1

2

3

3

1

2



Alternative sources of lower quality: better than no data?

Alternatives

– Expired cache data

– Google, Xyleme, The WayBack Machine

– Backup-disk, tape

– Etc.

Source Speed Quality

Local cache Fastest Fresh

Original source Fast? Freshest

Expired cache Fastest Old

Backup source Fast/slow Very old



In practice?

Sources with equal priority chosen at random


Result: Algorithm for Fetching XML Data


Experiments

• 1st experiment: fetching a 137 KB dimension– Start 8 queries, when first 3 respond, (cancel) last 5,

when fastest query finish, (cancel) remaining 2– Fast reply = good indication of overall speed

• 2nd experiment: search local cache, then Google cache


Schema Changes In XML Sources

How to synchronize XML views after schema change? (solution described in separate paper)

Bibliography

Publisher

PName Book

Author

AName

Title Price

/Bibliography/Author[AName=”Orwell”]/Book/Title

Bibliography

Publisher

PName

Book

Author

AName

Title Price


Additional Challenges

• Changes to federation schema– Cache may be invalidated– Discard affected cache results (unproblematic)

• OLAP data changes– Cache may be invalidated– Less frequent than XML data changes => cache will

often have expired anyway• OLAP schema changes

– Federated schema may be invalidated– Rare and easy to detect (and correct)


Integrating Techniques - Architecture


Integrating Techniques – Query Processing

• Query Evaluator splits query into XML+OLAP parts and determines query plan based on cost

• Execution Engine coordinates and executes plan• Cache Manager maintains cache, e.g., through ICE• XML Component interface fetches XML data, chooses

between available XML sources (Algorithm 1)• View Synchronizer handles schema changes• Metadata Manager manages info about external

dimensions and measures + XML component characteristics


Other Applications

All XPath-based views on XML data

Links to parts of XML documents• Web pages• Documents (DocBook)• Software applications

and many more…

Automatic recreation of broken links

Increased fault tolerance and performance using alternative sources

?


Conclusion and Future Work

• Operational problems in OLAP-XML federations• XML data changes• Slow and unreliable XML sources

– Using several sources (Algorithm 1)– Experiment with Algorithm1

• Techniques integrated into federation architecture• Schema evolution and other challenges• Future work

– TARGIT implementation and testing– Using techniques in other applications


Related Work

• Data changes in XML/semistructured documents– Xyleme + Zhuge

• Schema changes in scientific documents– Not XML

• Adaptive/dynamic query optimization– Telegraph project– We use once per source, rather than per tuple

• Does not consider one or more of: OLAP+XML concepts, schema changes, slow and unreliable sources

• Own previous OLAP-XML work is not adaptive

achieving adaptivity for olap-xml federations torben bach pedersen aalborg university joint work...

Documents

fetching xml

torben bach pedersen

data sources

xml sources quick

unreliable schema changes

current xml databases

mins data

targit slide