Huge volumes of unstructured and extensible data are pushing the limits of traditional BI

Traditional BI has significant technical limitations

Why Existing Business Intelligence Products are Inadequate The Next Generation of BI Has Arrived

The Changing Business EnvironmentBusiness Intelligence (BI) technologies and solutions have been around for more than a decade, but never has the information environment they address been more complex. Today’s information environment has evolved to encompass new information channels, new sources of data, and new analysis and reporting imperatives.

New communication channels such as web, e-mail, voice over IP (VOIP), cellular and wireless networks, chat, instant messaging, text messaging, and video on demand (VOD) not only offer new ways to drive revenue, customer satisfaction, and quality of service, but also introduce huge volumes of data into the system. Eighty percent of data today comes from one of these new channels either as unstructured data (data that doesn’t neatly fit into a predefined format, such as email), or extensible data (data that can be dynamically added to or extended in real time, such as web logs).

Corporate data governance requirements, driven by increased security and risk exposure, compel companies to identify and manage all data transfers across these channels as well as other system channels, such as firewalls, intrusion detection systems (IDS), anti-viruses, security card swipers, and surveillance systems.

While most businesses are eager to turn this new data into useful information, many find that their current BI technology is not up to the task. Traditional BI technology, designed for a much simpler data analysis landscape, cannot accommodate the data from new channels.

Limitations of Traditional BI TechnologyTraditional BI infrastructure has inherent technological constraints that limit its ability to deliver robust and thorough data analysis.

The first issue is that data must undergo multiple transformations in a traditional BI system before it can be analyzed. Data from transactional systems is in a form that is best suited for data collection (relational models), but data for analysis needs to be in a very different form (star schema, snowflake). Data must therefore first be moved from the place where it is being collected (transactional systems) to the place where it is being analyzed (data warehouse). Another data transformation then occurs when data from the data warehouse is brought into an analytical model for aggregation, consolidation, and roll-up. Only then can the actual analysis and reporting take place. The result is an excessively slow time to analysis.

Data integration is also a huge technical challenge. New information channels have overlapping business objectives, and the data from these channels must be analyzed together and correlated in order to understand their impact on a particular aspect of the business. With traditional BI, data from different channels must first be brought together into a common model in order to analyze and report on it. This adds another layer of complexity to the picture and further delays analysis.

Traditional BI no longer meets today’s business needs

Finally, these technical limitations exist in an environment where data volumes are skyrocketing. Non-relational data is often produced in volumes that hugely exceed the typical output of a transactional system, creating a data management nightmare even before this data becomes an analytical challenge.

Typical Work-Around Solutions

Vendors of traditional BI technologies often recommend and apply work-around solutions to deal with these challenges:

1. Reduce the amount of data pulled into the BI system. The approach favored by Hyperion, Microsoft, Business Objects, and Microstrategy entails employing a number of ETL (Extract Transform & Load) techniques to force a subset of non-relational data into a relational format. This approach not only causes significant loss of information, it also strips data of useful metadata that could be utilized during data analysis and reporting.

2. Introduce unconventional data at the final reporting stage. Cognos in partnership with Composite Software is employing this approach. While this approach does not cause any information loss, it does not allow for analysis of a very significant data segment, such as XML, HTML, HTTP, e-mail, and text.

3. Build a fixed analytical model with fixed data mapping. The main drawback of this approach is total lack of flexibility. Any additions or changes to analysis or data sources requires a re-do of both the model and the mapping. There are hundreds of vendors utilizing this approach. Most of them offer analytical applications as point solutions and do not scale beyond a single solution.

4. Simplify the data transformation processes by keeping the entire BI stack on a single hardware appliance. Netezza is the prominent example of this type of approach. While performance benefits are achieved, by streamlining the data movement through the BI stack, the appliance approach shares the same drawbacks of a traditional BI architecture.

Picture 1: A traditional BI stack, with its many disparate components, forces data through multiple transformations as it moves from model to model

BI vendors are trying to address the new challenges without solving the core problem

New Business Environment Calls for New BI TechnologyCompanies are sitting on a goldmine of data and are ready for a radical new approach. Today’s business environment requires BI technology that can:

• Operate on both relational and non-relational data at the same time without needing to transform this data into a common relational format.

• Produce data aggregates, summaries, and roll-ups without having to move data at all.

• Deliver business insight without a lengthy data modeling process.

• Analyze data regardless of volume.

• Move beyond simple metrics reporting to provide root cause analysis and a much deeper level of business insight.

Skytide—The Next Generation of BIThe Skytide analytical platform (eXtensible OLAP, or XOLAP) was designed to address these data analysis challenges and deliver a BI solution for today’s next-generation data environment.

Skytide’s breakthrough technology uses XML as a common layer to tie together different pieces of the BI stack into a single, cohesive analytical framework. XML is used to describe every aspect of Skytide Analytical Platform, which dramatically reduces complexity while offering advanced functionality that cannot be achieved by traditional BI technology:

• Skytide renders all the models in XML. There is no need for multiple data transformations, multiple data models or multiple analytical models.

• Skytide can operate on data directly where it resides. Data does not need to be moved and stored for analysis.

• Skytide streams and analyzes XML-rendered data in a single step regardless of data volume.

• Skytide automatically generates analytical models by using XPath to describe business rules. This in turn produces analytical models when executed against the data.

• Skytide’s dynamic analytical models facilitate complex analysis such as:

– Correlation analysis – Hierarchy analysis – Path and traffic analysis – Entity matching analysis

Picture 2: Skytde eXtensible OLAP, or XOLAP, is driving business decisions across all corporate data sources in a fraction of the time and cost of traditional BI solutions

Skytide XOLAP works across all data sources regardless of the format and size, while delivering analytical results through a conventional presentation layer

© 2007 Skytide, Inc. All rights reserved. Skytide and the Skytide logo are registered trademarks of Skytide, Inc. All other trademarks are the property of their respective owners.

Skytide, Inc.

1820 Gateway Drive, Suite 300, San Mateo, CA 94404

Phone: 1.650.292.1900

Fax: 1.650.312.1400

E-mail: info@skytide.com

Internet: www.skytide.com

Picture 3. The Skytide XOLAP architecture is built on open industry standards that facilitate easy integration into any business technology environment.

Skytide is architected for

• Extended data access

• Modeling engine

• Unique analytical capabilities

• No schema dependency

• XML-based environment

• Standard presentation layer

• Open, standards-based API SDK (MDX, ODBC/JDBC)

Skytide analytical platform features:

• Complex extensible data

• Instant insight

• Rapid analytical modeling

• End-user driven process

• No IT, ETL, or data warehouse

Its unique analytical capabilities include:

• Business rule and data driven modeling layer based on XML and XPath

• Reusable analytical models based on analytical engines

Fast Deployment and Ease-of-UseSkytide not only reduces the complexity of the BI stack, but is also easy to install with very little footprint. A complete package consists of the Skytide Server and the Skytide Designer. Skytide can also be utilized in conjunction with—or instead of—traditional BI and data warehousing technologies.

Skytide’s intuitive system is easy for business users to learn. They simply point Skytide at the data sources and then use Skytide’s point-and-click interface to define business rules for analysis. The server uses business rules and data sources to generate the analytical model and to populate it with analytical results. The analysis and reports can then be rendered through the Designer, Excel or any third-party reporting tool.

About SkytideSkytide enables organizations to answer a new array of difficult questions about revenue opportunities and operational risk more quickly and thoroughly than ever before. The patent-pending Skytide Analytical Platform is agile enough to respond to rapidly changing intelligence needs, and powerful enough to perform multi-dimensional analysis on the most complex and dynamic data. Skytide customers are able to perform new types of analysis that are impossible to perform using traditional analytical tools. Skytide is a privately held company funded by Granite Ventures and El Dorado Ventures.