7 Seven Trends That Will Change Business Intelligence As We Know It

Introduction 3

Trend #1: Self Service BI 4

Trend #2: BI Goes Mobile 5

Trend #3: Collaborative and Social Features 6

Trend #4: Business Intelligence in the Cloud 7

Trend #5: Open Source Takes Over 8

Trend #6: Big Data 9

Trend #7: Real-time Insight 10

Conclusion 11

Learn More 11

Contents

Introduction

After years of relative stability, the world of business intelligence is now experiencing a sea change.

The old rules of thumb no longer apply, and companies cannot assume that yesterday’s BI strategies continue to be viable. Consider these fundamental shifts:

• Wemanagedifferently.Instead of relying on instinct, today’s businesses place more stock in number-crunching and objectively measur-ing performance.

• Usersexperiencedatadifferently.Most people have used web-based applications to manage and interact with their personal data for years. These “consumer grade” applications rely on an intuitive and highly interactive experience with near-instantaneous response.

• Greatexpectations.From MapQuest, Gmail and Facebook to more specialized solutions including those for blog traffic analysis and portfolio management, these tools have raised expectations for end users. If we want users to be engaged, our BI apps had better do the same.

• Informationmanagementtechnologyisleapingahead.Now it’s possible to derive genuine business insight from large quantities of data more quickly, for dramatically less cost and effort than ever before.

Theruleshavechanged.

Collectively, these changes mean that many of the old BI technologies, architectures, and approaches no longer apply.

Now, businesses want to make fact-based decisions, based on the analysis of data from a variety of data sources, in ways never before possible. Users want a compelling, effective experience. And the technologies to support both of these are quickly coming online.

Theoldrules.

You’ve heard them—you may even have repeated them aloud:

• “The average end-user isn’t sophisticated enough to do their own analysis.”

• “Many important analysis approaches are simply not technically feasible.”

• “Real-time reporting and analysis aren’t feasible: they require exorbitant budgets and performance-killing access to trans-action systems.”

• “Comprehensive BI is an expensive game, requiring pricey software and hardware.”

Thenewrules... are being written now. The changes are significant. So let’s take a look at seven trends that will force us all to re-think—and re-tool.

Reporting and analytics user expectations have changed, More and more end users create and modify their own output and Dashboards and mashboards abound

In the past decade, most of us have become true information consumers—at least in our personal lives. We use data such as historical prices, ratings and scores to decide which movie to see, which schools to attend, to how much to pay for a home. We’re able to access and analyze this data because our favorite Internet applications have evolved familiar, easy-to-use, and intuitive interfaces for data exploration.

This mentality—and the expectations it fosters—have spilled over into our business lives. Users have become more sophisticated—and more analytical. And managers—at all levels—want to make quick, accurate business decisions based on hard data rather than intuition.

Not long ago, managers were accustomed to—and believed in—printed reports. When deeper understanding was required, they relied on business analysts: power-users who wrote SQL code, defined and revised reports, and (occasion-ally) employed complex tools to explore the data.

Butthatwasthen.Thisisnow.

Today, business users (including managers) expect to do more themselves, depending less on business analysts—who have, in turn, become more specialized, focusing on proactively identifying trends and opportunities instead of building reports on demand.

The reports have also changed. No longer static, they’re highly interactive, allowing individual users to access them via the web, drilling, filtering and modifying them as needed.

Analysis has changed, too. Now, sophisticated data visual-ization tools—commonly imbedded with an application or based in-memory to enable setup ease—deliver at-a-glance insight.

Theriseofdashboardsandmashboards.

With the consumerization of information access and interac-tivity, there have evolved better methods for users to com-bine, share and interact with data. Users can now personalize their dashboards to meet their needs, including combining external data with internal corporate data. They can also interact with a wide range of powerful visualizations:

• Gauges and meters deliver at-a-glance info.

• Conditional formatting highlights exception conditions and simplifies visual navigation, helping users pinpoint specific items of inter-est.

• Sparkline charts help users quickly pinpoint departures from trends.

Today,dashboardsarereal-time,making them useful for displaying operational data. And users can interact with them without any need to involve busy IT resources.

In Web 2.0-speak, these are called Mashup applications. No longer does a one-size-fits-all conglomeration of a few im-portant (though frequently disjointed) reportlets, designed by a single person in IT, make sense. Today’s dashboards deliver important new capabilities that are personalized, interactive, and immediate.

Doingmorewithless.

Self-service reporting may be the most effective means at IT’s disposal for improving its own BI productivity. In a recent TDWI survey, 66% of IT professionals said implementing self-service BI was high or very high in a list of potential cost-saving measures. Why? Users get reports more to their liking, more quickly—while IT frees up resources to work on more strategic initiatives.

A Self-service BI checklist. As organizations examine BI solu-tions that can support self-service business intelligence, they should look for:

• Interactive Dashboards with drill-down and drill-through capabilities.

• Dashboard frameworks that let individual us-ers choose dashboard layout and content.

• Flexible dashboard designers that can easily combine relational, non-relational and public addressable information.

• Parameter-driven and column-based filtering tools that let users focus on specific data.

• Granular data security with column and row suppression capabilities.

• Advanced visualization and easy-to-use chart-ing.

• Conditional formatting.

Trend 1Self-Service BI

Until recently, few businesses could deploy mobile BI. In a mid-2008 Aberdeen Group survey, only 17% 1 of companies were delivering BI data to mobile phones—though fully 78% were interested in eventually doing so 2.

Here’s why:

• The standard approach to mobile BI required display of reports via web browser—but de-vices rendered HTML inconsistently, and often quite poorly.

• The displays on mobile devices were too small to allow practical report viewing.

• Network speeds were too slow (until the wide spread adoption of 3G).

• Most mobile devices allowed limited in-teractivity, preventing users from filtering, re-sorting, drilling down or performing other important actions.

• Although users were interested, they weren’t yet feverishly demanding mobile BI.

• Rapid evolution of mobile devices and plat-forms added complexity.

Nearly all of these obstacles have been overcome in the last two years—in part due to the introduction of advanced mobile devices (iPhone, iPad, Android) and low cost, higher bandwidth wireless networks.

• The proliferation of iPad and iPhone apps has spurred demand for anywhere- anytime information access.

• Web pages can be displayed as designed, without the re-rendering required by less-capable mobile phones.

• 3G and 4G networks now offer significantly improved download speeds.

• Touch-screens have improved the user experi-ence.

Aberdeen’s 2010 study reveals starkly different attitudes from two years previous, with 23% of 146 companies having deployed, at this point, a mobile BI application or dash-board—and an additional 31% planning to do so within the next year.3 By mid-2011, over 50% of companies will likely be using some form of mobile BI.

Most of these deployments will allow users to view reports and perform lightweight analyses via “small, portable dash-boards”4 —but will not support creating reports or heavy-data exploration. Many will use alerting or notifications, to let the on-the-go user immediately know when data begins to stray significantly from the norm.

In short, the Mobile BI5 era has arrived. It is quickly becoming a must-have for many business intelligence initiatives, espe-cially where there’s a need to disseminate critical, real-time business information to large numbers of people.

In the 2010 Aberdeen Group survey, nearly a third of respon-dents intended to augment (or abandon) current BI vendors in support of Mobile BI because their current platforms were inadequate.

Organizations searching for a BI solution with strong support for Mobile BI should focus on tools that:

• Render reports within a web-browser, using HTML formatting that’s visually strong, both on computer monitors and on a wide variety of mobile devices.

• Facilitate the creation of custom BI apps aimed at specific devices.

• Support some level of report and analysis interactivity—especially sorting, filtering and hyperlinking—on mobile devices.

• Offer strong support for business rule-driven alerts and notifications.

• Feature well-designed security infrastructures that protect and withhold sensitive data from unauthorized mobile users.

1 Computerworld, “Business intelligence goes mobile”, July 14, 2010.2 The Aberdeen Group: 11/30/2010 “Mobile Business Intelligence: A Path to Pervasive BI”3 ComputerWorld August 9, 2010. “Business intelligence apps go mobile,” by Michael Fitzgerald.4 Monash, Curt: DBMS2.com, “What matters in mobile business intelligence” 7/15/2010.5 Dresner Advisory Services, LLC. DAS Mobile Business Intelligence Market Study. 2010.

Trend 2BI Goes Mobile

Trend 3Collaborative and Social Features

Many BI users have been taking advan-tage of social networking venues like Facebook, Twitter, and LinkedIn for years, and have come to expect “social” functionality in all applications—and for good reason.

Over the last few years, social networking has in fact explod-ed—and not just among young people (Twitter is seeing its highest growth rates among users aged 39 to 51).

Users expect apps—BI included—that help them more effectively collaborate with others, rapidly improve the understanding of content and make better-informed deci-sions. Employees and customers, who use the same social networking features they enjoy outside of work, are able to reach new levels of creativity.

Here are some examples of collaborative and social features as they’re already being applied to business intelligence:

• Report tagging lets individuals help others to more quickly and easily find the reports they need.

• Comment and discussion threads let users ask questions of report authors, share views on report value, offer instructions for use, suggest improvements and more.

• Annotation features enable users to make reports more meaningful by setting context and providing background information; think of a store manager linking a given product’s low sales to supply problems.

• Up- and down voting enables users to more rapidly locate useful content by exposing a community’s assessment of a data point and/or report value.

• User-provided ratings on report designs, dash-boards, etc., give feedback to creators and help guide future output for consumers.

• Privacy settings enable users to selectively share their reporting, analysis and dash-boarding designs and output with the right audience.

• Workflow features enable rules-based busi-ness intelligence collaborative processes including review and feedback, approval rout-ing, decision tracking and more.

Trend 4Business Intelligence in the Cloud

Why is the cloud topic one? Because it provides unprecedented flexibility in de-ploying applications, and because it has a huge potential for reducing costs.

With cloud technology, companies generally pay only for the hardware, software, bandwidth and other resources they use—and not until they actually use it. Adjusting capacity is easy, and adjustments are immediately reflected in costs. “The cloud” is an umbrella term for three different ways to deploy software:

1.SoftwareasaService (SaaS). Here, the focus is on applica-tion software. Customers pay a periodic (usually monthly) subscription fee to access application software hosted by a solution provider; access is through a web-browser. Subscribers and end users avoid the issues of hardware setup, software installation, configuration or upgrades. Salesforce.com is an example of a SaaS product.

2.PlatformasaService (PaaS). With PaaS, subscribers pay for everything they need to build, maintain and run their own applications; everything is hosted by the provider, and accessed via web browsers.

3.InfrastructureasaService (IaaS). IaaS lets organizations “rent” access to hardware, deploying the applications they build or buy to these off-site servers. The subscriber performs some limited aspects of server management. Amazon Web Services provides a form of IaaS: access to fractional servers on a pay-as-you-go basis.

The first two approaches—SaaS and PaaS— are most rele-vant to Business Intelligence. In the SaaS model, for example, organizations could subscribe to a specific analytic applica-tion, with all user access via browsers. The same hardware infrastructure and underlying software platform are used by all subscribers, all end users. Subscribers have nothing to install, configure or maintain beyond the devices on which their browsers run.

In the PaaS model, the subscriber develops, implements and operates any specific applications, using the hosted devel-opment and runtime resources; access to those applications is via web browsers. BI architectures that are standards-based—and easy to integrate via web services—will be best suited for the PaaS model.

TheroleoftheCloud.

In a recent TDWI survey, 31% of survey respondents saw cloud deployments as a major way to work with 2010’s rela-tively small IT budgets.6

Companies are learning to use cloud-based BI creatively to meet business goals and lower costs. Here are some of the reasons:

• Rapid, hands-on evaluation of business intel-ligence solutions.

• Reduced costs during the development cycle, thanks to the ability to pay for fractional shares of dedicated servers.

• Elimination of capacity planning—and the shortages or overspending that accompany it. Organizations can simply pay for the comput-ing capacity their application uses, as it’s used.

Organizations searching for cloud-based BI should look for:

1. Web-based user interfaces and access methods.

2. A broadly available environment, addressed from a variety of global locations and through and a wide range of services, preferably using web-based APIs.

3. Pay-as-you go pricing (the best way to gain the full eco-nomic benefit of cloud computing).

4. A secure environment—at network, identity management and authentication layers.

5. Elasticity: the ease of adjusting hardware capacity to meet changing needs.

6. Full support for APIs consistent with open web standards.

7. SaaS solution providers—or those who may go that route in the foreseeable future—should seek multi-tenancy features.

6 TDWI Best Practices Report: BI on a Limited Budget: Strategies for Doing More With Less, By Wayne W. Eckerson, July 2010

• Responsiveness to occasional or periodic heavy computational workloads such as quarter-end or year-end analyses

• Fast and cost-effective means of executing software trials or proofs of concept, and of deploying short-term sandboxes for one-off analytic needs

Trend 5Open Source Takes Over

Open source is now becoming main-stream in business intelligence. The tools are mature, proven, and making deep inroads in companies large and small.

In a TDWI Technology Survey in May, 2009, more than one-third of organizations reported using open source software in business intelligence, data integration or data warehous-ing applications.

Just as the LAMP stack (Linux, Apache web server, MySQL database, and PHP/Perl/Python scripting) technology stack has come to dominate the development and deployment of web-based applications and websites, the BI sector is de-veloping de facto open source software stacks as well. These typically include open source database, data integration and business intelligence tools.

Why has open source BI become so popular?

1.Lowcost.87% of recent Gartner survey respondents expected considerable TCO (total cost of ownership) savings from using open source. With pressure to make every dollar count in a tough economy, open source is an obvious strategy.

2.Modern,lightweight,standards-basedtechnology means developers can more readily and easily integrate BI with in-house technologies.

3.Maturesolutions.At this point, a wide range of open source solutions has been proven suitable for production by many customers. Jaspersoft Reporting, for example—commercially deployed hundreds of times—has now been in use for more than 10 years.

4.Minimalrisk. The open source model means organiza-tions can “try before they buy.” BI projects, particularly those involving data warehouses, have traditionally faced substantial risks associated with uncertain (but costly) technology choices, some of which can amount to several hundred thousand dollars. Open source lets BI projects rapidly prototype—and even stress-test solu-tions with high volumes and user count simulation—with minimal risk.

5.Strongusercommunities. Open source communities such as JasperForge greatly improve the stability and us-ability of open source software, by offering deep insight into user needs and providing low-cost, highly valuable pre-release feedback.

6.Flexibility.Open source technologies are typically newer, with open, forward-looking architectures, and are more flexible in terms of data source support and integration of new capabilities.

Trend 6Big Data

According to a Gartner study, the vol-ume of data generated in 2009 alone was greater than in the preceding 5000 years combined. Enterprise data in the next five years will grow an additional 650%.

An umbrella term coined in 2008, Big Data refers to tech-nologies that allow organizations to quickly analyze and derive insight from massive data sets, the scale of which was unimaginable just a few years ago. Less expensive “commod-ity” hardware is also an enabler of these solutions, which can analyze and aggregate petabytes of data at once (1 petabyte = 1000 terabytes).

HowbigisBig?

“Mankind created 150 exabytes (billion gigabytes) of data in 2005. This year, it will create 1,200 exabytes.”

—TheEconomist,2/27/2010

While most companies have nowhere near a petabyte of data today, any organization storing more than a few tera-bytes can benefit from Big Data technologies. With tradi-tional database, data warehousing, and business intelligence technologies—even when applying current best practices—“exceeding 10TB is difficult.” 7 The predicted data growth rates mean organizations with only two or three TB of data will soon exceed the limits of traditional technologies.

Big Data technologies can also help any organization seeking to dramatically reduce query times, or to complete complex analysis tasks within smaller time windows.

BigDataInnovations

New technologies to store and speed the processing of mas-sive amounts of data fall into three categories (some vendors offer hybrid solutions incorporating two or more of these approaches):1.Hardwareacceleration.These database appliances use

DRAM or Flash memory instead of hard disks, improving physical read/write performance, and may also modify database code to take better advantage of multi-core processing and other advances.

2.MassivelyParallelProcessing(MPP)Databases.These SQL-compliant databases are designed to spread the pro-cessing of data over many machines—typically so-called commodity servers. Some MPP databases also employ “shared nothing” architectures, which spread data storage over several machines as well, eliminating potential scal-ability bottlenecks.

3.Map-Reduce,Hadoop,andotherNoSQLapproaches.These approaches, collectively called “NoSQL,” or “Not Only SQL,” enable data access via programming languages without the use of SQL-based interfaces, and can spread data across many separate machines. Data structures are typically flat-file or other non-relational formats.

NoSQL approaches are especially important for analysis tasks that cannot be efficiently coded in SQL. These include analyses that involve several passes over the data —such as Monte Carlo simulations—and cases where the data is un- or semi-structured, such as text analysis.

Map-Reduce and Hadoop are the most widely known NoSQL approaches. Map-Reduce is an algorithm that spreads data analysis tasks over several nodes, and then “reduces” all of the nodes’ results into a single result set. Hadoop is an open source implementation of Map Reduce, plus additional functionality, available from Apache.

With these Big Data technology innovations, businesses can now perform analyses formerly considered infeasible, whether because there was too much data, the analyses took too long, or because the processing required was poorly suited to SQL.

PreparingforBigDataBI.

Regardless of which Big Data technologies are employed for an analytic task, business users still need to view and understand results. Given the potentially huge output sets, business users need ad-hoc reporting and analysis tools designed to “find needles in haystacks”—including powerful data visualizations, summary-to-fine-grain drill-down, and dynamic filtering and sorting.

These reporting, analysis and user interface capabilities must also be data format-agnostic, working seamlessly traditional RDBMSs, MPP databases, or files spread across the network.

7 TDWI “Next Generation Data Warehouse Platforms” by Philip Russo, 2009. Page 11.

Trend 7Real-Time Insight

One challenge to the BI industry is a longstanding one: the need to reduce the time required to perform and com-plete query and analysis tasks. But the explosion of data and compression of business cycles has made this require-ment more universal.

BI technologies have responded, incorporating man of the advances discussed here, with impressive results. Examples of analysis tasks shrinking in duration from days to min-utes—and hours to seconds—are becoming more and more common.

The business impact is significant—and will become more so as these capabilities proliferate. Organizations are demanding—and will realize—the ability to drive decision-making based on up-to-the-minute, fully-analyzed data.

Consider the impact on marketing organizations, which spend millions on campaigns to generate interest and demand. Where it might have taken weeks or months to optimize these campaigns—while marketers awaited data collection and analysis—they’re approaching the ability to evaluate, adjust and optimize these campaigns in a matter of hours. Advances like these are crucial in the age of search marketing campaigns that must evolve in real time—over-night just isn’t fast enough.

Similarly, operational decisions across the enterprise are becoming more urgent, further underscoring the need for real-time analytics. It’s no longer enough to understand what happened—and why—last month, or even last week.

Real-time insight can still be challenging in some situations, but the requirements will likely persist. BI solutions with forward-looking architectures are more likely to give enter-prises the ability to meet these needs.

ConclusionThe business intelligence sector has undergone major changes, impacting enterprise appetites for operational data and strategic insight, while enhancing the technology avail-able to deliver these insights.

Not every IT and software development organization can act on all of these trends. However, as BI strategies evolve and BI project decisions are made, understanding these trends can help enterprises reap maximum benefit from their BI investments.

Learn MoreThe Economist, 2/27/2010. Special Edition on “Data, Data, Everywhere”.

Using Collaboration to Extend the Reach of BI, by Colin White, BI Research. At International Summit on Data Warehousing and Business Intelligence, Rome, June 2010.

TDWI Technology Survey of May 2009

Gartner: Open Source in Information Management: State of Play. By Donald Feinberg, Business Intelligence Summit, April 12-14, 2010, Las Vegas.