· all businesses want better insight industry typical questions retail what stores should be...

44
<Insert Picture Here> Extreme Performance Data Warehousing

Upload: others

Post on 17-Mar-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

<Insert Picture Here>

Extreme Performance Data Warehousing

Page 2:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

The Rise of the Intelligent Economy

“From recession comes an opportunity to reset a number of industry structures…there is an opportunity to infuse

industries with technologies that position them to operate more effectively in the next 50 years.”

Lessons Learned in Building the Intelligent Economy, May 2010

Page 3:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

All Businesses Want Better Insight

Industry Typical Questions

Retail What stores should be closed or sold?

Which customers will respond to new promotion?

Telecommunication

s

What are the issues effecting churn by region?

What is the average revenue per user (ARPU)?

Healthcare What are most common patient service requests?

What is average level of clinical supplies on-hand?

Financial Services How will new online services impact deposits?

How does average loan compare to last year?

Utilities Who do we target for energy efficiency program?

What resources are needed to restore an outage?

Public Sector What is the trend on budget and expenditures?

What is most cost-effective way to manage waste?

Page 4:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Source: TDWI Next Generation Data Warehouse Platforms Report, 2009

Challenge: Much More Data to AnalyzeData Warehouse Size and Growth

Page 5:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Challenge: No Single Source of TruthExpensive Data Warehouse Architecture

ETL

OLAP Data Mining

OLAPData Mining

ETL

Data Marts

Data Marts

Page 6:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Source: TDWI Next Generation Data Warehouse Platforms Report, 2009

Challenge: User Requirements Not MetHigh Churn in Data Warehouse Platforms

Page 7:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Consolidate Onto a Single PlatformFaster Performance, Single Source of Truth

Oracle Database 11g

Oracle Exadata Database Machine

DataData

MartsMarts

Data MiningData Mining

Online Online

AnalyticsAnalytics ETLETL

Page 8:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Oracle Exadata Database MachineFor OLTP, Data Warehousing & Consolidated Workloads

• Improve query performance by 10x

– Better insight into customer requirements

– Expand revenue opportunities

• Consolidate OLTP and analytic workloads

– Lower admin and maintenance costs

– Reduce points of failure

• Integrate analytics and data mining

– Complex and predictive analytics

• Lower risk

– Streamline deployment

– One support contact

Page 9:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Oracle Exadata Database Machine FamilyOracle Exadata Database Machine X2-2

Oracle Database Server Pool

• 8 2-processor Database Servers

– 96 CPU Cores

– 768 GB Memory

– Oracle Linux or Solaris 11 Express

Exadata Storage Server Pool

• 14 Storage Servers

– 5 TB Smart Flash Cache

– 336 TB Disk Storage

Unified Server/Storage Network

• 40 Gb/sec Infiniband Links

Available in full, half, quarter racks

Page 10:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Oracle Exadata Database Machine FamilyOracle Exadata Database Machine X2-8

Oracle Database Server Pool

• 2 8-processor Database Servers

– 128 CPU Cores

– 2 TB Memory

– Oracle Linux or Solaris 11 Express

Exadata Storage Server Pool

• 14 Storage Servers

– 5 TB Smart Flash Cache

– 336 TB Disk Storage

Unified Server/Storage Network

• 40 Gb/sec Infiniband Links

Full and multi-rack configuration

Page 11:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Select sum(sales)where salesdate=‘22-Jan-2010’…

Sum

Return entire Sales table

Traditional Query Problem

What Were Yesterday’s

Sales?

• Data is pushed to database server for processing

• I/O rates are limited by speed and number of disk drives

• Network bandwidth is strained, limiting performance and concurrency

Discard most of

sales table

Page 12:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Select sum(sales)where salesdate=‘22-Jan-2010’…

Sum

Return Sales for Jan 22 2010

Exadata Smart ScanImprove Query Performance by 10x or More

What Were Yesterday’s

Sales?

• Data intensive processing runs in Exadata Storage Servers

• Rows and columns filtered as data streams from disks

• Complex operations also run in storage

• Parallelize query execution and removes bottlenecks

Page 13:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Built-in Analytics Secure, Scalable Platform for Advanced Analytics

• Complex and predictive analytics embedded into Oracle Database 11g

• Reduce cost of additional hardware, management resources

• Improve performance by eliminating data movement and duplication

Oracle Data Mining

Uncover and predict

Oracle OLAP

Analyze and summarize

Page 14:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Exadata Storage IndexTransparent I/O Elimination with No Overhead

• Maintain summary information about table data in memory

• Eliminate disk I/Os if MIN / MAX never match “where” clause

• Completely automatic and transparent

A B C D

1

3

5

5

8

3

Min B = 1

Max B =5

Index

Min B = 3

Max B =8

Select * from Table where B<2 -Only first set of rows can match

Page 15:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Exadata Hybrid Columnar CompressionReduce Disk Space Requirements

3x

10x 15x

1.4x

2.5 x

UncompressedData

Data Warehouse Appliances

OLTP Data

DW Data

Archive Data

Oracle

Page 16:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Benefits MultiplyConverting Terabytes to Gigabytes

10 TB of User Data

20 GB of User Data 5 GB of User Data

No IndexesWith Storage Indexes

100 GB of User Data

10 TB of User Data

10 TB of User Data 1 TB of User Data

With Partition PruningWith 10x Compression

With Smart Scan

Sub second “10 TB” Scan

Page 17:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Partition to Manage Data Growth Compress Data and Lower Storage Costs

• Distribute partitions across multiple compression tiers

• Free up storage space and execute queries faster

• No changes to existing applications

Active Data

3x OLTP Compression

Read Only Data

10x DW Compression

Archive Data

15-50x Archive

Compression

Page 18:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Turkcell Runs 10x Faster on Exadata Compresses Data Warehouse by 10x

• Replaced high-end SMP Server and 10 Storage Cabinets

• Reduced Data Warehouse from 250TB to 27TB

• Using OLTP & Hybrid Columnar Compression

• Ready for future growth where data doubles every year

• Experiencing 10x faster query performance

• Delivering over 50,000 reports per month

• Average report runs reduced from 27 to 2.5 mins

• Up to 400x performance gain on some reports

Page 19:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Softbank Runs 2x–8x Faster on Exadata36 Teradata Racks Replaced by 3 Exadata Racks

Teradata36 Racks

Exadata3 Racks

Page 20:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Oracle Exadata for Data Warehousing

Page 21:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Vinod Haval

Vice President and Manager, Database

Paroducts

Bank of America

"We need one solution, one architecture. From that perspective, Exadata provides the right platform for consolidating database operations.”

Page 22:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Oracle Exadata MomentumRapid adoption in all geographies and industries

Page 23:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Exadata Smart Flash CacheExtreme Performance for OLTP Applications

• Full rack has 5 TB of Smart Flash Cache

• Can process over 1 million IOs per second

• 50 GB/sec query throughput on uncompressed data

• 5x more I/Os than 1000 Disk Enterprise Storage Array

Infrequently Used Data

Frequently Used Data

Page 24:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Oracle Database 11gThe Best Database for Data Warehousing

• World record performance for fast access to information

• Manage growing volumes of information cost-effectively

• Reduce costs through server and data consolidation

Real Application Clusters

Advanced Compression

Partitioning

OLAP

Data Mining

Page 25:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

ETL with Oracle Database 11g

• Fast data loading using DBFS and External Tables

• Fast transforms in Oracle Database 11g via Parallel DML operations

• Best-in-class performance for large batch oriented data loads

Non-Oracle Source

Staging Raw Files

Oracle Source

Data Pump Unload SCP

FTP

BCP Unload

Parallel Loads

Page 26:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

The Concept of PartitioningMaintain Consistent Performance as Database Grows

SALES SALES

Jan Feb

SALES

Jan Feb

Europe

USA

Large Table

•Difficult to Manage

Partition

•Divide and Conquer

•Easier to Manage

•Improve Performance

Composite Partition

•Higher Performance

•Match to business needs

Page 27:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Partition for PerformancePartition Pruning

What was the total sales amount for May 20 and May 21 2010?

Select sum(sales_amount)

From SALES

Where sales_date between

to_date(‘05/20/2010’,’MM/DD/YYYY’)

And

to_date(‘05/22/2010’,’MM/DD/YYYY’);

5/20

5/21

5/22

5/19

Sales Table

• Performs operations only on relevant partitions

• Dramatically reduces amount of data retrieved from disk

• Improves query performance and optimizes resource utilization

Page 28:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

In-Memory Parallel ExecutionEfficient use of memory on clustered servers

• Compress more data into available memory on cluster

• Intelligent algorithm – Places table fragments in memory on different nodes

• Reduces disk IO and speeds query execution

© 2010 Oracle Corporation

In-Memory Parallel Query in Database Tier

Page 29:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Automated Degree of Parallelism

• Optimizer derives the best Degree of Parallelism

• Based on resource requirements of all concurrent operations

• Less DBA management, better resource utilization

Automatically determine

DOP

Enough parallel servers available

Execute immediately

Queue statements if not enough parallel servers available

When required number of servers are available, execute first statement

8

64 32 16

Page 30:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

• Pre-summarized information stored within Oracle Database 11g

• Separate database object, transparent to queries

• Supports sophisticated transparent query rewrite

• Fast incremental refresh of changed data

Summary ManagementImprove Response Time with Materialized Views

Date

Products Channel

SQL QuerySales by Date

Sales by Product

Sales by Region

Sales by Channel

Region

Materialized ViewsRelational Star

Schema

Query Rewrite

Page 31:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

• Exposes Oracle OLAP cubes as relational materialized views

• Provides SQL access to data stored in an OLAP cubes

• Any BI tool or SQL application can leverage OLAP cubes

Region Date

Products Channel

Cube Organized Materialized Views

SQL Query

Automatic Refresh

Query Rewrite

Summaries

Page 32:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

• Multidimensional analytic engine that analyzes summary data

• Offers improved query performance and fast, incremental updates

• Embedded in Oracle Database instance and storage

Oracle OLAPBuilt-in Access to Analytic Calculations

• How do sales in the Western region this quarter compare with sales a year ago?

• What will sales next quarter be?

• What factors can we alter to improve the sales forecast?

Page 33:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Oracle OLAP and OBIEECalculations Computed Faster in OLAP Engine

Page 34:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

• Collection of data mining algorithms that solve business problems

• Simplifies development of predictive BI applications

• Embedded in Oracle Database instance and storage

Oracle Data MiningFind Hidden Patterns, Make Predictions

Retail Financial Services

• Customer Segmentation

• Response Modeling

• Credit Scoring

• Possibility of default

Communications Utilities

• Customer churn

• Network intrusion

• Product bundling

• Predict power line failure

Healthcare Public Sector

• Patient outcome prediction

• Fraud detection

• Tax fraud

• Crime analysis

Page 35:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Oracle Data Mining and OBIEEPrediction and Probability Results Integrated in Reports

Page 36:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

• Enrich BI with map visualization of Oracle Spatial data

• Enable location analysis in reporting, alerts and notifications

• Use maps to guide data navigation, filtering and drill-down

• Increase ROI from geospatial and non-spatial data

Oracle Spatial and OBIEE

Page 37:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Data Models

Exadata

Business Intelligence

Oracle Exadata Intelligent WarehouseFor Industries

• Combine deep industry knowledge with data warehousing expertise

• Help jump-start design and implementation of data warehouses

• Available for Retail and Communications industries

Page 38:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

• Proactive advice to reduce deployment costs and risk

• Preventive assessments to identify & resolve issues

• Predictive management for high quality of service

Advanced Customer Servicesfor Oracle Data Warehousing and Exadata

Lifecycle Services Operations Management Expert Services

Installs & configuration Quality of Service monitoring Solution architectures

Upgrades Incident & problem management Service delivery

Patching Configuration management Data loading & migration

Performance assessment Onsite or remote management Backup & recovery

Solution Support Center

24x7 Expert Service Desk Problem priority & escalation Integrated with Premier support

Page 39:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Oracle Exadata for Data Warehouse

© 2010 Oracle Corporation

Page 40:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Yaz Iida

Chief Executive Officer

LinkShare

“Our continued investments in resources and technology, including the new Oracle Exadata database, is providing advertisers and publishers with the increased performance, usability, and innovation that will help drive strong revenue growth today and in the future.”

Page 41:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

“Oracle Database 11g, along with Oracle Real Application

Clusters, Advanced Compression and Partitioning, all lend

themselves to delivering highly available, high performance

data warehousing.”

Henry Lovoy Data ManagerHealthSouth Corporation

Page 42:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Extreme Performance Data Warehousing Integrated Technology Stack

• Single source of truth

• Easy to deploy and manage

• Extreme performance

• Meets all end user requirements

• Lower cost of ownership

Smart StorageSmart Storage

DatabaseDatabase

Data ModelsData Models

ELT ToolsELT Tools

BI ToolsBI Tools

BI ApplicationsBI Applications

Page 43:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?

Oracle #1 for Data Warehousing

Source: IDC, July 2009 – “Worldwide Data Warehouse Management Tools 2008 Vendor Shares”

Page 44:  · All Businesses Want Better Insight Industry Typical Questions Retail What stores should be closed or sold? Which customers will respond to new promotion?