· all businesses want better insight industry typical questions retail what stores should be...
TRANSCRIPT
<Insert Picture Here>
Extreme Performance Data Warehousing
The Rise of the Intelligent Economy
“From recession comes an opportunity to reset a number of industry structures…there is an opportunity to infuse
industries with technologies that position them to operate more effectively in the next 50 years.”
Lessons Learned in Building the Intelligent Economy, May 2010
All Businesses Want Better Insight
Industry Typical Questions
Retail What stores should be closed or sold?
Which customers will respond to new promotion?
Telecommunication
s
What are the issues effecting churn by region?
What is the average revenue per user (ARPU)?
Healthcare What are most common patient service requests?
What is average level of clinical supplies on-hand?
Financial Services How will new online services impact deposits?
How does average loan compare to last year?
Utilities Who do we target for energy efficiency program?
What resources are needed to restore an outage?
Public Sector What is the trend on budget and expenditures?
What is most cost-effective way to manage waste?
Source: TDWI Next Generation Data Warehouse Platforms Report, 2009
Challenge: Much More Data to AnalyzeData Warehouse Size and Growth
Challenge: No Single Source of TruthExpensive Data Warehouse Architecture
ETL
OLAP Data Mining
OLAPData Mining
ETL
Data Marts
Data Marts
Source: TDWI Next Generation Data Warehouse Platforms Report, 2009
Challenge: User Requirements Not MetHigh Churn in Data Warehouse Platforms
Consolidate Onto a Single PlatformFaster Performance, Single Source of Truth
Oracle Database 11g
Oracle Exadata Database Machine
DataData
MartsMarts
Data MiningData Mining
Online Online
AnalyticsAnalytics ETLETL
Oracle Exadata Database MachineFor OLTP, Data Warehousing & Consolidated Workloads
• Improve query performance by 10x
– Better insight into customer requirements
– Expand revenue opportunities
• Consolidate OLTP and analytic workloads
– Lower admin and maintenance costs
– Reduce points of failure
• Integrate analytics and data mining
– Complex and predictive analytics
• Lower risk
– Streamline deployment
– One support contact
Oracle Exadata Database Machine FamilyOracle Exadata Database Machine X2-2
Oracle Database Server Pool
• 8 2-processor Database Servers
– 96 CPU Cores
– 768 GB Memory
– Oracle Linux or Solaris 11 Express
Exadata Storage Server Pool
• 14 Storage Servers
– 5 TB Smart Flash Cache
– 336 TB Disk Storage
Unified Server/Storage Network
• 40 Gb/sec Infiniband Links
Available in full, half, quarter racks
Oracle Exadata Database Machine FamilyOracle Exadata Database Machine X2-8
Oracle Database Server Pool
• 2 8-processor Database Servers
– 128 CPU Cores
– 2 TB Memory
– Oracle Linux or Solaris 11 Express
Exadata Storage Server Pool
• 14 Storage Servers
– 5 TB Smart Flash Cache
– 336 TB Disk Storage
Unified Server/Storage Network
• 40 Gb/sec Infiniband Links
Full and multi-rack configuration
Select sum(sales)where salesdate=‘22-Jan-2010’…
Sum
Return entire Sales table
Traditional Query Problem
What Were Yesterday’s
Sales?
• Data is pushed to database server for processing
• I/O rates are limited by speed and number of disk drives
• Network bandwidth is strained, limiting performance and concurrency
Discard most of
sales table
Select sum(sales)where salesdate=‘22-Jan-2010’…
Sum
Return Sales for Jan 22 2010
Exadata Smart ScanImprove Query Performance by 10x or More
What Were Yesterday’s
Sales?
• Data intensive processing runs in Exadata Storage Servers
• Rows and columns filtered as data streams from disks
• Complex operations also run in storage
• Parallelize query execution and removes bottlenecks
Built-in Analytics Secure, Scalable Platform for Advanced Analytics
• Complex and predictive analytics embedded into Oracle Database 11g
• Reduce cost of additional hardware, management resources
• Improve performance by eliminating data movement and duplication
Oracle Data Mining
Uncover and predict
Oracle OLAP
Analyze and summarize
Exadata Storage IndexTransparent I/O Elimination with No Overhead
• Maintain summary information about table data in memory
• Eliminate disk I/Os if MIN / MAX never match “where” clause
• Completely automatic and transparent
A B C D
1
3
5
5
8
3
Min B = 1
Max B =5
Index
Min B = 3
Max B =8
Select * from Table where B<2 -Only first set of rows can match
Exadata Hybrid Columnar CompressionReduce Disk Space Requirements
3x
10x 15x
1.4x
2.5 x
UncompressedData
Data Warehouse Appliances
OLTP Data
DW Data
Archive Data
Oracle
Benefits MultiplyConverting Terabytes to Gigabytes
10 TB of User Data
20 GB of User Data 5 GB of User Data
No IndexesWith Storage Indexes
100 GB of User Data
10 TB of User Data
10 TB of User Data 1 TB of User Data
With Partition PruningWith 10x Compression
With Smart Scan
Sub second “10 TB” Scan
Partition to Manage Data Growth Compress Data and Lower Storage Costs
• Distribute partitions across multiple compression tiers
• Free up storage space and execute queries faster
• No changes to existing applications
Active Data
3x OLTP Compression
Read Only Data
10x DW Compression
Archive Data
15-50x Archive
Compression
Turkcell Runs 10x Faster on Exadata Compresses Data Warehouse by 10x
• Replaced high-end SMP Server and 10 Storage Cabinets
• Reduced Data Warehouse from 250TB to 27TB
• Using OLTP & Hybrid Columnar Compression
• Ready for future growth where data doubles every year
• Experiencing 10x faster query performance
• Delivering over 50,000 reports per month
• Average report runs reduced from 27 to 2.5 mins
• Up to 400x performance gain on some reports
Softbank Runs 2x–8x Faster on Exadata36 Teradata Racks Replaced by 3 Exadata Racks
Teradata36 Racks
Exadata3 Racks
Oracle Exadata for Data Warehousing
Vinod Haval
Vice President and Manager, Database
Paroducts
Bank of America
"We need one solution, one architecture. From that perspective, Exadata provides the right platform for consolidating database operations.”
Oracle Exadata MomentumRapid adoption in all geographies and industries
Exadata Smart Flash CacheExtreme Performance for OLTP Applications
• Full rack has 5 TB of Smart Flash Cache
• Can process over 1 million IOs per second
• 50 GB/sec query throughput on uncompressed data
• 5x more I/Os than 1000 Disk Enterprise Storage Array
Infrequently Used Data
Frequently Used Data
Oracle Database 11gThe Best Database for Data Warehousing
• World record performance for fast access to information
• Manage growing volumes of information cost-effectively
• Reduce costs through server and data consolidation
Real Application Clusters
Advanced Compression
Partitioning
OLAP
Data Mining
�
�
�
�
�
ETL with Oracle Database 11g
• Fast data loading using DBFS and External Tables
• Fast transforms in Oracle Database 11g via Parallel DML operations
• Best-in-class performance for large batch oriented data loads
Non-Oracle Source
Staging Raw Files
Oracle Source
Data Pump Unload SCP
FTP
BCP Unload
Parallel Loads
The Concept of PartitioningMaintain Consistent Performance as Database Grows
SALES SALES
Jan Feb
SALES
Jan Feb
Europe
USA
Large Table
•Difficult to Manage
Partition
•Divide and Conquer
•Easier to Manage
•Improve Performance
Composite Partition
•Higher Performance
•Match to business needs
Partition for PerformancePartition Pruning
What was the total sales amount for May 20 and May 21 2010?
Select sum(sales_amount)
From SALES
Where sales_date between
to_date(‘05/20/2010’,’MM/DD/YYYY’)
And
to_date(‘05/22/2010’,’MM/DD/YYYY’);
5/20
5/21
5/22
5/19
Sales Table
• Performs operations only on relevant partitions
• Dramatically reduces amount of data retrieved from disk
• Improves query performance and optimizes resource utilization
In-Memory Parallel ExecutionEfficient use of memory on clustered servers
• Compress more data into available memory on cluster
• Intelligent algorithm – Places table fragments in memory on different nodes
• Reduces disk IO and speeds query execution
© 2010 Oracle Corporation
In-Memory Parallel Query in Database Tier
Automated Degree of Parallelism
• Optimizer derives the best Degree of Parallelism
• Based on resource requirements of all concurrent operations
• Less DBA management, better resource utilization
Automatically determine
DOP
Enough parallel servers available
Execute immediately
Queue statements if not enough parallel servers available
When required number of servers are available, execute first statement
8
64 32 16
• Pre-summarized information stored within Oracle Database 11g
• Separate database object, transparent to queries
• Supports sophisticated transparent query rewrite
• Fast incremental refresh of changed data
Summary ManagementImprove Response Time with Materialized Views
Date
Products Channel
SQL QuerySales by Date
Sales by Product
Sales by Region
Sales by Channel
Region
Materialized ViewsRelational Star
Schema
Query Rewrite
• Exposes Oracle OLAP cubes as relational materialized views
• Provides SQL access to data stored in an OLAP cubes
• Any BI tool or SQL application can leverage OLAP cubes
Region Date
Products Channel
Cube Organized Materialized Views
SQL Query
Automatic Refresh
Query Rewrite
Summaries
• Multidimensional analytic engine that analyzes summary data
• Offers improved query performance and fast, incremental updates
• Embedded in Oracle Database instance and storage
Oracle OLAPBuilt-in Access to Analytic Calculations
• How do sales in the Western region this quarter compare with sales a year ago?
• What will sales next quarter be?
• What factors can we alter to improve the sales forecast?
Oracle OLAP and OBIEECalculations Computed Faster in OLAP Engine
• Collection of data mining algorithms that solve business problems
• Simplifies development of predictive BI applications
• Embedded in Oracle Database instance and storage
Oracle Data MiningFind Hidden Patterns, Make Predictions
Retail Financial Services
• Customer Segmentation
• Response Modeling
• Credit Scoring
• Possibility of default
Communications Utilities
• Customer churn
• Network intrusion
• Product bundling
• Predict power line failure
Healthcare Public Sector
• Patient outcome prediction
• Fraud detection
• Tax fraud
• Crime analysis
Oracle Data Mining and OBIEEPrediction and Probability Results Integrated in Reports
• Enrich BI with map visualization of Oracle Spatial data
• Enable location analysis in reporting, alerts and notifications
• Use maps to guide data navigation, filtering and drill-down
• Increase ROI from geospatial and non-spatial data
Oracle Spatial and OBIEE
Data Models
Exadata
Business Intelligence
Oracle Exadata Intelligent WarehouseFor Industries
• Combine deep industry knowledge with data warehousing expertise
• Help jump-start design and implementation of data warehouses
• Available for Retail and Communications industries
• Proactive advice to reduce deployment costs and risk
• Preventive assessments to identify & resolve issues
• Predictive management for high quality of service
Advanced Customer Servicesfor Oracle Data Warehousing and Exadata
Lifecycle Services Operations Management Expert Services
Installs & configuration Quality of Service monitoring Solution architectures
Upgrades Incident & problem management Service delivery
Patching Configuration management Data loading & migration
Performance assessment Onsite or remote management Backup & recovery
Solution Support Center
24x7 Expert Service Desk Problem priority & escalation Integrated with Premier support
Oracle Exadata for Data Warehouse
© 2010 Oracle Corporation
Yaz Iida
Chief Executive Officer
LinkShare
“Our continued investments in resources and technology, including the new Oracle Exadata database, is providing advertisers and publishers with the increased performance, usability, and innovation that will help drive strong revenue growth today and in the future.”
“Oracle Database 11g, along with Oracle Real Application
Clusters, Advanced Compression and Partitioning, all lend
themselves to delivering highly available, high performance
data warehousing.”
Henry Lovoy Data ManagerHealthSouth Corporation
Extreme Performance Data Warehousing Integrated Technology Stack
• Single source of truth
• Easy to deploy and manage
• Extreme performance
• Meets all end user requirements
• Lower cost of ownership
Smart StorageSmart Storage
DatabaseDatabase
Data ModelsData Models
ELT ToolsELT Tools
BI ToolsBI Tools
BI ApplicationsBI Applications
Oracle #1 for Data Warehousing
Source: IDC, July 2009 – “Worldwide Data Warehouse Management Tools 2008 Vendor Shares”