oracle exadata production features -...

51
1 Patterns Oracle Exadata Production Features New York Oracle User Group (NYOUG) 13 th June 2017 Y V Ravi Kumar Oracle ACE Director Oracle Certified Master (OCM) Oracle ACE Spotlight Co-Author of Upgrade and Migration Methods

Upload: hoangxuyen

Post on 17-Mar-2018

299 views

Category:

Documents


5 download

TRANSCRIPT

Page 1: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

1Patterns

Oracle Exadata Production Features

New York Oracle User Group (NYOUG)13th June 2017

Y V Ravi KumarOracle ACE DirectorOracle Certified Master (OCM)Oracle ACE SpotlightCo-Author of Upgrade and Migration Methods

Page 2: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

2Patterns

AB

OU

T M

E

Oracle Certified Master (OCM) Oracle ACE Director Oracle ACE Spotlight for the month – June 2016 “Community Expert” in DELL’s Toad World “Expert” in Oracle Technology Network (OTN) community

Oracle Speaker @ Oracle Technology Network (OTN) New York Oracle User Group (NYOUG) Independent Oracle User Group (IOUG) Sangam (Largest Oracle Event in India) All India Oracle User Group (AIOUG)

Author of 80+ articles Oracle Technology Network (OTN) Toad World - Connected-Driven Innovation OTech Magazine All things ORACLE from Redgate UKOUG Library

CO-FOUNDER OF

Y V RAVI KUMAR

ORACLE CERTIFICATIONS

Oracle Certified Master (OCM)Oracle 10g & 11g: RAC Certified ExpertOracle 11g: Performance Tuning Certified ExpertOracle Exadata 11g EssentialsOracle Golden Gate 10 EssentialsOracle Database 11g: SQL Tuning Certified ExpertOracle 9i & 10g: Oracle On Linux Certified ExpertOCP – Oracle 12c, 11g, 10g, 9i and 8iSUN Certified – Solaris System Administrator in SUN Solaris 9

Page 3: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

3Patterns

INTRODUCING DATABASE MACHINE

Fully Integrated platform for Oracle Database

ORACLE REAL APPLICATION CLUSTERS

(RAC) and ORACLE PARTITIONING are highly recommended software options

Database Machine is highly complimentary with clustering and parallel operations

Database Machine is Simple and Fast to Implement

Balanced Hardware Configurations(DATABASE SERVER, STORAGE SERVER, INFINIBAND SWITCHES)

High Performance and High Availability for all Oracle

Database Workloads

Page 4: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

4Patterns

KEY FEATURES OF EXADATA

Smart ScanRuns portions of a database query in storage

Hybrid Columnar Compression (HCC)Reduces the number of blocks (and I/O) for compressed data

IO Resource Manager (IORM)Prioritizes I/O bandwidth based on database or workload within a database

Smart Flash CacheUses flash for physical I/O

Smart Flash LogUses flash to improve log I/O latency

Storage IndexEliminates Unnecessary I/Os

Exadata Storage Server SecurityImplements security level @ASM level and @Database level

DBFSIntermediate data store for Exadata Database Machine

Disk ScrubbingValidate the integrity of the mirrored ASM extents

KEY FEATURES OF THE

EXADATA STORAGE SERVER SOFTWARE

Page 5: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

5Patterns

KEY FEATURES OF EXADATA STORAGE SOFTWARE 12.2.1.1.0

Full storage offload functionality for Database 12.2Database 11.2, 12.1, and 10.2 can coexist along side 12.2 on the same system

Complete investment protection

All new software features work on all supported Exadata hardware generations

Updated Oracle Linux kernel and Oracle VM improve robustness and scalability

Oracle Linux 6.8 with UEK4, Oracle Virtual Machine 3.4.2

Over 30 unique software features and enhancementsBetter analytics, better transaction processing, better consolidation, more secure, faster and more robust upgrades, and easier to manage

Smart OLTP: End-to-End IO Latency Capping

Database 12.2 automatically redirects slow read I/O operations to another Exadata storage server

Smart OLTP: Extended Distance ClustersInfiniBand connected for high performance & Limited to 100m optical cables in 2016 (best for fire cells)

Page 6: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

6Patterns

KEY FEATURES OF EXADATA STORAGE SOFTWARE 12.2.1.1.0

Smart Availability: Efficient Rebalance and Restore

1. Intelligent and flexible rebalance power setting

Dynamically change ASM_POWER_LIMIT

2. ASM rebalance restores redundancy first

Drastically reduces secondary failure exposure window

Exposed via new REBUILD phase in v$asm_operation

3. Exadata leverages flash cache for rebalance reads

Improves performance of redundancy restoration by up to 30%

Smart Availability: New Redo Log Write Metrics

1. Redo Log Write response times are very critical to large scale OLTP systems

2. New metrics to measure

Overall IO latency

Networking and other overhead

IOs serviced by Flash Log

Overall latency per storage server

3. Improves visibility of redo log write performance

Page 7: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

7Patterns

ACFS SUPPORT ON EXADATA DATABASE MACHINE

Oracle ACFS Support on Exadata Database Machine

1. Starting with Oracle Grid Infrastructure 12c R1 12.1.0.2, Oracle ASM File System (ACFS) supports all database files and general purpose files on Oracle Exadata Database Machine.

2. DBFS or a NAS filer is also a consideration when using Exadata Database Machine

3. On Exadata Database Machine, Oracle ACFS supports the following database versions: Oracle 10g R2 (10.2.0.4 and 10.2.0.5) Oracle 11g (11.2.0.4 and higher) Oracle 12c (12.1.0.1 and higher)

4. Creating AFCS file systems on Exadata storage requires database servers with Oracle Linux.

5. Database files stored in ACFS on Exadata storage are subject to the following guidelines and restrictions:

Oracle ACFS replication or security/encryption/audit is only supported with general purpose files. These functions are currently notsupported with ‘database files’ in an Oracle ACFS file system.

Oracle ACFS does not currently support the Exadata offload features (e.g. Smart Scan, Storage Indexes, IORM, Network RM). Exadata Smart Flash Logging is not supported.

Page 8: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

NOT ALL EXADATA FEATURES CAN BE DISABLED

These features can be turned on and off

a. Smart Flash Cache / Flash Logb. Smart Scan (query offload to storage)c. Hybrid Columnar Compressiond. I/O Resource Management

These inherent features are always ona. Scale-out Exadata Storageb. InfiniBand 40Gb/sec Connectivityc. End-to-End Tuning

NOTE:

Because inherent features

cannot be disabled, the true

Exadata advantage exceeds

the measured results

Page 9: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

9Patterns

SITUATIONS PREVENTING SMART SCAN

Scan on an index-organized table.

Scan on a clustered table.

Fast full scan on a reverse key indexes.

Fast full scan on a compressed index.

LOB or LONG column is being selected or queried.

More than 255 columns are referenced in the query.

ORA_ROWSCN pseudocolumn is being fetched.

Command is CREATE INDEX using NOSORT.

If _serial_direct_read is turned off (NEVER).

If table has CACHE property.

High CPU usage on the Storage Cells.

Page 10: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

10Patterns

Enabling or Disabling Cell Offload SystemwideSQL> alter system set cell_offload_processing=false;SQL> alter system set cell_offload_processing=true;

Enabling or Disabling Cell Offload for a SessionSQL> alter session set cell_offload_processing=false;SQL> alter session set cell_offload_processing=true;

ENABLING OR DISABLING CELL FEATURES

Enabling or Disabling Cell Offload for a Specific SQL StatementSQL> select /*+ opt_param('cell_offload_processing','false') */ count(1) from sales;SQL> select /*+ opt_param('cell_offload_processing','true') */ count(1) from sales;

Enabling and Disabling Smart Scan for an ASM Disk GroupSQL> alter diskgroup data set attribute 'cell.smart_scan_capable'='FALSE';SQL> SELECT name, value, group_number FROM v$asm_attribute;

Page 11: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

11Patterns

Influencing Exadata’s decision -1: Forcing full table scans with hintsSQL> select /*+ full (sales) */ from sales;

Influencing Exadata’s decision -2: Making indexes invisibleSQL> alter index <index name> invisible;

Influencing Exadata’s decision -3: Ignoring Optimizer hintsSQL> alter session set “_optimizer_ignore_hints” = TRUE;

Influencing Exadata’s decision -4: Forcing direct readsSQL> alter session set “_serial_direct_read” = always;

INFLUENCING EXADATA SMART SCAN FEATURES

Page 12: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

12Patterns

ORACLE EXADATA

Smart Flash Cache

Page 13: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

13Patterns

EXADATA SMART FLASH CACHE

Query Offloading and EXADATA Smart Scan

Offloads the data search and retrieval processing to the storage cell

Runs portions of a database query in storage

High Performance Cache for frequently accessed objects

Excellent for repeated random reads

CELLSRV knows where is the block searched

Page 14: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

14Patterns

INTELLIGENT CACHING

Frequently accessed Data and Index Blocks are cached

Control File Reads and Writes are cached

File Headers reads and writes are cached

DBA can influence caching priorities

CELL_FLASH_CACHE = Keep, Default, None

1. Keep: Uses Smart Flash Cache more aggressively

2. Default: Uses Smart Flash Cache normally

3. None: Specifies that Smart Flash cache is not used

Keep blocks are automatically ‘unpinned’ if:

↗ Object is dropped, shrunk or truncated

↗ Object is not accessed on the cell within 24 hours

↗ Block is not accessed on the cell within 48 hours

Page 15: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

CellCLI>

VERIFY FLASH CACHE CONFIGURATION AND CONTENT ON THE CELL SERVER

list celldisk where diskType=FlashDisk detail

CellCLI> list flashcache detail

CellCLI> list flashcachecontent where objectnumber=76848 detail

cachedKeepSize: 0cachedSize: 7077888dbID: 195867650dbUniqueName: EXADBhitCount: 0missCount: 0objectNumber: 76848tableSpaceNumber: 8

NOTE

MOVE TO DATABASE SESSION TERMINAL AND SELECT THE FC_TBL TABLE

SQL> alter system flush buffer_cache;System altered.

Elapsed: 00:00:00.08

Page 16: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

16Patterns

ORACLE EXADATA

Disk Scrubbing

Page 17: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

17Patterns

DISK SCRUBBING @STORAGE SERVERS

Disk scrubbing is a new feature introduced in Oracle 11.2.0.4 and Exadata 11.2.3.3.0 storage s/w

version.

Usage of disk scrubbing is to periodically validate the integrity of the mirrored ASM extents and thus

eliminate latent corruption.

Disk Scrubbing is designed to schedule on production servers when average I/O utilization is minimal

because disk scrubbing can cause spikes in disk utilization and latency and adversely affect database

performance.

By default, the hard disk scrub runs every two weeks.

The following Parameters controlling the disk scrubbing:

1. hardDiskScrubInterval2. hardDiskScrubStartTime

Page 18: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

18Patterns

DISK SCRUBBING @STORAGE SERVERS

Schedules available to enable Harddisk scrub Activity

hardDiskScrubInterval=dailyhardDiskScrubInterval=weeklyhardDiskScrubInterval=biweekly

Ways to check alert log in Oracle Exadata Storage Server

1. ADRCI2. CELLTRACE

[celladmin@CellServer01 ~]$ cd $CELLTRACE[celladmin@CellServer01 trace]$ pwd/opt/oracle/cell12.1.2.3.3_LINUX.X64_161109/log/diag/asm/cell/CellServer01/trace

[celladmin@CellServer01 ~]$ adrciadrci> show alert

Page 19: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

19Patterns

DISK SCRUBBING @STORAGE SERVERS

Page 20: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

20Patterns

DISK SCRUBBING @STORAGE SERVERS

Command to verify Harddisk scrub Activity enabled on Oracle Exadata:[celladmin@CellServer01 ~]$ cellcli -e list cell attributes name,hardDiskScrubIntervalCellServer01 biweekly

Command to stop Harddisk scrub Activity enabled on Oracle Exadata:[celladmin@CellServer01 ~]$ cellcli –e alter cell hardDiskScrubInterval=none

When to schedule:

Decide on hardDiskScrubStartTime to start over weekend/non-peak hours and set appropriately. CellCLI> ALTER CELL hardDiskScrubStartTime='<Specify the Time>'

Change the interval to BIWEEKLY if the previous action plan was implemented to stop the disk scrub.CellCLI> ALTER CELL hardDiskScrubInterval=biweekly

Page 21: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

21Patterns

DIAGNOSTIC PACKAGE IN EXADATA

DIAGPACK diagnostic package, contains logs and traces that are useful for troubleshooting all type of disks problems Example: F320 Flash cards or nvme disks in a system.

In case of the failure of Accelerator F320 PCIe Card or nvme disk, Collect the Diagpack bundle from the system and upload to SR for further troubleshooting

a. Diagpack feature is in image 12.1.2.3.0 and later.b. The location of the diagnostic packages is $LOG_HOME.c. You can find diagpack location in alerthistory as below

Example: alerthistory-detail.out

You can also find the alert in ./cell/ms-odl.trc

You can also find the diagpack with below commandCellCLI> LIST DIAGPACK DETAIL

Page 22: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

22Patterns

ORACLE EXADATA

Hybrid Columnar Compression (EHCC)

Page 23: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

23Patterns

INTRODUCTION

EXADATA Storage Server includes EXADATA Hybrid Columnar Compression (HCC) feature Provides very high levels of Data Compression Allows the Database to reduce number of physical I/Os required to scan a table

DESIGNED FOR

Infrequently updated Data

Low concurrency environments

All index types (B-Tree, Bitmap)

Compression during Bulk Loads only

Taking advantage of Processing Power, Memory and Infiniband Network

Extending to Pillar Axiom and Sun ZFS Storage Appliance (ZFSSA)

Page 24: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

24Patterns

TYPES OF EHCC

Choice can be made from 2 types of Exadata Hybrid Columnar Compression (EHCC)

WAREHOUSE COMPRESSION

This option is for Optimizing Query Performance

Suitable for data warehouse applications

Two Options: Query High and Query Low

ONLINE ARCHIVAL COMPRESSION

This option is for Optimized for Maximum Compression Ratios

Suitable for data changes very rarely

Two options: Archive High and Archive Low

TABLE COMPRESSION SYNTAX

QUERY COMPRESSION SYNTAX CREATE TABLE emp (…) COMPRESS FOR QUERY [LOW | HIGH];

ARCHIVE COMPRESSION SYNTAX CREATE TABLE emp (…) COMPRESS FOR ARCHIVE [LOW | HIGH];

Page 25: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

25Patterns

EHCC LEVELS

Creating a HCC Table Compressed for Query HighSQL> create table sales_queryhigh (empname varchar2(30),empno number,empdescr varchar2(100),hire_date date) tablespace users compress for query high;

Creating a HCC Table with HCC Partitions

SQL> create table orders (cid, pid, sid, price, discount, odate) partition by range (cid) partition p1 values less than (100000) nocompress, partition p2 values less than (200000) compress for archive low, partition p3 values less than (300000) compress for query high, partition p4 values less than (maxvalue) compress for query low) enable row movement as select * from prev_orders;

Compression attributes for Tablespace

SQL> create bigfile tablespace bigtbsdatafile '+DATA' size 2g autoextend on next 1mextent management local autoallocatesegment space management autodefault compress for query low;

SQL> select tablespace_name, def_tab_compression, nvl(compress_for,'NONE') compress_forfrom dba_tablespaces;

Page 26: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

26Patterns

ORACLE EXADATA

Storage Index

Page 27: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

27Patterns

INTRODUCTION

STORAGE INDEX Used to reduce the amount of physical I/O performed on EXADATA cells Tracks the maximum & minimum column values so that this information

can be used to avoid unnecessary I/O at the cell Maintained automatically in the memory, occupies almost negligible

space Works for several data types such as NUMBER, DATE, VARCHAR

Automatically used when the query has a predicate on the storage index column with operators =, >, <, != or NULL or NOT NULL

Used only during the full table scans - for the better utilization and performance the data inserted into the tables should be in sorted form, so that the storage index can be created on that column

NLS DATA TYPES ARE NOT SUPPORTED!

There is no need to CREATE or DROP the StorageIndex.When the cell nodes are rebooted/recycled thestorage indexes are dropped and the first querythat is issued after the cell is reboot willautomatically trigger the storage index to berebuild

Page 28: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

28Patterns

ADVANTAGES OF STORAGE INDEX

STORAGE INDEX

↗ Is created automatically and transparently

↗ Used automatically if present during Smart Scans

↗ Do not take up any physical storage (Memory Only)

↗ Have negligible impact on DML statements

↗ Can make corresponding Database Index redundant

↗ As they’re used during Smart Scans, when used can return

just the rows/columns of interest to the database

Page 29: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

STATISTICS RELATED TO STORAGE INDEXES

Before disabling storage indexes, we will execute a test query and show the storage index I/O savings:

There is only one database statistic related to storage indexes. The statistic, 'Cell Physical IO Bytes Saved by Storage Index', keeps track of the accumulated I/O that has been avoided by the use of Storage Indexes.

Since the statistic is cumulative, it must be checked before and after a given SQL statement in order to determine whether Storage Indexes were used on that particular statement.

Run the following query to check the statistics for Storage Indexes :SQL> select name, value from v$sysstat where name like '%storage%';

Page 30: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

STATISTICS RELATED TO STORAGE INDEXES

Page 31: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

31Patterns

ORACLE EXADATA

I/O Resource Manager

Page 32: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

32Patterns

INTRODUCTION

I/O Resource Manager (IORM)

An integral software component of the Oracle Exadata Storage Server

Runs within the Cell Services (Cellsrv) on each Cell

Each Cell Disk maintains an I/O queue for each Consumer Group and

for each Database

Page 33: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

33Patterns

IORM PLANS

Intradatabase Resource PlanDBRM can be used to create an interdatabase plan; when created using DBRMautomatically sent to each storage servercell.

When bandwidth is dividing in differentworkload in same database, it's calledintradatabase resource management.

Interdatabase Resource PlanManaged with an InterdatabasePlan.

When bandwidth is dividing fordifferent databases, it's calledinterdatabase IORM. In thisscenario IORM need to beconfigured on cell level.

Category

Used together by Oracle® EXADATA to allocate I/O resources on EXADATA Storage Server Cell

Page 34: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

34Patterns

IMPLEMENTING IORM PLAN – FLASH CACHE AND FLASH LOG

CellCLI> alter iormplan objective=balanced

CellCLI> alter iormplandbplan=((name=Sales Database, level=1, allocation=70),

(name=Finance Database, level=1, allocation=30)),catplan=((name=INTERACTIVE, level=1, allocation=60),

(name=BATCH, level=1, allocation=40))

IORM can control whether a database can use Exadata Smart Flash Cache. IORM can control whether a database can use Exadata Smart Flash Log.

CellCLI> alter iormplandbPlan=((name=Sales Database, level=1, allocation=80, flashCache=on, flashLog=on),

(name=Finance Database, level=1, allocation=20, limit=50, flashCache=off, flashLog=on), (name=others, level=2, allocation=100, flashCache=off, flashLog=off))

Page 35: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

35Patterns

ORACLE EXADATA

Smart Flash Logging

Page 36: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

36Patterns

SMART FLASH LOGGING FEATURES

Smart Flash Logging uses the space which is shared among a set of flash devices Each Storage Server contains 4 PCI Flash Memory Cards Each Card contains 4 Flash devices for a Total of 12 Flash devices By default Exadata Smart Flash Logging

Uses a Total of 512MB per each Storage Server

Page 37: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

37Patterns

ORACLE EXADATA

Write Back Flash Cache

Page 38: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

38Patterns

WRITE-BACK FLASH CACHE

Write Back Flash Cache provides the ability to cache write I/Os directly to PCI

flash in addition to read I/Os.

Exadata storage software version 11.2.3.2.1 is the minimum version required to

use write back flash cache.

Using Exadata storage software version 11.2.3.2.1, Exadata Smart Flash Cache is

persistent across Exadata Storage server restarts.

Write Back Flash Cache is the best suitable option when:

• Application is ‘write-intensive’

• Significant waits for ‘free buffer waits’ are noticed

• High I/O times to check for write bottleneck in AWR reports

Page 39: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

39Patterns

In release 11.2.3.2.0, FlashCache is “WriteThrough” by default. ALTER CELL command is used tochange this mode to “WriteBack”

WriteThrough: Suitable for absorbing repeated random reads WriteBack: Suitable for write intensive workloads

Write Back Flash Cache significantly improves the write intensive operations because writing to flashcache is faster than writing to Hard disks

Cell Attribute flashCacheMode determines this mode.

Execute:#dcli -g ~/cell_group -l root cellcli -e "list cell attributes flashcachemode"

Results:flashCacheMode: WriteBack -> write back flash cache is enabledflashCacheMode: WriteThrough -> write back flash cache is not enabled

SIGNIFICANCE OF FLASH CACHE

Page 40: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

40Patterns

METHODS FOR ENABLING WRITE BACK FLASH CACHE

ROLLING METHOD

NON-ROLLING METHOD

Assuming that RDBMS & ASM instances are up while enabling Write Back Flash Cache in One

Cell Server at a time

Assuming that RDBMS & ASM instances are downwhile enabling Write Back Flash Cache

Check all griddisk asmdeactivationoutcome and asmmodestatus to ensure that all griddisks on allcells are Yes and ONLINE respectively

Check that all of the flash cache are in the normal state and that no flash disks are in a degraded orcritical state:

NOTE: Before performing steps for these methods, perform the following check as root from one of the compute nodes

# dcli -g cell_group -l root cellcli -e list griddisk attributes asmdeactivationoutcome, asmmodestatus

# dcli -g cell_group -l root cellcli -e list flashcache detail

Page 41: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

41Patterns

ORACLE EXADATA

Flash Cache Compression

Page 42: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

42Patterns

INTRODUCTION

FLASH CACHE COMPRESSION

Dynamically increases the logical capacity of the flash cache by

transparently compressing user data as it is loaded into the flash cache

Allows much more data to be kept in flash and decreases the need to

access data on disk drives

The compression and decompression operations are completely

transparent to the application and database, and have no performance

overhead, even when running at rates of millions of I/O per second

Depending on the user data compressibility, EXADATA Storage Server

Software dynamically expands the Flash Cache size up to two times

Page 43: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

43Patterns

INTRODUCTION (CONTINUED)

Compression benefits vary based on the redundancy in the data

Tables and Index that are uncompressed have the largest space reductions

Tables and Index that are OLTP compressed have significant space reductions

Tables that use Hybrid Columnar Compression have minimal space reductions

Oracle Advanced Compression Option is required to enable flash cache

compression

User data is not retained when enabling flash cache compression

Page 44: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

44Patterns

EXADATA STORAGE SOFTWARE 12.1.2.2.0

8 TB Hard Disk Support

8 TB high capacity disks require the following software:

Exadata 12.1.2.1.2 or higher

Running CellCLI Commands from Compute Nodes

The new ExaCLI utility enables you to run CellCLI commands on cell nodes remotely from compute nodes. This is useful in cases where you locked the cell nodes by disabling SSH access.

Disabling SSH on Storage Servers

By default, SSH is enabled on storage servers. If required, you can "lock" the storage servers to disable SSH access. You can still perform operations on the cell using ExaCLI, which runs on compute nodes and communicates using https and REST APIs to a web service running on the cell.

Page 45: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

45Patterns

EXADATA STORAGE SOFTWARE 12.1.2.2.0

Oracle Exadata Storage Statistics in AWR Reports

The Exadata Flash Cache Performance Statistics sections have been enhanced in the AWR report:

Added support for Columnar Flash Cache and Keep Cache. Added a section on Flash Cache Performance Summary to summarize Exadata storage cell statistics along with database

statistics.

Updating Nodes Using patchmgr

Starting with Exadata release 12.1.2.2.0, Oracle Exadata database nodes (releases later than 11.2.2.4.2), Oracle Exadata Virtual Server nodes (dom0), and Oracle Exadata Virtual Machines (domU) can be updated, rolled back, and backed up using patchmgr. You can still run dbnodeupdate.sh in standalone mode, but using patchmgr enables you to run a single command to update multiple nodes; you do not need to run dbnodeupdate.sh separately on each node. patchmgr can update the nodes in a rolling or non-rollingfashion.

Cell-to-Cell Rebalance Preserves Flash Cache Population

Minimum software required: 12.1.0.2 BP11When a hard disk hits a predictive failure or true failure, and data needs to be rebalanced out of it, some of the data that resides on this hard disk might have been cached on the flash disk, providing better latency and bandwidth accesses for this data

Page 46: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

46Patterns

EXADATA STORAGE SOFTWARE 12.1.2.3.0

Performance Improvement for Storage Server Software Updates

Updating Oracle Exadata Storage Server Software now takes significantly less time. By optimizing internal processing even further, the cell update process is now up to 2.5 times faster compared to previous releases.

Quorum Disk Manager UtilityIn earlier releases, when Oracle Exadata systems with fewer than 5 storage servers were deployed with HIGH redundancy, the voting disk for the cluster was created on a disk group with NORMAL redundancy. If two cells go down in such a system, the data is still preserved due to HIGH redundancy but the cluster software comes down because the voting disk is on a disk group with NORMAL redundancy.Quorum disks enable users to deploy and leverage disks on database servers to achieve highest redundancy in quarter rack or smaller configurations. Quorum disks are created on the database servers and added into the quorum failure group.

Cell-to-Cell Rebalance Preserves Storage IndexStorage index provides significant performance enhancement by pruning I/Os during a smart scan. When a disk hits a predictive failure or a true failure, data needs to be rebalanced out to disks on other cells. This feature enables storage index entries, created for regions of data in the disk that failed, to be moved along with the data during the cell-to-cell offloaded rebalance to maintain application performance.

Page 47: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

47Patterns

EXADATA STORAGE SOFTWARE 12.1.2.3.0

ASM Disk Size Checked When Reducing Grid Disk Size

Suppose the following command:CellCLI> list griddisk DATAC1_CD_00_adczarcel04 attributes name,asmdisksize

returns the following output:DATAC1_CD_00_adczarcel04 14880M

When you try to reduce the size of the grid disk to be smaller than the ASM disk:CellCLI> alter griddisk DATAC1_CD_00_adczarcel04 size=10G

the command returns an error:CELL-02894: Requested grid disk size is smaller than ASM disk size. Please resize ASM disk DATAC1_CD_00_ADCZARCEL04 first.

Page 48: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

48Patterns

Exadata Database Node Monitoring and Alerts

• DBMCLI monitors, and manages DB nodes

– Similar to CELLCLI for storage server

• Comprehensive monitoring and healing

– Hard disks, battery, InfiniBand ports, ILOM, CPU, memory, temperature

– CPU utilization, memory utilization, network interface throughput, file system utilization

– Auto collect console history and alert on kernel panic

– Threshold based alerts

DBMCLI on Database Servers

• New service dbserverd

– Replaces exadata_mon_hw_asr.pl

– MS Management Server

– RS Restart Server

– If OVM runs in dom0 only

• Command line dbmcli

• New users

– dbmadmin, dbmmonitor (welcome)

– dbmsvc (WLS owner, no login)

• Java is back - jdk-1.7.0_72

# service dbserverd status

rsStatus: running

msStatus: running

[dbmadmin@slcc12adm01 ~]$ dbmcli

DBMCLI: Release - Production on Tue Dec 16

01:23:48 MST 2014

Copyright (c) 2007, 2014, Oracle. All rights

reserved.

DBMCLI> list dbserver

slcc12adm01 online

DBMCLI> list alerthistory

Page 49: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

49Patterns

EXADATA CLOUD SERVICE: SCALABLE, AVAILABLE, SECURE, COMPATIBLE

Advanced Security, Label Security, Database Vault

Multitenant

Active Data Guard

Partitioning

AdvancedCompression

Real Application Testing

OLAP, Analytics, Spatial and Graph

Management Packs

Real Application Clusters

In Memory Database InfiniBand Fabric

Columnar Flash Cache

HCC

10:1

I/O I/O I/O

PCI FlashSmart Flash Cache,

Smart Flash Log

Hybrid ColumnarCompression

I/O Resource Management

Exafusion Direct-to-Wire Protocol

Offload SQL to Storage

Network Resource Management

In-Memory Fault Tolerance

All ExadataPlatform Innovations

All Oracle Database Innovations

Page 50: Oracle Exadata Production Features - NYOUGnyoug.org/.../07/Oracle-Exadata-Production-Features-NYOUG_13th_June...Oracle Exadata Production Features New York Oracle User Group (NYOUG)

50Patterns

X6-2 Elastic

Database

Server

X6-2 Elastic

Storage ServerX6-2 Eighth X6-2 Quarter X6-2 Half X6-2 Full X6-8 Half X6-8 Full

Database Servers 1 - 2 2 4 8 2 2

Total Database Cores 44 - 44 88 176 352 288 288

Total Memory (GB) 256 (max 768) 128512 DDR4

(max 1536)

512 DDR4

(max 1536)

1024 DDR4

(max 3072)

2048 DDR4

(max 6144)

4096 DDR3

(max 12288)

4096 DDR3

(max 12288)

InfiniBand Switches - - 2 2 3 3 2 3

Management Ethernet Switch - - 1 1 1 1 1 1

Storage Servers - 1 3 3 7 14 3 14

Total Storage Cores - 20 30 60 140 280 60 280

Raw PCI Flash Capacity EF - 25.6 TB 38.4 TB 76.8 TB 179.2 TB 358.4 TB 76.8 TB 358.4 TB

HC - 12.8 TB 19.2 TB 38.4 TB 89.6 TB 179.2 TB 38.4 TB 179.2 TB

Raw Hard Disk CapacityEF 4 X 600 GB

(expands to 8)

- - - - - - -

HC 96 TB 144.0 TB 288.0 TB 672.0 TB 1344.0 TB 288.0 TB 1344.0 TB

Usable capacity (Normal

Redundancy)

EF - 9.3 TB 13.9 TB 27.9 TB 65.0 TB 130.0TB 27.9 TB 130.0 TB

HC - 36.3 TB 54.5 TB 108.9 TB 254.2 TB 508.3 TB 108.9 TB 508.3 TB

Usable capacity (High

Redundancy)

EF - 7.3 TB 10.9 TB 21.9 TB 51.0 TB 102.0 TB 21.9 TB 102.0 TB

HC - 28.5 TB 42.7 TB 85.4 TB 199.4 TB 398.7 TB 85.4 TB 398.6 TB

EXADATA X6 QUICK REFERENCE CARD