Running E-Business Suite Database on Oracle Database Appliance

Session ID# 11792

Maris Elsins, Yury Velikanov

About Maris

Google: Elsins Oracle [phone|email]

LinkedIn, Twitter, Blog, Slideshare, Latvia

Oracle [Apps] DBA 8 years PL/SQL Developer 3 years

Oracle Certified Master 9i, 10g, 11g, 11i, R12 OCP

Speaker at Oracle Conferences since 2007, 9 of them international

Pythian since 2011

About Yury (co-author)

Google: Yury Oracle [phone|email]

Twitter, LinkedIn, Blog, Slideshare, YouTube

Oracle ACE Director (RAC SIG international chair)

Oracle Certified Master (9i/10g/11g + 15 OCP certificates)

Oracle DBA with 12+ years Apps experience

About Pythian •  Recognized Leader:

-  Global industry-leader in remote database administration services and consulting for Oracle, Oracle Applications, MySQL and Microsoft SQL Server

-  Work with over 250 multinational companies such as Forbes.com, Fox Sports, Nordion and Western Union to help manage their complex IT deployments

•  Expertise: -  Pythian’s data experts are the elite in their field. We have the highest

concentration of Oracle ACEs on staff – 10 including 3 ACE Directors – and 2 Microsoft MVPs.

-  Pythian holds 7 Specializations under Oracle Platinum Partner program, including Oracle Exadata, Oracle GoldenGate & Oracle RAC

•  Global Reach & Scalability: -  Around the clock global remote support for DBA and consulting, systems

administration, special projects or emergency response

Agenda

• Oracle Database Appliance as the DB platform for e-Business Suite

• How Pythian migrated e-Business Suite DB to ODA

Oracle Database Appliance?

- Engineered by Oracle for Oracle - Simple deployment / pre-configured - Reliable / pre-tested - Highly Available - Good performance - Easy management - One vendor - Pay as you grow licensing - 50K$ (60K$ for X3-2)

Inside the box…

• 2 DB servers, each configured with: ODA ODA X3-2

2* Xeon X5675 6-core CPUs 2* Xeon E5-2690 8-core CPUs 96Gb Memory 256Gb Memory

500Gb - mirrored storage for OS / SW 600Gb - mirrored storage for OS / SW 1x 10 GbE and 3x 1 GbE 4x 10 GbE

Inside the box…

• Shared storage ODA ODA X3-2

+DATA +RECO

20x 600Gb 3.5inch 15Krpm SAS 4TB ASM High redundancy

6TB ASM Normal Redundancy (v2.4)

20x 900Gb 2.5-inch 10Krpm SAS-2 6TB ASM High Redundancy

9TB ASM Normal Redundancy

+REDO 4x 73Gb SSD drives for +REDO 90Gb ASM High Redundancy

4x 200Gb SSD drives for +REDO 260Gb ASM High Redundancy

Non-expandable *Optional 2nd storage shelf doubles the capacity

Limitations • Limitations by design

*  No Scaling Up *  No Scaling Out *  possibility of using NFS Storage (why would you do that)

*  Storage Expansion is available for X3-2

• Supportability limitations *  Use oakcli (OracleApplianceKitClient) for any patching *  Additional software not allowed (except agents)

*  Virtualization support option (v2.5 / X3-2) can help!

*  One-off patches are not supported [ID 1399055.1]

ODA Application Kit Releases • Every ~3 months

*  Bundle patch *  Bare Metal OS ISO Images (Patch 12999313)

• Major new features (ODA v1) *  v2.1 – (Oct 2011) – Single 11.2.0.2 OH *  v2.2 – Upgrade to 11.2.0.3 *  v2.3 – Support for multiple OHs 11.2.0.[23] *  v2.4 – Normal redundancy for +DATA and +RECO *  v2.5 – (Feb 2013) Oracle VM /X3-2 starts here/

• Think about the upgrade strategies!

Certification and Support • How certification works?

*  Oracle certifies HW platforms and Operational Systems *  Oracle does not certify specific hardware configurations

for software products

*  ODA is a cluster of two x86-64 OEL 5 servers *  11gR2 is certified on Linux x86-64 with OEL5 *  eBS is certified with 11gR2

eBS DB is certified on ODA 11gR2 and R12.1 are in premier support

Monitoring and Management • How to operate ODA in your IT infrastructure?

*  [ID 1415773.1] 3rd party agents are supported! *  Agents “manage, monitor, backup, replicate,

authenticate, or otherwise act on the database, the server, or the environment”

*  NetBackup, Networker, etc.

• Monitoring is important! *  HW: Use Auto Service Request (ASR)

*  Monitors HW Faults *  Collects information and creates SRs automatically

*  SW: Use your favorite monitoring framework

How Pythian Migrated R12.1 e-Business Suite DB to ODA v2.2

Scope of the project

• The Source - R12.1.3, 10g RAC, old Hardware

• The Target - Migrate the DB to ODA - Upgrade the DB from 10.2.0.5 to 11.2.0.3 - Consolidate 4 DBs into one ODA - Install 900+ e-Business Suite patches

Important MOS notes

- Interoperability Notes EBS 12.0 and 12.1 with Database 11gR2 [ID 1058763.1]

- Using Oracle 11g Release 2 Real Application Clusters with Oracle E-Business Suite Release 12 [ID 823587.1]

eBS DB tier requirements

• Requirements colliding with ODA supportability *  Install products from the 11g Examples CD *  8 one-off patches for 11.2.0.3 *  Requires dedicated oracle listener *  Customized TNS_ADMIN and ORA_NLS10 variables *  No specific requirements for the grid home

Default ODA Oracle Home is not suitable

How we did it…

• /u01/cst for the customized software • Separate Oracle Inventory

*  /u01/cst/oraInventory *  Allows keeping the default inventory clean

• Separate Oracle Home *  /u01/cst/{DB_NAME}/product/11.2.0.3 *  Cloned it from the default OH *  Installed the Products from Examples CD *  Installed required one-off patches

Challenge: the DB upgrade • Why it was not simple?

*  Source DB is 10.2.0.5 on old/slow hardware *  compatible.rdbms = 11.2.0.2 for all ASM diskgroups on

ODA

• What options did we review? *  In-place upgrade + migration to ODA = SLOW *  Install 10gR2 on ODA and create standby = IMPOSSIBLE

because of Database Compatibility of ASM diskgroups *  Rebuild the ASM diskgroups with lower compatible.rdbms

setting = TOO INTRUSIVE

Solution: the DB upgrade

- FAQ about Restoring Or Duplicating Between Different Versions And Platforms [ID 369644.1]

*  Restore 10.2 backup using 11.2 software *  Apply archived logs periodically *  alter database open resetlogs upgrade *  Continue the upgrade process

The DB upgrade completed in 1.5 hours

Initialization parameters

• Mandatory settings for eBS in [ID 396009.1] • From a DB created by “oakcli create database” *  _disable_interface_checking=TRUE *  _ENABLE_NUMA_SUPPORT=FALSE *  _FILE_SIZE_INCREASE_INCREMENT=2143289344 *  _gc_policy_time=0 *  _gc_undo_affinity=FALSE *  _KGL_CLUSTER_LOCK_READ_MOSTLY=TRUE *  _kill_diagnostics_timeout=140 *  _lm_rcvr_hang_allow_time=140 *  db_block_checking='FULL'

*  db_block_checksum='FULL' *  db_lost_write_protect='TYPICAL' *  filesystemio_options='setall' *  parallel_adaptive_multi_user=FALSE *  parallel_execution_message_size=16384 *  parallel_min_servers=0 *  parallel_threads_per_cpu=2 *  use_large_pages='ONLY' *  compatible=11.2.0.3

Separate listener for eBS

• Don’t use port pool 0 for eBS *  The ODA default listener runs on 1521 *  Avoid port conflicts!

• Create the listener *  srvctl add listener -l LISTENER_EBSDB -o

$ORACLE_HOME –p 1522 *  srvctl setenv listener -l LISTENER_EBSDB -T

TNS_ADMIN=$ORACLE_HOME/network/admin

Clusterware DB resource

-  srvctl add database -d EBSDB -o /u01/cst/EBSDB/product/11.2.0.3 -p +DATA/EBSDB/PARAMETERFILE/spfile.704.792145311 -a "DATA,RECO,REDO" -n EBSDB

-  srvctl setenv database -d EBSDB -t "TNS_ADMIN=$ORACLE_HOME/network/admin,ORA_NLS10=$ORACLE_HOME/nls/data/9idata”

-  srvctl add instance -d EBSDB -i EBSDB1 -n oda01a-net1 -  srvctl add instance -d EBSDB -i EBSDB2 -n oda01b-net1

Net Services configuration

• Cluster services use default TNS_ADMIN *  $ORACLE_HOME/network/admin *  It has to be the same on both nodes

• AutoConfig TNS_ADMIN differs on each node *  $ORACLE_HOME/network/admin/$CONTEXT_NAME

• Set the IFILE settings: *  echo "IFILE=${ORACLE_HOME}/network/admin/$

{CONTEXT_NAME}/sqlnet.ora" > $ORACLE_HOME/sqlnet.ora *  Repeat the same steps for tnsnames.ora and listener.ora

Impact on ODA’s patching • Minimal impact

*  Default OH and inventory are not customized.

• eBS configuration points visible to oakcli *  /etc/oraInst.loc – switch between inventories manually *  /etc/oratab

*  Patch ODA’s components one by one [ID 1458353.1]: use --infra, --grid, --database options of “oakcli update”

*  Avoid running automated update of all DBs on ODA *  Clusterware resources for eBS DB

*  Should not cause issues *  Easy to remove and recreate if needed

*  Data on ASM diskgroups

Impact on eBS DB patching • No restrictions to apply one-off patches on the

eBS DB Oracle Home • ODA’s Patch Bundle is needed for - Patchsets and PSUs to eBS database

*  Can’t be installed separately, dependencies on grid OH *  Grid OH has to be patched using ODA Patch Bundles

- Security Updates to eBS database *  PSU patches include security fixes *  eBS database has a PSU installed (11.2.0.3.2) *  ID 854428.1 – If a PSU was installed, future security fixes have to

be applied using subsequent PSUs

Future Perspective

• Transformation from ODA to *ODAA *  v2.5 – Oracle VM on ODA *  *ODAA = Oracle Database and Application Appliance J *  Oracle eBS is certified with Oracle VM 3.1.1 [465915.1]

• ODA X3-2 *  It’s unlikely ODA v1 will ever support HW upgrades *  Bigger! - more HDD/RAM/CPU *  Better? – (careful with IO)

Conclusions

- ODA is great for small/medium sized DBs - No official support for e-Business Suite DBs yet - Low-impact customization needed to run eBS DB - 5 months in production – 0 crashes

Questions? Comments?

THANK YOU

#11792 #C13TCH #C13DEN elsins@pythian.com

@MarisElsins @yvelik @Pythian