techupgradebestpractices-1560353.pdf

7/28/2019 techupgradebestpractices-1560353.pdf

1/57

1 Copyright 2012, Oracle and/or its affiliates. All rightsreserved.

Insert Information Protection Policy Classification from Slide 8


2/57



E-Business Suite Release 12.1 Upgrade Best Practices -

Technical Insight

Udayan Parvate, Director, EBS Release Engineering

Uday Moogala, Senior Principle Engineer, Applications Performance


3/57



The following is intended to outline our general productdirection. It is intended for information purposes only,

and may not be incorporated into any contract. It isnot a commitment to deliver any material, code, orfunctionality, and should not be relied upon in making

purchasing decisions.The development, release, and timing of any features orfunctionality described for Oracles products remains at

the sole discretion of Oracle.


4/57



Agenda

R12.1 Upgrade Overview

R12.1 Supported Upgrade Paths R12.1 Upgrade Resources

R12.1 Upgrade Best Practices to Minimize Downtime

Reference (Customer Upgrade Snapshots and more)

Q&A


5/57



R12.1 Upgrade Overview


6/57



R12.1 Rapid Install (RI)R12.1 Maintenance Pack (MP)

R12.1 was generally available (GA) in May 2009 Via Oracle Software Delivery Cloud (formerly known as ElectronicProduct Delivery (EPD)) and Oracle Stores

Can be used by new and upgrading customers (11.5.9

and above) to go directly to R12.1 If you are on R11i, use R12.1 RI from the software delivery cloud.Follow instructions from the Upgrade Guide: 11i to 12.1 and 12.1.1Release notes

If you are on R12.0.X, use R12.1 MP (7303030) from My OracleSupport (MOS). Follow instructions from R12.1 Maintenance PackInstall Instructions (752619.1)


7/57



EBS R12.1 Release Update Pack (RUP) 3

(12.1.3) Release

EBS 12.1.3 was released in J ul 2010 and deliversbugfixes and targeted functionality enhancements

Available from My Oracle Support (MOS) as a patch

Can ONLY be applied after upgrade to R12.1Currently, EBS 12.1.3 is the latest RUP available for R12.1

EBS 12.1.3 Installation instructions : 1080973.1


8/57



R12.1 Technology Stack

For details Oracle E-Business Suite Release 12.1 Maintenance Pack Installation Instructions (752619.1)

TECHNOLOGY

COMPONENT

VERSION

INCLUDED

11i10CU2

VERSION

INCLUDED

12.0.4 RI

VERSION

INCLUDED

12.1 RI

VERSION CERTIFIED

WITH

MINIMUM REQUIRED

VERSION

Apps Mid tier-Forms/Reports

6.0.8.25 10.1.2.2 10.1.2.3 - 10.1.2.3

Apps Mid tier-

J ava OracleHome/

Apps Mid tier-J DK

1.0.2.2/1.4.2 10.1.3.0/1.5 10.1.3.4/1.6.0 10.1.3.5 10.1.3.4/1.6

Database 9.2.0.6 10.2.0.3 11.1.0.7

10gr2: 10.2.0.5

11gr2: 11.2.0.1, 11.2.0.2,11.2.0.3

10gr2 : 10.2.0.5

11gr1 : 11.1.0.7

11gr2 : 11.2.0.2


9/57



R12.1 Key Facts11i10CU2 R12.0.4 RI R12.1 RI 12.1.3 RUP

#of Product

Schemas209 195

(25 removed,11 added)

201

(25 removed,17 added)

No changes

#file calls in DB

portion of the U

driver

104242 144940 156622 23408

PROD db size

File system size

31 GB

26 GB

45 GB

28 GB

50 GB

28 GB

NA

#files shipped in

RI268359 357778 385755 NA

#of Changed +

New fi les in DB

portion of the U

driver

NA ~95488 ( Vs 11.5.10.2 ) ~104057 ( Vs 11.5.10.2 )

~31843 ( Vs 12.0.4 )

~23474 ( Vs 12.1.1)


10/57



R12.1 Supported Upgrade Paths


11/57



R12.1 Upgrade PathsMinimum EBS suite level for direct upgrade to R12.1

11i9, 11i9CU1, 11i9CU2 or above

11i10, 11i10CU1, 11i10CU2 or above R12.0 and above

Minimum EBS suite level required for database versions

10.2.0.4 requires 11i9CU2

10.2.0.5/11.1.0.7/11.2.0.1/11.2.0.2/11.2.0.3 require 11i10CU2/R12.0.4

Based on R12.1 DB preparation guide (761570.1), certifiedupgrade path options can be categorized into :

A.Upgrade database and EBS level in a single downtime

B.Upgrade database and EBS level in separate downtimes

C.Apply upgrade interoperability DB patches and then upgrade EBS


12/57



R12.1 Upgrade Paths Continued..11.0/11i.X => 12.1

< = 11.5.8

11.5.9/cu1

11.5.10/cu1

12.110.2.0.4

12.1

10.2.0.5

12.111.2.0.1/11.2.0.2

11.2.0.3

12.1

11.1.0.7

11.011i10cu2

A

B ,CB ,C

B ,C

B ,C

B ,C

A. Upgrade database and EBS level in a single downt ime

B. Upgrade database and EBS level in separate downtimes

C. Apply upgrade interoperability DB patches and then upgrade EBS

SOURCE

TARGET

11i9cu2


13/57



R12.1 Upgrade Paths Continued..12.0.X => 12.112.1

10.2.0.4

> = 12.0.4

12.110.2.0.5

12.111.2.0.1/11.2.0.2

11.2.0.3

12.1

11.1.0.7

< = 12.0.3 A

B ,C

B ,C

B ,C

B ,C

B ,C

A

A. Upgrade database and EBS level in a single downt ime

B. Upgrade database and EBS level in separate downtimes

C. Apply upgrade interoperability DB patches and then upgrade EBS

SOURCE

TARGET


14/57



R12.1 Upgrade Resources


15/57



R12.1.3 EBS Data Model Comparison Report

Per product database object comparison between two releases forthe following object types (1290886.1)

Regular tables, Partitioned tables, Index organized tables, Globaltemporary tables, Queued tables

Views, Materialized views, Materialized view logs

Indexes, Sequences, Advanced queues, Packages, Triggers

Available for R12.1.3 Vs (11i9,11i10cu2,12.0.4,12.0.6,12.1.1,12.1.2)

Benefits

Customers can focus on what has changed

Easier to analyze impact on customizations, planned test coverageDifferences viewable for all products in the same report via simple UI


16/57




17/57



R12.1.3 EBS ATG Seed Data Comparison Report

Per product EBS ATG Seed data type comparison between two releases(1327399.1) BIP-Defns-Temps, RSG-Metadata, Components, Concurrent Programs, Contents

Descriptive Flexes, Diagnostics, Functions, Integrators, Key Flexes, Security Rules

Layouts, Lookups, Mappings, Menus, Messages, Sequences, Value Sets

Parameter Lists, Printer Styles, Profiles, Request Groups, Request Sets, Responsibilities

Available for R12.1.3 Vs (11i10cu2,12.0.4,12.0.6, 2.1.1) Benefits

Meant for Advanced user with prior knowledge about EBS Seed data delivery

Easier for developers/consultants/testing team to analyze impact on customizations,planned/desired test coverage

Post Go-live, to answer end-user questions


18/57




19/57



R12.1 Upgrade Best Practices


20/57



Upgrade Best Practices

Identification of required patches

Prepare a complete list of pre and post patches andrecommended code levels

Keep the system current on AD/ATG/OAM code e.g. latest AD/ATGRUPs on 11i/R12.0 and once on R12.1

High priority patches from MOS. Consolidated Upgrade Patches (CUP) and required one-offs thatneed to be applied in pre-install mode. e.g. EBS R12.1 CUP1(7303029:12.1.0), FIN upgrade patches from 1127593.1

Review Known-issues sections from key documents to determineadditional pre or post upgrade patches: Release notes, MP InstallInstructions


21/57



Upgrade Best Practices Continued..

Patch merging, sequencing and adpatch options

Use non-interactive patching Merge patches (228779.1). Merge NLS patches per language.

Perform uptime maintenance when possible

hot patching of iHelp

Install on test system and point production it. Switch back onceonline help is upgraded online. Saves lot of time

NLS patches, upload patch history and HRGLOBAL (1330470.1)

Use adpatch options such as nomaintainmrc, phtofile, nolink,

nogenform, nogenrep, nocompile jsp, noautoconfig, novalidate(1078973.1)


22/57




Downtime reduction procedures Run autoconfig in parallel on a multi-node system (387859.1)


23/57


24/57


Insert Information Protection Policy Classification from Slide 8The above reflects Oracles current development plans which are subject to change at any time

adpatch

workers 1-10

adpatchworkers 1-10

adpatchworkers 1-10

adpatchworkers 1-10

Admin/CM

Server

WebServer

WebServer

FormsServer

DatabaseServer

adpatch

workers 1-10

adpatchworkers 11-20

adpatchworkers 21-30

adpatch

workers 31-40

Upgrade Best Practices Continued..Shared APPL_TOP and Distributed AD

Shared

APPL_TOP


25/57


Insert Information Protection Policy Classification from Slide 8The above reflects Oracles current development plans which are subject to change at any time

Upgrade Best Practices Continued..Staged APPL_TOP

Production

Stage

1. Clone1. Clone 4. Sync Patch History

2. Upgrade to R12

2. Upgrade DB

3. Sync APPL_TOP

3. Update DB


26/57




Downtime reduction procedures Use Staged APPL_TOP (734025.1) for regular maintenance

and upgrade Saves time to patch the file system (C/G portion) by using a

patched up copy of production instance file system

Use in 11i => R12.1 upgrade to avoid applying NLS C/G portion

Can use for R12.0.X => R12.1 upgrade and once on R12.1


27/57




Downtime reduction procedures continued.. Use TUMS (The Upgrade Manual Script)

To avoid running tasks not relevant to your system

Import statistics Export statistics gathered during test runs, and import during final

run

Automate as much as possible

Time every activity and deal with bottlenecks


28/57




Downtime reduction procedures continued.. For the duration of the upgrade, consider

Disabling custom triggers and business events

Disabling auditing if enabled

If possible run in noarchivelog mode

Disabling flashback DB

Removing TDE from volume tables


29/57



Pre-upgrade activities Identify and execute tasks that could be completed in a

separate downtime period, prior to the production upgrade Use applicable steps mentioned in the "Downtime reduction" and

Upgrade By Request appendices E and G of the R12.1 upgrade guide

Upgrade RDBMS version to latest certified for the current APPS level (11.2.0.2 / 11.1.0.7 / 10.2.0.5 )

Convert to Oracle Applications Tablespace Model (OATM)

Compacts data, optimizes storage settings and I/O

Other planned HW or OS upgrades



30/57



Pre-upgrade activities continued...

Convert to Multiple Organization architecture (Document210193.1)

Drop MRC schema ( 11.5.10 and above )

Assign post upgrade jobs to specialized CM queue (byrequest_type, see Document 399362.1)



31/57



Pre-upgrade activities continued... Flush all the interfaces, such as Autoinvoice, J ournal entry

import, order import etc..

Review and disable all debug, logging at site,responsibility, user level

Gather stats with GATHER_AUTO option for all schemasclose to the start of downtime



32/57



Pre-upgrade activities continued... Pre-create new large indexes created by upgrade

constrained to indexes on existing columns

Minimize historical data to be upgraded as per businessrequirements Upgrade By Request

Post-upgrade hot-patch of additional historical data outlined in604893.1

Purge old and/or transient data before upgrading

Over 260 standard purge programs in R12

Use new Purge Portal in OAM to administrate



33/57



Purging Use OAM to configure, initiate and monitor purge programs

Set the execution frequency and view program history Programs tagged with the Purge program type

System Administrator >Oracle Applications Manager >Purging/Critical Activities



34/57



Key recommended fixes

Affecting Upgrade Performance AD-Parallel improved scalability: 8917381 (Part of 9179588 AD

CUP)

Forms/reports generation regression with 11g: 8557019

Optimizer:Dynamic Sampling on indexes 12942119 (11.2.0.2) General

Review 244040.1 for latest EBS recommended performance fixes



35/57



Database tier configuration

Maximize SGA and PGA sizing An upgrade only involves 10's of concurrent sessions;

starting rules of thumb ...

log buffer = 30 to 100 Mb

shared pool = 1 to 4 Gb pga target = 3 to 20 Gb

buffer cache = multi Gb, be generous without causingexcessive paging or swapping

Adjust with help from AWR pool advisories



36/57



Database tier configuration

Other upgrade specific init.ora changes If specified, remove db_file_multiblock_read_count

Maximize multiblock I/O sizes

Set job_queue_processes = #of CPUS

adobjcmp.sql (Phases : plb+90 and last+63)

Set parallel_max_servers = 2 X CPUs

Helps with large index creation, stats gathering and some large upg+phase jobs

Need to test with production-like DB server and I/O subsystem

Shutdown other RAC instances



37/57



Batch size and #workers

Batch size 10K is suitable for most installs, you can test other values from 1K up to100K if time allows

# Workers

Starting rule-of-thumb is between 1 and 1.5 x #COREs

It is critical to do multiple rounds of testing, adjusting above settings tomaximize server utilization, but constrained by factors such as

Memory utilization (no swapping/ excessive paging)

CPU utilization (scale down if at 100%)

I/O response times (scale down if averages > 20 ms)



38/57



Performance testing, monitoring and additional

optimizations... Analyze long runners via timing report, ad_task_timing analysis Mine ad_task_timing to identify low worker utilization due to phasing

waits and review responsible culprits

Review targeted AWR Instance and SQL reports awrrpt.sql and awrsqrpt.sql

Use My Oracle Support to check for known issues and workaroundsfor the longest running jobs



39/57




optimizations continued... Outside-the-box optimizations Requires Support Approval* Long running index creation and Statistics gathering

As these tasks run with little or no concurrent tasks,consider using alter system commands to increaseparallel_max_servers and pga_aggregate_target

(Need to test so as to not exhaust machine resources)

Large indexes: Pre-create, but do not alter columndefinitions

For stats (adsstats.sql ; phase: last+63) : Consider statsimport or on RAC using all nodes when gathering stat



40/57




optimizations continued... Outside-the-box optimizations Requires Support Approval* Long running mv related xdfs (en phases)

Cleanup and truncate large MV logs (requires MV complete refresh)

Look more closely at long running jobs that you suspect may not be required for

your system Thousands of jobs and customer product/data/setup configurations

Some of the heavy lifting work may not be required for a specific site

Check with Oracle Support to validate

A fix maybe available or possible to optimize for your case

New 12.1.3 AD timing report by module



41/57



Appendix

Customer Upgrade Snapshots


42/57



Toyota Motor Europe Release: 11.5.9+ on IBM AIX to R12.1.3 on Oracle Linux 5 Appl tier DB size: 800 GB

#Workers: 32

#CPUs on DB server: 8 cores

Downtime reduction measures

Online NLS patch application

#hrs for the 12.1.1 D driver: 21 hrs

#hrs for the 12.1.3 US upgrade: 4 hrs

#hrs for the 4 languages NLS patching 11 hrs

Customer Upgrade Snapshots Continued...11i to 12.1.3


43/57



Customer Upgrade Snapshots11i to 12.1

CPS (Chicago Public Schools) Release: 11.5.10.2 to 12.1

DB size: 900GB

#Workers and batch size: 32, 10000

#CPUs on DB server: 2 node RAC, 8 CPUs per node

Downtime reduction measuresDistributed AD

Upgrade RDBMS to 10.2.0.4 in a separate downtime

#hrs for the D driver: 22 hrs

Customer snapshothttp://www.oracle.com/customers/snapshots/chicago-public-schools-ebs-snapshot.pdf


44/57




Cisco Release: 11i to R12.0.6

DB size: 600GB


#CPUs on DB server: 16 Downtime reduction measures

Distributed AD

#hrs for the D driver: 5.5 hrs


45/57




Dell Release: 11i10 to R12.1.3

DB size: 15TB , 16 node RAC Cluster


#CPUs on DB server: 8 Downtime reduction measures

Distributed AD

Pre-create large indexes

#hrs for the D driver: ~30 hrs


46/57



Customer Upgrade Snapshots Continued...11i to 12.1

GE

Large EBS HRMS Implementation (~300K employees,10 Lang) Release:11.5.10.2 to R12.1.3

DB size: 838 G

Hardware: App Tier- 2 SUN T5240s(64x64),

DB Tier - SUN M8000 (12 Dual Cores) #Workers and batch size per App Server: 48, 10000

Downtime reduction measures

Distributed AD, Staged APPL_TOP

#hrs for D driver:~10 hrs US, ~13 hrs NLS (11.5.10.2 -> R12.1.1)

#hrs for DB Portion:~2.5 hrs US, ~1 hr NLS (R12.1.1 -> R12.1.3)


47/57



Customer Upgrade Snapshots Continued...12.0 to 12.1

Zebra Technologies Corporation Release: 12.0.6 to 12.1 DB Size: 106GB


#CPUs on DB server: 8

Downtime reduction measures Staged APPL_TOP

#hrs for the U driver: 12 hrs

Customer snapshothttp://www.oracle.com/customers/snapshots/zebra-technologies-corporation-ebs-snapshot.pdf


48/57



Oracle GSI Release: 12.0.3+ to R12.1

DB size: 17TB


#CPUs on DB server: 88 processors

Downtime reduction measuresStaged APPL_TOP for US and ten languages

Ran data fixes for problems found in test upgrades prior to productionupgrade to minimize stoppages

Distributed AD (4 servers,15 workers each)

Pre-create large indexes (6 hrs savings)


Customer Upgrade Snapshots Continued...12.0 to 12.1


49/57



Oracle GSI Release: 12.1+ to R12.1.3

DB size: 17TB


#CPUs on DB server: 150 processors


Ran data fixes for problems found in test upgrades prior to productionupgrade to minimize stoppages

Distributed AD (4 servers, 50 workers each)

Pre-create large indexes


Customer Upgrade Snapshots Continued...12.1 to 12.1.3


50/57



AT&T Release: 12.0+ to R12.1.2

DB size: 10 TB


#CPUs on DB server: 32 Processors


Distributed AD


Customer Upgrade Snapshots Continued...12.0 to 12.1.2


51/57



References


52/57



References R12.1 documentation roadmap (790942.1)

Oracle E-Business Suite Release 12.1 Info center (806593.1)

Database preparation guidelines for R12.1 upgrade (761570.1)

Patching FAQs (459156.1, 225165.1)

Using staged or shared APPL_TOP and distributed AD (734025.1, 384248.1,236469.1)

OAM Patch Wizard overview and FAQ (976188.1, 976688.1)

AD Command Line Options for Release R12 (1078973.1)

EBS 12.1.3 Data Model Comparison Report (1290886.1)

EBS ATG Seed Data Comparison Report (1327399.1)

Recommended Performance Fixes (244040.1)

R12 Upgrade Sizing & Best Practices (399362.1)


53/57



Additional Resources


54/57



Additional Resources

White papers

Planning Your Oracle E-Business Suite Upgrade from Release 11i toRelease 12.1 (987516.1)

R12 Upgrade considerations by product: Financials (889733.1)

Oracle E-Business Suite Upgrades and Platform Migration(1377213.1)

Have upgrade questions ?

Post on OTN R12 upgrade forum

http://forums.oracle.com/forums/forum.jspa?forumID=395&start=0

New!


55/57



Q&A


56/57




57/57



techupgradebestpractices-1560353.pdf

Documents