Upgrading to Oracle E-Business Suite 12.1: Technical

and Functional Panel

Session ID: S307961

Panel Members

Name Role Or Organization

Nadia Bendjedou Host: E-Business Suite Strategy

Sandra Vucinic VLAD Group Inc

John Stouffer Consultant Moderator

Tony Kane AT&T

Amrita Mehrok E-Business Suite Strategy

Nigel Cooper Oracle Application IT - GSI

Deep Ram Oracle Consulting

Dennis Horton Oracle Consulting

Oracle E-Business Suite Release 12.1 in Actionhttp://launch.oracle.com/?EBUS

4

http://forums.oracle.com/forums/forum.jspa?forumID=395

Release 12 Install & Upgrade Forum

Further Information

• E-Business Suite R12 Info Center: 401740.1

• EBS R12 Best Practices WhitePaper: 580299.1

• EBS Proactive Intelligence Center – Accounts Payable:

578232.1

• Oracle EBS Suite Upgrade Center: 461705.1

• EBS Maintenance Wizard Overview: 215527.1

• E-Business Suite Diagnostics Installation Guide: 421245.1

• EBS R12 Financials Critical Patches: 557869.1

AT&T Transforms Business with Oracle E-Business

Suite R12

Session ID: S307961

Presenter’s Name: Anthony Kane

Manager – AT&T

<Insert Picture Here>

Agenda

• Introduction

• Project Scope – Deployment Options

• CFAS R12 Upgrade & INTL Consolidation

<Insert Picture Here>

Introduction

• AT&T is one of the world’s largest Telecommunications companies

• AT&T Global Headquarters is based in Dallas, TX, US

• AT&T has approximately 290,000 employees

• AT&T has more than 75 million wireless customers

• AT&T has more than 14 million high speed internet subscribers

• Exclusive U.S. provider of iPhone 3G / 3GS

AT&T’s Journey with Oracle eBiz

Oracle 10.7 Financials

Oracle 11i.5.X Oracle 11i.5.10 Oracle 11i.5.10.2

DB 10g.2.0.2

Classic ‘T’ instance

Consolidation

Bellsouth Migration

Oracle R12.0.4(Rup5)

INTL-CFAS Consolidation

1997 2003 2005 2006 2007 2008 2009 20091994

Oracle 10.4.2 Financials

AT&T CFAS Environment Pre-R12 upgrade / Upgrade Objective

AT&T’s Consolidated Financial Application Suite (CFAS) running on Oracle eBusiness Financials along with Project Accounting.

• Oracle Financials (AR, AP, GL, PA) and Fixed Assets. - 11.5.10.2

• OIE, CM, AX, IR, OTL Modules

• HP-UX 11.11 / Sun Solaris 9

• Oracle 10gR2.0.2

• 6,000+ users

• 4.5 TB Domestic Instance and 3 International Instances Supporting 95 countries

Upgrade CFAS 11i E-Biz instance to R12 and migrate International regionally located Oracle Financial instances (EMEA & CLA, APAC, JAPAN, Canada) into the CFAS platform to create a Single Global Instance.

AT&T CFAS Systems Metrics11i.5.10.2 R12.0.4 INTL Global CFAS Instance

Metric Nov 2008

(Pre-R12)

Mar 2009

(Post-R12)

July 2009

(Post-INTL)

Database Space Usage 4.85 TB 6.23 TB 7.37 TB

User Connections

Total connections fm DB Tier 217909 280057 234820

Total connections fm Web Tier

- APPS

348460 448373 394190

Total DB Logins 1974015 2097754 2505755

Concurrent Manager

Total jobs submitted 194902 192984 344979

Total time used (Hours) 9870 Hours 16717 Hours 16780 Hours

Long running CC jobs 1282 2905 2638

AT&T CFAS Instance

Current Data Statistics

7.45 terabytes > 74 million customers

80+ operating units3.5 million Projects (30K new

additions every month)

177 Ledgers 12 million Assets

NLS languages(60+)36 million AP Invoices (~1.2 million AP

Invoices per month)

9 years of SLA data 1.21 million GL Code Combinations

CFAS Architecture

Gig

E

CFAS Production Environment

Hardware Architecture (May 2009)

CFAS R12 DB Server

ØOracle DB 10gR3 (10.2.0.3)

ØOracle Apps DB 10g (10.1.2)

ØOracle Financials 12i (12.0.4 RUP5)

ØProduction off-host DB Backup /

Recovery via Veritas NetBackup

Advanced Client and Oracle RMAN

with EMC TimeFinder/Mirror

Directory

PFIN11

125 GB

SGA 2 GB

BCV4

Group

off-host DB

Test clone

BCV3

Group

off-host DB

Test clone

BCV2

Group off-

host backup

BCV1

Group off-

host backup

CFAS

CFASPRD1

6,400 GB

SGA 7 GB

BCV3 group

cfaststb_dg1 is

visible to host

chtcfas1 as

instance cfaststb

BCV1 group

cfasprd1_dg1 and

BCV2 group

cfasprd1_dg2

alternately visible

to host chpfas1d

for off-host tape

backups

BCV4 group

cfaststq_dg1 is

visible to host

chtcfas1 as

instance cfaststq

AT&T PROPRIETARY INFORMATIONThe Information contained herein is for use only by authorized employees of AT&T Services, Inc.,

and is not for general distribution within or outside the respective companies.

fc

Fib

er C

ha

nn

el

CISCOSYSTEMS

Tape SAN

CISCO 9509

CXCDT100

CXCDT103

CXCDT104

CXCDT105

2Gbit fiber,

redundant

fabrics (A &B)

CISCOSYSTEMS

2gb fc

CISCOSYSTEMS

Disk SAN

CISCO 9509

192-port

CXCDD13

CXCDD23

2Gbit fiber,

redundant

fabrics (A &B)

CISCOSYSTEMS

2gb fc

2gb fc

Production Partitioned

Server

GigE

SUN V880 csprod97

Cspvms4.sldc.sbc.com

NetBackup Master Server

and

IBM LTO SiloCILTO 005 Lib for

NBU CSPVMS3

CILTO 011 LIB for

NBU CSPVMS4

22 IBM LTO 3584-L52

Drives

LAN

10

0b

ase

T

Eth

ern

et

EMC DMX3000

HK190102684

360 300GB Drives

240 146GB Drives

77.8 TB Usable Space

Production Partitioned Server

Ø2 HP SX1000 (Superdome)

Ønpar0 - DB server, 40-1.1Ghz CPU,

164GB Memory

Ønpar1 - Appl. Server and off-host DB

backup, 12-1.1Ghz CPU, 120GB

Memory, configured as 5 partitions

ØVpar5 used for off-host database

ØOS - HPUX 11.11(64-bit PA-RISC)

Frame: chf051

npar 0: chpcfas1

npar 1: chp051a

vpar1: chp051a1

vpar2: chp051a2

vpar3: chp051a3

vpar4: chp051a4

vpar5: chp051a5

Foundry Load Balancing

Production

Apps Servers

CHPCFAS2-CHPCFAS6

Production Application WebHP N4000 8-550 MHz CPU

24GB Memory HPUX 11.11(64-bit)

100baseT Ethernet

<Insert Picture Here>

Project Scope and

Deployment Options

Project Scope

Functional Footprint

• AP: Accounts Payable

• AR: Accounts Receivable (out of scope)

• FA: Fixed Assets

• GL: General Ledger

• PA: Project Accounting

• PO: Purchasing (out of scope)

• OIE: Employee Expense

• CM: Cash Management

• AX: Global accounting engine (replaced by R12 Sub Ledger Accounting)

• IR: Integrated Receiving (out of scope)

• HR: HR data for use in other modules

• OTL: Time and Labor

Technical Footprint• CFAS instance upgrade to R12

• CFAS database character set conversion to Unicode

• CFAS database will be upgraded to 10.2.0.3

• International instances will remain on 11.5.10.2/10.2.0.4 and be supported until AR and Brazil is migrated into CFAS

Impact Considerations

• AR out of scope – Need to maintain and support 11.5.10.2 AR in all regional instances with shadow GL/AP/CM

• Brazil /Portuguese out of scope - Need to maintain and support 11.5.10.2

• International migration into CFAS will be mapped to NEW chart of account structure using FAH and SLA implementation

• Instance in EMEA & CLA, APAC and Japan will migrated into the CFAS

• Oracle plans to support EBS 11i10 till Nov-2010. Extended Support will be offered for the period of December 1, 2010 - November 30, 2013

• CFAS database will be upgraded to 10.2.0.4 during August 2009

Option 1 (High Risk, Less Time)Upgrade CFAS to R12, implement new R12 functionality and then migrate

International data

Perform

International

setups into R12

CFAS

Perform R12

Upgrade

CFAS Instance (11510)

Create R12 iSetup

snapshots

Perform

International data

conversion into

R12 CFAS

Validate

instance

Validate

instance

Validate

instance

END

Setup performed

manually. Potentially

could use automation:

1. FNDLOAD

2. iSetup

3. Dataloader

These could be

referenced as ‘gold’

setups and used during

each testing cycle and

eventually into

Production

• Pros

• Ability to create GOLD iSetup snapshots for

majority of the setups (speed up International

setup during Production migration)

• Ability to test the GOLD snapshots during

multiple testing iterations

• Cons

• Require individual product teams to perform

R12 setups manually

• Difficult in tracing International data issues to

R12 upgrade or to the conversion process

• R12 is fairly new release and has potential

stability concerns – could run into unidentified

bugs

• Need to quickly learn R12 functionality

• Single consolidated instances will need to meet

the service level agreements (SLA)

requirements of the business

Option 2 (Less Risk, More Time)Convert International 11510 into CFAS 11510 and then upgrade CFAS to R12,

implement new R12 functionality • Pros

• Distinct stages in the process allows the ability to

rollback for issue resolution

• Greater in-house knowledge and experience for

many of the stage

• Provides a methodology that adds reliability, rollback

and traceability

• Take advantage of existing code reuse for the

conversion activities

• Cons

• Two milestones to focus on – 1) International

conversion into 11510 2) R12 Upgrade -- Increase in

testing effort

• Will be migrating International setup data that may be

obsolete after adopting R12 features (e.g.

responsibilities)

• This approach may require a significant post upgrade

setup task (e.g. combining LEs, Setup Ledger Sets)

• CFAS CEMLIs need to be made compliant to work

with 11510 and R12

• Single consolidated instances will need to meet the

service level agreements (SLA) requirements of the

business

Perform

International

setups into

R11510 CFAS

Perform R12

Upgrade

CFAS Instance (11510)

Create R12

iSetup

snapshots for

the additional

R12 setups

Perform

International

data conversion

into R11510

CFAS

Validate

instance

Validate

instance

Validate

instance

END

Setup performed

manually. Potentially

could use automation:

1. FNDLOAD

2. iSetup (may need to

‘maasage’ the extract to

reflect new chart of

accounts)

3. Dataloader

Perform

additional R12

related setups

Option 3 (Medium Risk, Less Time)Two Instance - CFAS R12 and International R12; Consolidate GL from

International into CFAS

• Pros

• Downtime impact for maintenance is reduced

• Complexity of setups in the CFAS instance is reduced

• Rapid deployment

• Reduction in training the CFAS folks on International processes

• Less impact on CFAS

• Lack of dependency between CFAS and International instances

• Cons

• Increase in maintenance to support two instances

• Potentially zero savings on headcount

• No single sign-on for the sub-ledger users

EMEA (11510)

Perform

additional R12

related setups

Perform

International data

conversion into

11510 EMEA

Validate

instance

Validate

instance

END

Perform

International

setups into

R11510 EMEA APAC (11510)

Japan (11510)

Upgrade EMEA

instance to R12

Validate

instance

Perform R12

Upgrade

CFAS Instance (11510)

Perform

additional R12

US related

setups

Validate

instance

END

Perform R12

International GL

related setups

(e.g. COA)

GL Balances

Imported into CFAS

Project Issues and Risks• Issues

Ø Challenge of communication amongst the different sub-ledger implementation teams

Ø Limited International functional knowledge amongst the CFAS implementation team

Ø Need expertise in the deployment and configuration of a single global instance

Ø CFAS CEMLIs are not multi-org compliant

Ø Need to convert Japanese data which is in Shift-JIS to Unicode

• Risks

Ø February 2009 is very aggressive timelines for CFAS R12 upgrade

Ø May 2009 is very aggressive timelines for a combined (R12 upgrade and instance

consolidation) project

Ø Oracle R12 is a very new release and AT&T may run into unidentified bugs

Ø Unknown level of effort for major tasks (e.g. International conversion, R12 Upgrade)

Ø Need to develop conceptual architecture illustrating and discussing the single global

instance, performance, high availability, downtime SLA

<Insert Picture Here>

CFAS R12 Upgrade & INTL

Consolidation

CFAS R12 Upgrade – Activities

• Phase 1 – Pre-Upgrade tasks

• Phase 2 – Upgrade tasks

a) Apply Application R12 Upgrade Main Patches

b) Perform Application Upgrade to 12.0.4

• Phase 3 - Post-Upgrade tasks

a) Apply Application R12 RUP 5 Patch

b) Apply Application R12 Post Upgrade Patches

Feb 08 Mar 08 May 08 Oct 08 Dec 08 12 Feb 09 –16 Feb 09

17 Feb 09 May 09

Project Kickoff Unit Testing System Testing UAT Performance Testing R12 Upgrade

Go-live INTL Consolidatio

n

CFAS R12 Upgrade – Activities

• Phase 4 – Post-Main Upgrade

a) Post Application R12 Upgrade Configuration

b) Configure Application Server Processes

c) Perform Application Post Upgrade Custom Tasks

• Phase 5 – Super User Validations

a) Perform Application Verification Tasks

b) Post Upgrade Setup/Validation Tasks - Functional Teams

c) Perform Final Application Verification Tasks - Note: ALL Post Upgrade Tasks Must Be Completed

• Phase 6 – Go-live

a) Turnover Environment To Clients

b) Execute Final Tasks - Can Be Done When The Application is Live

Pre-Upgrade Activities– Technical

• Upgrade Technical documentation and Technical Brownbag sessions

• Preparation of the inventory of software components for R12 upgrade

• Note 399362.1 – R12 Upgrade, Sizing and Best Practices

• Evaluate CPU, Memory, I/O subsystem requirements -OS level Patching

• Hardware upgrade considerations

Pre-Upgrade Activities– Technical

• DB upgrade, 10.2.0.2 10.2.0.3

• DB character set conversion, WE8ISO8859P1 Unicode Character set

• Oracle Applications Table management (OATM) considerations

• Tools

Ø TUMS – identifies steps that can be skipped

• Review of Concurrent Requests and Concurrent Program incompatibilities.

Pre-Upgrade Activities – Functional

• Upgrade functional documentation and Functional Brownbag training sessions

• Understanding of R12 new modules, FAH, SLA, TCA and Payments

• Inventory of customizations (CEMLIs). Consulting for CEMLI conversion

• Plan for required Financials Set ups

• Migration to BI Publisher (where applicable)

R12 Upgrade – Actual Down-time

Upgrade Activities

• Shutdown 11i at 07:30 on 2/12/09

• Start of R12 upgrade activities at 07:58 on 2/12/09

• Database back up completed at 09:47 on 2/12/09

• Main upgrade driver patch from 19:00 on 02/12/09 to 23:16 on 2/15/09

• Post-upgrade patches started at 00:10 on 2/16/09

• R12 available for super user testing at 10:30 on 2/16/09 & go/no go decision

• Abbreviated Daily batch cycle (Key jobs) on 02/16/09

• R12 available for all users on 2/17/09

• Regular Batch processing (All jobs) on 02/17/09

R12 Upgrade – Post go-live

• Performance Tuning

• Tech Stack Tuning

• Create Accounting Process

• Payment Batch Cycle

• Successful 1st Month end close with in two weeks after go-live

• Monitor Concurrent Mangers Activities

INTL Instances Consolidation

• Consolidated following regionally located INTL instances in to CFAS during May 2009

• UK/EMEA/CLA/Canada -- Located in England

• APAC -- Located in Hong Kong

• Japan -- Located in Tokyo

• First consolidated month end close on CFAS completed successfully May 2009