Oracle Enterprise Manager

Cloud Control 12c

Trials and Tribulations

Simon PaneFirst4 Database Partners

Simon.Pane@first4db.comOctober 17, 2013

Bio

Partner at First4 Database PartnersWorking with the Oracle Database since

version 6Oracle Certified Database Administrator

Oracle 8, 8i, 9i, 10g, 11gMOS Communities & OTN

Username: Simon_DBA

Objectives

Recommendations from practical experience

Point out items and issues beyond the documentation and MOS notes

The Name Game

Grid ControlCloud ControlEnterprise ManagerOEMEM 12cCC 12cEMGC 12c

Why to Use?

Status / Inventories / Patching / Global Commands / Jobs / Incident Lifecycle

DB tuning and monitoring (requires licensing)

Architecture A Quick Review

Mix or dont mix production & non-production

Illustration Source: http://docs.oracle.com/cd/E24628_01/doc.121/e25353/overview.htm#BABGGJDA

New Technology with 12c

Grid Control 10g used Oracle Application ServerWith a OC4J container

Cloud Control 12c based on Oracle Web LogicNew WLS instance with a GCDomain

Be Prepared to Use Support

With version 12.1.0.2 implementations:One PSU13 OMS patches5 agent patches plus configuration adjustments18 SRs with Oracle Support so far and counting

Upgrade Options

Dont try to manually create a parallel environmentOld and new agents share a portWont transfer existing data/users/jobs/UDMs

Use either the 1-system (in place) or 2-system (out of place) upgrade methods

The Two System Approach

MOS 1363162.1 for 2-system upgradeInstall a patch into the old/existing Grid ControlAutomatically brings across users, security, and

jobsFrom the old console, push out new software,

upgrade configuration and metrics, and eventually transition agents

Transition agents either individually or via groups, not all at once

Moving the OMR

Using the 2-system approach

Cant easily move SYSMAN via exportRMAN duplication or TTS instead

RMAN CatalogSYSMAN 10g

Ops CenterSYSMAN 12c

ORAREP ORATMP

Required Patches

MOS: Information Center: Enterprise Manager Base Platform Release Cloud Control 12c (ID 1379818.2)

MOS: Oracle Enterprise Manager 12c Configuration Best Practices (ID 1553342.1)

New Concept: Patch Plans

Instead of using OPatch, push out patches to agents or databases through Patch Plans in the consoleTakes care of blackouts and restartsCan deploy to multiple targets at once

Prerequisite:MOS: Enterprise Manager Patches required for setting up Provisioning, Patching and Cloning (Deployment Procedures) (ID 427577.1)

New Concept: The Software Library

Patches and updates are downloaded to a centralized Software Library on the OMSThen pushed out via Patch Plans

WARNING: Only configure the Software Library as SYSMANOtherwise duplicate update jobs are createdCan disabled these jobs but never removeOpened bug 17568293 for this problem

Batch Updates

May want to use if not using Patch PlansExamples:

Changing Oracle Home after manual home cloning patching

Updating the DBSNMP passwordEasy to do via

emcli modify_target -name="ORCL" -on_agent -type="oracle_database"

-properties="OracleHome:/u01/app/oracle/product/11.2.0/dbhome_2"

Post Upgrade and Stabilization

Drop database link: PREUGP_EMREPO_LINKRun the post migration steps

Dont forget sign-off: removes the old agent software (pre-NG agents )

MOS: EM 12c: DBPROV and DB SECURITY Jobs Stuck in Enterprise Manager 12.1.0.2 Cloud Control Post Upgrade Tasks Console under Deferred Data Migration tab (ID 1539853.1)

Configuration Suggestions

Modify Console Timeout: Defaults to 45 minutesUpdate (and then restart OMS):

emctl set property name oracle.sysman.eml.maxInactiveTime value 90

Schedule Purging and Rotation of CC LogsMOS:

12c Cloud Control: Which WLS Log Files Can be Removed / Purged Manually at Regular Intervals in 12c OMS Installation for Space Considerations? (Doc ID 1445743.1)

12c Cloud Control: How to Enable Log Rotation Policy to Automatically Delete Older GCDomain.log, EMGC_ADMINSERVER.log and access.log Files? (Doc ID 1450535.1)

EM 12c Cloud Control: Steps to Locate and Manage the Various Logs/Trace files in a 12c OMS Installation (Doc ID 1448308.1)

Startup / Shutdown

Slow startup:Easily up to 10 minutesConfirmed with Oracle that this is normal

Shutdown:Usually need to use: emctl stop oms -all forceOften rebooting the server is best

New Agent Deployment

Six Agent Installation Methods:1. Agent Deployment Wizard

Pushed from the OMS through the Console

The typical and recommended approach

Must still apply patches after deployment

2. Silent install via a response file3. RPM: build, copy, and install and RPM4. Agent cloning wizard (through the Console)5. Manual cloning6. Shared Oracle Home agents

Agent Issues

Agents sometimes getting targets stuck in the Status Pending state

Agents sometimes falsely report the HAS as being down

Typically need to execute:emctl stop agentemctl start agentemctl upload agentemctl clearstate agent

New Concept: Target Promotions

No more agentca d commandTarget discoveries happen automatically

daily or can be manually initiatedTargets must be manually promoted after

discovery Auto Discovery Results can help detect

inconsistencies in the environmentExtra databases discovered may indicate issue

with configuration (i.e. left over in listener.ora file)

New Concept: Incident Rules

Replaces Notification Rules Incident Rules determine what actions should

be taken when problems and events occurPerforming a notification can be one such

actionMOS:

12c Cloud Control: Steps for Subscribing / Unsubscribing to Email or Page Notifications in the Incident Management RuleSet (Doc ID 1389460.1)

EM 12c How to Configure Notifications for Job Executions ? (Doc ID 1386816.1)

New Concept: Compliance Framework

Security metrics have been moved to the new Compliance Framework

By default the Security Recommendations for Oracle Products standard is applied and assigned to host (and related) targetsRecommend disabling or de-associating targets

Old metrics under Basic Security Configuration For Oracle Database

Incident Rules can be used with compliance events

Help Video Library

Many new videos, mostly published through YouTube:

WLS with AD Credentials

Web Logic and Active Directory integrationRelatively easySetup through Web Logic

No longer any need for a DB user (in the OMR) for CC usersLess databases usersLess passwords for CC users

Harvester Replaces OCM

No need to use or run the Oracle Configuration Manager against each database/Oracle homeSame info is already being collected by CC12c

can be uploaded to MOS from thereMust upgrade and configure OCM in the

OMS homeMOS: How to install and configure the OCM Collector for EM 12c Cloud Control? (Doc ID 1556506.1)

Other Recommendations

Use the Diagnostic UtilitiesRepvfy, Omsvfy, Agtfy

Download from MOS and install separatelyKeep up to date manually be checking MOS for new

versions

Dont OEM use to build Data Guard environmentsDoesnt make parameters bi-directional

Looking Forward

Migrate UDMs to Metric Extensions Implementation of Administration GroupsAdd 12c granular Role SecurityAuditingAdd and use Named Credentials

Increased security via credential sharingHA for OMS and OMR

Active/Passive OMS and Data Guard for OMR

Book Report

Oracle Enterprise Manager Cloud Control 12c Deep Dive (Oracle Press)Just another rehash of the documentationDoesnt go into deep detailDoesnt go into tips or recommendations

Conclusions

Cloud Control 12c is the version you want to be at regardless of the version of targets

Many advantages to using Cloud Control

Be prepared to invest timeLearning the complexities of the productTroubleshooting and supportTuning and optimizing

Questions or Comments?

Simon.Pane@first4db.com