database & technology 1 _ martin power _ delivering oracles hight availability database services...

www.logicaltech.com.au

1300 858 117


1300 858 117

Expert@Oracle www.logicaltech.com.au

1300 858 117

Delivering Oracle’s High Availability Database Services in minutes, a case study on OEM 11G

•  Single Server imaging •  Gold Image development and deployments •  Data Guard services with Observers •  Automated Patch Management

MarCn Power, Oracle ACE

High Availability in Minutes


1300 858 117


1300 858 117


1300 858 117

Case Study

•  Large Public Sector Organiza=on •  Employs 60,000 plus staff •  Tradi=onally used Sun E series servers with Solaris

•  Goal – To move to a horizontally scalable High Availability database environment

•  Database Systems are required to move to 99.999% availability across 24 x 7 x 365 operaCng window with 2 scheduled outage per year, not exceeding 4 hours in total



1300 858 117


1300 858 117


1300 858 117

Case Study

What was used to achieve these requirements? •  Restricted list of SOE Hardware, Intel x86 – 64 bit servers. •  OEL 5.5 with hardening •  11.2.0.2 Grid Infrastructure •  11.2.0.2 RAC Database

•  OEM 11g Grid Control with the Provisioning Pack

•  Bare Metal and Single Server Image provisioning •  Gold Image Deployments

•  OEM Patch Plans and Patch Management

•  Data Guard Automated Failover and Fail back



1300 858 117


1300 858 117


1300 858 117

Bare Metal and Single Server Imaging Steps 1.  Build a single server with OS, Hardening and RPMs and Kernel

Configura=on

2.  Build a Bare Metal / Single Server image in OEM. 3.  Iden=fy the servers to Deploy to, PXE boot them from the

network to install the Bare Metal Image

4.  Deploy the Single server image to the servers



1300 858 117


1300 858 117


1300 858 117

!

Bare Metal and Single Server Imaging



1300 858 117


1300 858 117


1300 858 117


!

In OEM, Select, 1.  Deployments 2.  Images 3.  Create Image


1300 858 117


1300 858 117


1300 858 117


Select 1.   OVS Default Image

Enter details; Eg. Name: OEL5_5_x86_64_v1 Descrip=on: Prototype Linux Image Hardened. Product name/Patch number: OEL 5.5 Product version: 5.5

Vendor: Oracle CorporaCon


1300 858 117


1300 858 117


1300 858 117


Select the Target Host – e.g grid11g-‐n1.localdomain

! !


1300 858 117


1300 858 117


1300 858 117


Configure the remaining Image op=ons RPM Repository: select local-‐repo Root Password/Confirm Root Password Select =mezone: Australia/Brisbane

Deselect Use UTC Clock AddiConally you will set •  Username •  Group •  Password •  And enable SUDO Privileges

!


1300 858 117


1300 858 117


1300 858 117


!

On the Customize screen, accept the defaults and click Next


1300 858 117


1300 858 117


1300 858 117


!

Finally OEM submits a job to clone Linux OS DONE!


1300 858 117


1300 858 117


1300 858 117

Bare Metal and Single Server Imaging Deployment

Deployment 1.  PXE boot the new server(s) 2.  Deploy the Bare Metal Image 3.  Deploy the Single Server Image


1300 858 117


1300 858 117


1300 858 117


PXE Boot the servers

!

Update the /etc/dhcp.conf file on the PXE boot server with the server MAC address host llama0 { hardware ethernet 04:4B:80:80:80:03; fixed-‐address 192.168.0.254; op=on host-‐name "llama0"; filename "pxelinux.0"; }


1300 858 117


1300 858 117


1300 858 117


A Single Server Image contains an operaCng system based on the Default Image, together with tailored disk layout, profile and network detail components,

!!

Set Host Preferred CredenCals


1300 858 117


1300 858 117


1300 858 117


!

!

!

In OEM Grid Control, click through to the Deployments > Provisioning > Assignments

Create Image Assignment

Single Server Image


1300 858 117


1300 858 117


1300 858 117


!

!

1. Select buion beside the New Network

2. Choose Network screen, select the required Network Component

3. Edit Advanced ProperCes


1300 858 117


1300 858 117


1300 858 117


On the Edit Advanced ProperCes screen, enter the required custom details: Agent Base Installa=on Directory: /opt/oracle Oracle Central Inventory: /opt/oracle/oraInventory Enable SELinux: Permissive Kernel Parameter Sekngs: change the following line from: net.ipv4.conf.all.log_marCans = 1 …to: net.ipv4.conf.all.log_marCans = 0 (This is a bug that will leads to broadcasCng of errors by the network cards) Post Sec=on Scripts: add the following commands: mkdir /nfs mount lgest:/orastage /nfs -‐o nolock cat /nfs/scripts/post-‐install.sh > /etc/init.d/S90post_install cp /nfs/scripts/conf-‐files.tar /opt/oracle/ chmod 777 /etc/init.d/S90post_install ln -‐s /etc/init.d/S90post_install /etc/rc.d/rc3.d/ umount /nfs

!


1300 858 117


1300 858 117


1300 858 117


!

!

select the required Hostname

Click the Add buion


1300 858 117


1300 858 117


1300 858 117


!

ConfirmaCon screen is shown, indicaCng the image deployment is ready

Now we return to the server and reboot to pick up the Single Server Image.


1300 858 117


1300 858 117


1300 858 117


Select OEM Grid Control under Deployments > Provisioning > Assignments; click the Provision

!On the Provision Assignments page:

Set the Schedule Type: One Time (Immediately) Enter (and confirm) the Agent RegistraCon Password

Select the (previously) Assigned Image


1300 858 117


1300 858 117


1300 858 117


On the Assignment Status page, click the Schedule’s Job ID

!


1300 858 117


1300 858 117


1300 858 117


!

!

!

Press the refresh bugon to watch the Single Server Image provisioning process execute.

Successful comple=on


1300 858 117


1300 858 117


1300 858 117


Symptoms: NB: The following issue must be handled immediately once the Provisioning process has been started and must be completed before the server reboots. The single-‐server image tar and directories created by the assignment staging process are located on OMS in the /oracle/stage/Images directory; and comprise an oms.proper=es file, together with a tar file and corresponding sub-‐directories that match the provisioned single-‐server images. These single-‐server image tar/directories are named according to the NIC MAC address associated with the assignment. An example is: oms.proper=es File : 9B1CA81DDA4535B2E04400144FD42308_0.2_97D1A147B2AC1E63E04400144FD42308.tar File 9B1CA81DDA4535B2E04400144FD42308_0.2_97D1A147B2AC1E63E04400144FD42308 Directory 9DB50A0232DA68E3E04400144FD42308 Directory : An Oracle Service Request (SR) has been raised to resolve an issue with the above oms.properCes file, which is created with missing entries.

Issues


1300 858 117


1300 858 117


1300 858 117


Cause: Fault/bug in the provisioning process. Solu=on: The oms.proper=es file has to be manually updated with the missing details; namely the OMS Host and Port details. As an example, the original file contains: s_encrSecurePwd="05a61eae5282b1219ee52898b7bd189cd077b40996815fde61" …and should be updated (as the root user) to: s_encrSecurePwd="05a61eae5282b1219ee52898b7bd189cd077b40996815fde61" s_OMSHost="qhsz007p.LT-‐test.logicaltech.com.au" s_OMSPort="1159” Once updated, the oms.proper=es file needs to be copied into the single-‐server image’s subdirectory; for example: 9B1CA81DDA4535B2E04400144FD42308_0.2_97D1A147B2AC1E63E04400144FD42308 …overwri=ng the default one created therein by the assignment process. ised to resolve an issue with the above oms.properCes file, which is created with missing entries.


1300 858 117


1300 858 117


1300 858 117

Gold Image Deployment Sequencing 1.   Install 11gR2 Grid Infrastructure and Database as a Single

Node Cluster.

2.   Create a Component in OEM (Image of the server)

3.   IdenCfy the servers to Deploy to.

4.   Deploy the Gold Image to the servers



1300 858 117


1300 858 117


1300 858 117

!

Create the Gold Image



1300 858 117


1300 858 117


1300 858 117


!

IdenCfy the •  Host •  Grid Home •  Grid User & Group •  Etc.



1300 858 117


1300 858 117


1300 858 117


!

!

Job is Created and run

Confirms compleCon



1300 858 117


1300 858 117


1300 858 117

Deploying the Gold Image

!



1300 858 117


1300 858 117


1300 858 117


!

Select Image and schedule deployment



1300 858 117


1300 858 117


1300 858 117


!

Set the •  Cluster Name, •  Hosts •  Scan DNS Name •  Scan Port •  Network Bonding Details



1300 858 117


1300 858 117


1300 858 117


!

Set the •  Discovery Path •  ASM Disk Details •  OCR Disk Group



1300 858 117


1300 858 117


1300 858 117


!

A job is scheduled •  View the job process



1300 858 117


1300 858 117


1300 858 117

Case Study Comparison of 2 build methods •  Build a 3 Node Linux Cluster, 2 to 3 days

•  Deploy 3 Node Linux Cluster with Single server Images, 6 – 10 minutes

•  Build a 3 Node Grid Infrastructure & Database Cluster, 1 day •  Deploy a 3 Node RAC Cluster, 10 minutes

All nodes are idenCcal…. Use DBCA to create RAC databases



1300 858 117


1300 858 117


1300 858 117

Data Guard Deployment with Observer



1300 858 117


1300 858 117


1300 858 117


!

!

!

From OEM select, •  RAC Database •  Select Availability tab •  Select the Add Standby Database OEM Provisioning runs through all the step just like the Gold Image Deployment


In 5 minute you will have a single node standby database


1300 858 117


1300 858 117


1300 858 117


!

Convert to Cluster Database: Cluster CredenCals page, enter/select: In the Cluster CredenCals sec=on:

Oracle Home: for example /u02/app/oracle/product/11.2.0.2/db_1 Username (of the o/s Oracle so@ware owner): oracle Password: (as required)

In the ASM CredenCals sec=on: Username (of the SYSDBA user running the ASM instance): sys Password: (as required) Connect As: SYSDBA Tick the Save as Preferred CredenCals checkbox

!



1300 858 117


1300 858 117


1300 858 117


dgmgrl sys/[password]

show database verbose tadbsb;

Database -‐ tadbsb Role: PHYSICAL STANDBY Intended State: APPLY-‐ON Transport Lag: 0 seconds Apply Lag: 0 seconds Real Time Query: OFF Instance(s): tadbsb_1 (apply instance) tadbsb_2 Proper=es: DGConnectIdenCfier = '(DESCRIPTION = (LOAD_BALANCE = ON)(ADDRESS = (PROTOCOL = TCP)(HOST = qhrac2-‐002t-‐v)(PORT = 1521))(CONNECT_DATA = (SERVICE_NAME = tadbsb)))' ObserverConnectIden=fier = '' LogXptMode = 'ASYNC' DelayMins = '0'

Issues

Host address is wrong It should be the SCAN address


1300 858 117


1300 858 117


1300 858 117

Case Study Patching (Out of Place Patching)


Clone the Oracle Home

tar -‐cf -‐ * -‐-‐exclude=*.aud | (cd ../db_2;tar -‐xvf -‐ ) Update the OraInventory ./runInstaller -‐silent -‐clone ORACLE_HOME="/u02/app/oracle/product/11.2.0.2/db_2" ORACLE_HOME_NAME="OraDb11g_home2" ORACLE_BASE="/u02/app/oracle" Amend the database home details held in cluster services srvctl stop database -‐d tadbnew srvctl modify database -‐d tadbnew -‐o /u02/app/oracle/product/11.2.0.2/db_2 srvctl start database -‐d tadbnew


1300 858 117


1300 858 117


1300 858 117



Create Patch Plans

Patch Cache Maximum Size (MB): 700 Use Dump Directory Radio check buion set to Yes /var/tmp/


1300 858 117


1300 858 117


1300 858 117



Awer selec=ng the patches press the upload buion The patches are then staged and a success message returned Now click the apply to schedule the job that runs the patching process


1300 858 117


1300 858 117


1300 858 117




1300 858 117


1300 858 117


1300 858 117


Schedule the applicaCon of the patch set.


1300 858 117


1300 858 117


1300 858 117



•  Set the details for the patch type •  Select the patch sets


1300 858 117


1300 858 117


1300 858 117



•  IdenCfy the databases to patch •  Watch the job execute


1300 858 117


1300 858 117


1300 858 117

Case Study Patching

•  Based on Agreed Patch Plan Standards •  Rolling Hot Patch ApplicaCon •  3 Data Centers



1300 858 117


1300 858 117


1300 858 117

Case Study 1 Patching Sequence

Primary Database Standby Database

Normal Opera=on

Primary Database

Create a third at the primary data center

(Alternate Servers) 1

2

3

Standby Database

Switch to DR site 4 Patch the databases cold

Standby synchronizes



1300 858 117


1300 858 117


1300 858 117

Primary Database Standby Database

Normal Opera=on

Primary Database Create a third Standby

at the primary data center

(Alternate Servers)

6

5

7

Standby Database

Switch back to original Primary

9

Patch the databases cold

Standby synchronizes

8

Drop the temporary Standby database

11 Drop the temporary Standby database

10 Resynchronize the standby



1300 858 117


1300 858 117


1300 858 117

Summary

Case Study Benefits •  Build 3 Linux Cluster Nodes, 2 to 3 days ($2500 -‐ $4000)

•  Deploy 3 Linux Nodes with Single server Images, 6 – 10 minutes ($25.00)

•  Build a 3 Grid Infrastructure & Database Cluster, 1 day ($1400) •  Deploy a 3 Node RAC Cluster, 10 minutes ($25.00)

•  The hardware and licensing costs delivered a 30% to 50% saving on tradiConal costs with 3 5 x 9’s of availability



1300 858 117


1300 858 117

Business Solu=ons Guaranteed Service Availability Fixed Monthly Cost Scales with you business


1300 858 117


1300 858 117

database & technology 1 _ martin power _ delivering oracles hight availability database services...

Documents