rac implementation 10gr2

8/8/2019 RAC Implementation 10GR2

1/99

2 Node 10G Release 2 RAC IMPLEMENTATION

2 Node 10G Release 2 RAC Implementation

DOCUMENTATION FOR

[ is submitting this documentation to on the understandingthat the contents would not be divulged to any third party without prior written consentfrom ]

Prepared By:

Date: Oct 10th, 2009

Rev: 1.0

Page 1


2/99


Table of ContentsTable of Contents

1 REVISION HISTORY ...................................................................................................................3

2 APPROVALS ...............................................................................................................................4

3 EXISTING INFRASTRUCTURE ...................................................................................................5

4 TECHNICAL SOLUTION .............................................................................................................6

5 ORACLE CLUSTER WARE INSTALLATION OF 10.2.0.1 & CREATION OF STRUCTUREDATABASE ....................................................................................................................................6

5.1 CONFIGURATIONOF SHARED DISKS.................................................................................................65.2 USERAND GROUP CREATION.........................................................................................................6

5.3 CONFIGURING SSH ONALLCLUSTERNODES......................................................................................75.4 CREATIONOF DISKDRIVES............................................................................................................9# CHOWNROOT:DBA /DEV/RDSK/C4T600A0B8000566B30000007B04ACD6E75D0S0 ................. ..... .....9

5.5 TABLESPACE MANAGEMENTPROPERTIES.................................................................................. .......105.6 SETTING KERNELPARAMETERS.................................................................................................... .105.7 10G RELEASE 2 RAC PRE-INSTALLATION TASKS...........................................................................10

5.7.1 Establish Oracle environment variables:.......................................................................125.8 USING ORACLE UNIVERSAL INSTALLERTO INSTALL ORACLE CLUSTERWAREON WINDOWS..........................135.9 USING THE ORACLE UNIVERSAL INSTALLER TO INSTALL ORACLE 10.2.0.1 REAL APPLICATION CLUSTERSBINARIESSOFTWARE........................................................................................................................ .295.10 DATABASES PATCH APPLIED......................................................................................................365.11 STEPSFORCONFIGURING DATABASEAND LISTENER CONFIGURATION...................................................49

5.11.1 Configuration of Listener .............................................................................................49

6 DATABASE CREATION - INDCECDS ..................................................................................... .58

6.1 DATABASECREATIONWITH ASM ..................................................................................................59

7 CONFIGURING TAF FOR THE INDCECDS DATABASE .................................................. ...90

7.1 TAF-FAILOVERTESTINGFORORACLEDATABASEANDAPPLICATIONIN RAC ...........................................91TESTSPERFORMEDAFTERMIGRATION & POSTCONFIGURATIONOF TAF .......................................................92

8 10G RELEASE 2 RAC PRODUCT DOCUMENTATION ............................................ .............. .92

8.1 WHATIS ORACLE 10G RELEASE 2 REAL APPLICATIONS CLUSTERS? ...................................................928.2 ORACLE 10G REAL APPLICATION CLUSTERS CACHE FUSIONTECHNOLOGY.........................................938.3 TRANSPARENT APPLICATIONFAILOVER (TAF)..................................................................................94

8.3.1 Failover Basics .............................................................................................................948.3.2 Duration of Failover .......................................................................................................958.3.3 Client Failover ...............................................................................................................958.3.4 Transparent Application Failover ..................................................................................958.3.5 Elements Affected by Transparent Application Failover ................................................958.3.6 Uses of Transparent Application Failover .....................................................................968.3.7 Database Client Processing During Failover ........................................................ ........978.3.8 Transparent Application Fail over Processing During Shutdowns ...................... ..........988.3.9 Transparent Application Failover Restrictions ...............................................................98

Page 2


3/99


11 Revision HistoryRevision History

Date Author Description Version

10th Oct 2008 Elvis Carlo Final Document 1.0

Page 3


4/99


22 ApprovalsApprovals

Task Name Signature Date

Approved by M/s

Page 4


5/99


33 Existing InfrastructureExisting Infrastructure

The existing infrastructure is listed below

S.No Configuration DB Servers

1 Server Model T5140

2 CPU 2 * 8 Core 1.4GHz UltraSPARC T2 Plus3 Memory 32GB on each node

4 Storage Hitachi 6140

5 RAID concept RAID 6

6 Database version 10.2.0.3 Enterprise Edition (Base product: - 10.2.0.1)

7 Cluster Oracle Clusterware (CRS) 10.2.0.3

8 Load balancing Through TAF

9 Redundancy availability Storage Level

Page 5


6/99


44 Technical SolutionTechnical Solution

In order to improvise the database high availability and to enable to balance the load, opted for implementing 2 node 10G Release 2 Oracle Real Application Cluster onAutomatic Storage Management (ASM). Oracle CRS was installed on both the nodes, which wasthe base layer for installing the Oracle 10G Release 2 Real Application Cluster binaries. All thedata volumes were installed using ASM to enable better I/O performance & databasemaintenance.

IMPLEMENTATION PHASE

During the implementation phase, fail over testing was done at the database level afterconfiguring Transparent Application Fail over (TAF) at the server level. This feature enables tohave an effective fail over to another node, incase the connected node shuts down abruptly.

55 ORACLE CLUSTER WARE INSTALLATION OF 10.2.0.1 &ORACLE CLUSTER WARE INSTALLATION OF 10.2.0.1 &

Creation of Structure DatabaseCreation of Structure Database

5.1 Configuration of Shared Disks

Real Application Clusters requires that all each instance be able to access a set of ASMdiskgroups on a shared disk subsystem. The Oracle instances in Real Application Clusters writedata onto the ASM to update the control file, server parameter file, each datafile, and each redolog file. All instances in the RAC share these ASM diskgroups.

The Oracle instances in the RAC configuration write information to ASM defined for:

The control file

The spfile.ora

Each datafile

Each ONLINE redo log file

5.2 User and Group Creation

Create the OS groups and users as follows on both nodes.

# /usr/sbin/groupadd oinstall# /usr/sbin/groupadd dba# useradd -g oinstall -G dba d /oracle oracle

Page 6


7/99


Set the password for Oracle using

# passwd oracle

5.3 Configuring SSH on all cluster nodes

Before you install and use Oracle Real Application Clusters, you should configure secure shell(SSH) for the oracle user on all cluster nodes. Oracle Universal Installer uses the rsh and scpcommands during installation to run remote commands on and copy files to the other clusternodes. You must configure SSH (or RSH) so that these commands do not prompt for a password.

Use the following steps to create the RSA key pair. Please note that these steps will need to becompleted on both Oracle RAC nodes in the cluster:

1. Logon as the oracle user account.# su - oracle

2. If necessary, create the .ssh directory in the oracle user's home directory and set thecorrect permissions on it:$ mkdir -p /.ssh$ chmod 700 /.ssh

3. Enter the following command to generate an RSA key pair (public and private key) forversion 3 of the SSH protocol:$ /usr/bin/ssh-keygen -t rsaAt the prompts:

o Accept the default location for the key files.

o Enter and confirm apass phrase. This should be different from the oracle user

account password; however it is not a requirement i.e. you do not have to enterany password.

This command will write the public key to the /.ssh/id_rsa.pub file and the private key tothe /.ssh/id_rsa file. Note that you should never distribute the private key to anyone.

4. Repeat the above steps for each Oracle RAC node in the cluster.

Now that both Oracle RAC nodes contain a public and private key pair for RSA, you will need to

create an authorized key file on one of the nodes. An authorized key file is nothing more than asingle file that contains a copy of everyone's (every node's) RSA public key. Once the authorizedkey file contains all of the public keys, it is then distributed to all other nodes in the RAC cluster.

Complete the following steps on one of the nodes in the cluster to create and then distribute theauthorized key file. For the purpose of this article, I am using indc1s209.

1. First, determine if an authorized key file already exists on the node(/.ssh/authorized_keys). In most cases this will not exist since this article assumes you

Page 7


8/99


are working with a new install. If the file doesn't exist, create it now:$ touch /.ssh/authorized_keys$ cd /ssh

2. In this step, use SSH to copy the content of the /.ssh/id_rsa.pub public key from eachOracle RAC node in the cluster to the authorized key file just created(/.ssh/authorized_keys). Again, this will be done from indc1s209. You will be promptedfor the oracle user account password for both Oracle RAC nodes accessed. Notice thatwhen using SSH to access the node you are on (indc1s209), the first time it prompts forthe oracle user account password. The second attempt at accessing this node will promptfor the pass phrase used to unlock the private key. For any of the remaining nodes, it willalways ask for the oracle user account password.

The following example is being run from indc1s209 and assumes a 2-node cluster, withnodes indc1s209 and indc1s210:

$ ssh indc1s209 cat /.ssh/id_rsa.pub >> /.ssh/authorized_keysThe authenticity of host 'indc1s209 (10.200.1.55)' can't be established.RSA key fingerprint is a5:de:ee:2a:d8:10:98:d7:ce:ec:d2:f9:2c:64:2e:e5Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'indc1s209,10.200.1.55' (RSA) to the list of known hosts.oracle@indc1s209's password:$ ssh indc1s210 cat /.ssh/id_rsa.pub >> /.ssh/authorized_keysThe authenticity of host 'indc1s210 (10.200.1.57)' can't be established.RSA key fingerprint is d2:99:ed:a2:7b:10:6f:3e:e1:da:4a:45:d5:34:33:5bAre you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'indc1s210,10.200.1.57' (RSA) to the list of known hosts.oracle@indc1s210's password:

Note: The first time you use SSH to connect to a node from a particular system, you maysee a message similar to the following:

The authenticity of host 'indc1s209 (10.200.1.55)' can't be established.RSA key fingerprint is a5:de:ee:2a:d8:10:98:d7:ce:ec:d2:f9:2c:64:2e:e5Are you sure you want to continue connecting (yes/no)? yes

Enter yes at the prompt to continue. You should not see this message again when youconnect from this system to the same node.

3. At this point, we have the content of the RSA public key from every node in the cluster in

the authorized key file (/.ssh/authorized_keys) on indc1s209. We now need to copy it tothe remaining nodes in the cluster. In out two-node cluster example, the only remainingnode is indc1s210. Use the scp command to copy the authorized key file to all remainingnodes in the cluster:

$ scp /.ssh/authorized_keys indc1s210:.ssh/authorized_keysoracle@indc1s210's password:authorized_keys 100% 1534 1.2KB/s 00:00

Page 8


9/99


4. Change the permission of the authorized key file for both Oracle RAC nodes in thecluster by logging into the node and running the following:

$ chmod 600 /.ssh/authorized_keys

5. At this point, if you use ssh to log in or run a command on another node, you areprompted for the pass phrase that you specified when you created the RSA key. Forexample, test the following from indc1s209:

$ ssh indc1s209 hostnameEnter passphrase for key '/u01/app/oracle/.ssh/id_rsa':indc1s209$ ssh indc1s210 hostnameEnter passphrase for key '/u01/app/oracle/.ssh/id_rsa':indc1s210

5.4 Creation of Disk drives

It is mandatory to create the disk partitions prior to Clusterware & RAC installation.

The hosts should see the shared volumes ashaving the same device identification.Forexample, to a Solaris host, LUN 1 (Volume Name = Oracle Index) is seen as /dev/rdsk/c1t1d0s0on host #1. The same volume-to-LUN mapping must be /dev/rdsk/c1t1d0s0 on host #2. OracleRAC is not set up correctly unless allshared volumes have the same device identification.

In this setup, we have 1 raw device of 256MB for OCR, 1 raw device of 256MB for Voting Disk,1 raw device of 256GB for database files and 1 raw device of 256Gb for archivelogs.Theownership and permissions of the raw devices need to be set as follows.

# chown root:dba /dev/rdsk/c4t600A0B8000566B30000007B04ACD6E75d0s0

# chmod 660 /dev/rdsk/c4t600A0B8000566B30000007B04ACD6E75d0s0# chown root:dba /dev/rdsk/c4t600A0B8000566B64000007F54ACD6EB0d0s0# chmod 660 /dev/rdsk/c4t600A0B8000566B64000007F54ACD6EB0d0s0# chown oracle:dba /dev/rdsk/c4t600A0B8000566B30000007AD4ACD6C12d0s0# chmod 660 /dev/rdsk/c4t600A0B8000566B30000007AD4ACD6C12d0s0# chown oracle:dba /dev/rdsk/c4t600A0B8000566B64000007F34ACD6C5Bd0s0

# chmod 660 /dev/rdsk/c4t600A0B8000566B64000007F34ACD6C5Bd0s0

Page 9


10/99


5.5 Tablespace Management properties

The Oracle Database Configuration Assistant (DBCA) will create a structure database withdefault tablespaces.

Note: Automatic Undo Management requires an undo tablespace per instance therefore youwould require a minimum of 2 tablespaces.

5.6 Setting Kernel parameters

Create a new resource project using the command :

# projadd oracle

Assign the oracle project to the oracle user by adding the following line to the /etc/user_attr file

oracle::::project=oracle

Use the below command to set the SHMMAX to 8GB

# projmod -s -K "project.max-shm-memory=(priv,8gb,deny)" oracle

5.7 10G Release 2 RAC Pre-Installation Tasks

After configuring the raw volumes, perform the following steps prior to installation asadministrator user.

Require 10G Release 2 Clusterware software shipped in Oracle Medias or softwaredownloaded from metalink site only.

Create oracle user & dba group on 2 nodes with administrator user privileges onpartner node, to update the node registry information.

On the node from which you will run the Oracle Universal Installer, set up userequivalence by adding entries for all nodes in the cluster, including the local node.

On both the nodes setup the hosts file with the following entries:10.200.1.55 indc1s20910.200.1.57 indc1s210

192.168.1.2 indc1s209-priv192.168.1.3 indc1s210-priv

Page 10


11/99


10.200.1.59 indc1s209-vip10.200.1.60 indc1s210-vip

Post setting up the hosts file; please ensure to reboot both the nodes, to take effect ofthe new hosts file.

Ensure that the IP allotted to indc1s209-vip & indc1s210-vip is of the same class asthat of Public IP.

If one has to include the IP entries in the DNS file, include both the vip entriesincluding the PUBLIC IPs. All the listeners on both the nodes would be configured onvip entries.

Execute the following commands to check whether the required prerequisites are metprior to initiating the clusterware installation.

Go to /oracle/cluster/cluvfy & run the following command$ runcluvfy.sh stage -pre crsinst -n indc1s209,indc1s210

Please note :- An error for vip node detection failed could be ignored, as there is aknown bug in the runcluvfy.bat file for all IPs starting with 192, 10or 172series.

Determine the complete path for the raw devices or shared file systems, and set up thevoting disk and Oracle Cluster Registry partitions

o During installation, at the Cluster Configuration Storage page, you are asked

to provide paths for two files that must be shared across all nodes of thecluster, either on a shared raw device, or a shared file system file:

o The Cluster Synchronization Services (CSS) voting disk is a partition that

Oracle Clusterware uses to verify cluster node membership and status.Provide at least 256 MB disk space for the voting disk.

o The Oracle Cluster Registry (OCR) contains cluster and database

configuration information for the RAC database and for Oracle Clusterware,including the node list, and other information about cluster configuration andprofiles. Provide at least 256 MB disk space for the OCR.

o In addition, if you intend to use ASM, do not format the partitions that you

want to use for ASM.

o Ensure that you create at least the minimum required partitions for

installation.

Host names, private names, and virtual host names are not domain-qualified. If youprovide a domain in the address field during installation, then the OUI removes thedomain from the address.

Page 11


12/99


Private IP addresses should not be accessible as public interfaces. Using public interfacesfor Cache Fusion can cause performance problems.

Determine your cluster name, public node names, private node names, and virtual node

names for each node in the cluster

If you install the cluster ware during installation, and are not using third-party vendorcluster ware, then you are asked to provide a public node name and a private node namefor each node. Use your third-party vendor documentation to complete setup of yourpublic and private domain addresses.

When you enter the public node name, use the primary host name of each node. In otherwords, use the name displayed by the hostname command but without any portion of thedomain name that may be returned by the command.

In addition, ensure that the following are true:

o Determine a cluster name with the following characteristics:

It must be globally unique throughout your host domain

It must be at least one character long and less than 15 characters long

It must consist of the same character set used for host names: underscores(_), hyphens (-), and single-byte alphanumeric characters (a to z, A to Z,and 0 to 9). If you use third-party vendor clusterware, then Oraclerecommends that you use the vendor cluster name

o Determine a private node name or private IP address for each node. The private

IP address is an address that is only accessible by the other nodes in this cluster.Oracle uses private IP addresses for inter-node, or instance-to-instance CacheFusion traffic. Oracle recommends that you provide a name in the formatpublic_hostname-priv. Example: DB-priv.

o Determine a virtual host name for each node. A virtual host name is a public node

name that is used to reroute client requests sent to the node if the node is down.Oracle uses virtual IP addresses (VIPs) for client to database connections, so theVIP address must be publicly accessible. Oracle recommends that you provide aname in the formatpublic_hostname-vip. Example: DB-vip

5.7.1 Establish Oracle environment variables:

Set the following Oracle environment variables:

Environment Variable Variable values

ORACLE_HOME /oracle/product/10.2.0/db_1

ORA_CRS_HOME /oracle/product/10.2.0/crs_1

Page 12


13/99


5.8 Using Oracle Universal Installer to Install Oracle Clusterwareon Windows

Perform the following procedures to complete phase one, install Oracle Clusterwarewith the Oracle Universal Installer, of the installation of the Oracle Database 10g

Release 2 with Real Application Clusters (RAC):

1. Log in to oracle user & run the runInstaller.sh command on the OracleClusterware media from one of the node only (Primary node i.e. indc1s209),as the same installation window would install the software & would configurethe cluster information automatically on the partner node. This will open theOracle Universal Installer (OUI) Welcome page.

2. After you click Next on the Welcome page, you are prompted for the inventorylocation and the dba group. On the next screen, the Specify File Locationspage will allow you to accept the displayed path name for the OracleClusterware products or select a different one. You may also accept defaultdirectory and path name for the location of your Oracle Clusterware home or

browse for an alternate directory and destination. You must select adestination that exists on each cluster node that is part of this installation.Click Next to confirm your choices.

Page 13


14/99


3. Leave the source path unchanged. Modify the destination as required.

Page 14


15/99


4. The installer verifies that your environment meets all of the minimumrequirements for installing and configuring the products that you have chosento install. The results are displayed on the Product-Specific PrerequisiteChecks page. Verify and confirm the items that are flagged with warnings anditems that require manual checks. After you confirm your configuration, theOUI proceeds to the Cluster Configuration page.

Note:

If the check identifies an existing, local CSS, you must shutdown theOracle database and ASM instance from the Oracle home where CSS

is running. To accomplish this, run the following command, using theexisting Oracle home, in a separate window before you continue with

the installation:Oracle home\bin\localconfig delete

Page 15


16/99


5. The Cluster Configuration page contains predefined node information if theOUI detects that your system has the Oracle9i Release 2 clusterware.Otherwise, the OUI displays the Cluster Configuration page without predefinednode information.

Provide your own cluster name if you do not wish to use the name provided by

the OUI. Note that the selected cluster name must be globally uniquethroughout the enterprise and its allowable character set is the same as thatfor hostnames, that is, underscores (_), hyphens (-), and single-bytealphanumeric characters (a to z, A to Z, and 0 to 9).

Page 16


17/99


Page 17


18/99


6. Enter a public, a virtual, and a private host name for each node. Do notinclude a domain qualifier with the host names. When you enter the publichost name, use the primary host name of each node, that is, the namedisplayed by the hostname command. The virtual node name is the name tobe associated with the VIP for the node. The private node refers to an addressthat is only accessible by the other nodes in this cluster, and which Oracleuses for Cache Fusion processing. You should enter the private host name foreach node.

Note:

You may provide the cluster configuration information in a text fileinstead of entering it in the individual fields on the Cluster Configurationpage. The contents of your cluster configuration file should be similar tothe following example:Cluster configuration file:--------------------------------------------------# Cluster Namecrs

Page 18


19/99


# Node Information# Public Node Name Private Node Name Virtual Host Name

indc1s209 indc1s209-priv indc1s209-vip

indc1s210 indc1s210-priv indc1s210-vip

Click Next after you have entered the cluster configuration information. Thissaves your entries and opens the Specify Network Interface Usage page.

7. In the Specify Network Interface Usage page the OUI displays a list of cluster-wide interfaces. Use the drop-down menus on this page to classify eachinterface as Public, Private, or Do Not Use. The default setting for each

interface is Do Not Use. You mustclassify at least one interconnect as Publicand one as Private. Click Next when you have made your selections to openthe Select Disk Formatting Options page.

Page 19


20/99


8. On the Cluster Configuration Storage page, identify the disks that you want touse for the Oracle Clusterware files. Enter the path of each of these disks oneat a time

Notes:

The OUI page described in this step displays logical drives fromwhich you must make your selections.

If you are installing on a cluster with an existing cluster file systemfrom an earlier release of Oracle, then the OCR and voting disk willbe stored in that file system. In this case, you do not require newpartitions for the OCR and voting disk, even if you do not format alogical drive for data file storage.

Page 20


21/99


9. After you click Next, the OUI checks whether the remote inventories are set. Ifthey are not set, then the OUI sets up the remote inventories by settingregistry keys. The OUI also verifies the permissions to enable writing to theinventory directories on the remote nodes. After completing these actions, theOUI displays a Summary page that shows the cluster node information alongwith the space requirements and availability. Verify the installation that theOUI is about to perform and click Finish.

Page 21


22/99


10. When you click Finish, the OUI installs the Oracle Clusterware software on thelocal node and validates the installation again. After validating the installation,the OUI completes the Oracle Clusterware software installation andconfiguration on the remote nodes.

Page 22


23/99


11.Run the above scripts as mentioned on each node. Post the clusterwareinstallation, invoke the VIPCA from /oracle/product/10.2.0/crs_1/bin toconfigure the VIPs.

Page 23


24/99


Page 24


25/99


Page 25


26/99


Page 26


27/99


Page 27


28/99


Page 28


29/99


5.9 Using the Oracle Universal Installer to install Oracle 10.2.0.1 Real ApplicationClusters binaries software

Follow these procedures to use the Oracle Universal Installer to install the Oracle Enterprise

Edition Cluster Ware installation and the Real Application Clusters software. To install the Oracle10G Release 2 RAC binaries, perform the following:

Prior to installation of 10G Release 2 RAC, check the installation of clusterware bytyping the following command:

# /oracle/product/10.2.0/crs_1/bin/crs_stat

Execute the command on both the nodes. Once the commands are executed successfullyon both the nodes, perform the next steps

Login as the oracle user

Go to the /oracle/database & execute runInstaller.sh, to initiate the installation ofRAC installation

At the OUI Welcome screen, clickNext.

A prompt will appear for the Inventory Location (if this is the first time that OUI hasbeen run on this system). This is the base directory into which OUI will install files. The

Page 29


30/99


Oracle Inventory definition can be found in the file /oracle/product/10.2.0/oraInst.loc.ClickOK.

Select the installation type. Choose the Standard Edition option. The selection on this

screen refers to the installation operation, not the database configuration. The next screenallows for a customized database configuration to be chosen. ClickNext.

The File Location window will appear. Do NOT change the Source field. TheDestination field defaults to the ORACLE_HOME environment variable. ClickNext.

Page 30


31/99


Select the nodes to install the binaries.

Page 31


32/99


Select Software Only and click Next.

Page 32


33/99


The Summary screen will be presented. Confirm that the RAC database software will beinstalled and then clickInstall. The OUI will install the Oracle 10.2.0.1 software on tothe local node, and then copy this information to the other nodes selected.

Page 33


34/99


Once Install is selected, the OUI will install the Oracle RAC software on to the localnode, and then copy software to the other nodes selected earlier. This will take some time.During the installation process, the OUI does not display messages indicating thatcomponents are being installed on other nodes - I/O activity may be the only indicationthat the process is continuing.

Run the scripts as root user as described above

Page 34


35/99


Page 35


36/99


5.10 Databases Patch Applied

The base version of Oracle database installed is 10.2.0.1.0. The database patch-set applied onboth the servers is 10.2.0.4, which is an upgrade patch to take care of critical issues viz.ORA-600 errors & other memory leak issues. Moreover, required patches for Oracle RACwere automatically installed on indc1s209, indc1s210 servers.

Stop all the CRS services on both nodes

Navigate to the patch directory and invoke the runinstaller. Click next on thewelcome screen.

Page 36


37/99


Select the Home to be patched and click Next

Page 37


38/99


Select the nodes to install the Patch.

Page 38


39/99


Click on Next once the pre-requisite check is complete

Page 39


40/99


Click Next on the summary screen to start the Patch installation for CRS

Page 40


41/99


Login as root user and run the below scripts

Page 41


42/99


Page 42


43/99


Exit the installer and invoke it once again from the same location for patching thedatabase binaries

Page 43


44/99


Select the Home to be patched.

Page 44


45/99


Select the nodes to be patched.

Page 45


46/99


Click on Next once the pre-requisite check is complete

Page 46


47/99


Click install on the summary screen to start the patch installation.

Page 47


48/99


Post installation, run the scripts as root user as mentioned

Page 48


49/99


5.11 Steps for configuring Database and Listener Configuration

5.11.1 Configuration of Listener

Prior to running the DBCA it is necessary to run the netca utility to configure the listener or tomanually set up your network files. This will configure the necessary listener names and protocoladdresses, client naming methods, Net service names. Listener are configured with the followingnames i.e. LISTENER_INDC1S209andLISTENER_INDC1S210 to listen on port 1521. It should

be ensured that any new listeners configured should be running on different port other thanconfigured. The configuration file is attached below:

LISTENER_INDC1S209 =(DESCRIPTION_LIST =(DESCRIPTION =(ADDRESS = (PROTOCOL = TCP)(HOST = indc1s209-vip)(PORT = 1521)(IP

= FIRST))(ADDRESS = (PROTOCOL = TCP)(HOST = 10.200.1.55)(PORT = 1521)(IP =

FIRST)))

)

Page 49


50/99


LISTENER_INDC1S210 =(DESCRIPTION_LIST =(DESCRIPTION =(ADDRESS = (PROTOCOL = TCP)(HOST = indc1s210-vip)(PORT = 1521)(IP

= FIRST))(ADDRESS = (PROTOCOL = TCP)(HOST = 10.200.1.57)(PORT = 1521)(IP =

FIRST)))

)

To create the Listener, invoke the NETCA as the oracle user

Select Cluster configuration.

Select All Nodes

Page 50


51/99


Select Listener Configuration

Page 51


52/99


Select Add

Page 52


53/99


Select name as Listener

Page 53


54/99


Select TCP

Page 54


55/99


Select the default port 1521

Page 55


56/99


Select No

Page 56


57/99


Page 57


58/99


66 Database Creation - INDCECDSDatabase Creation - INDCECDSInvoke the database creation window through dbca & create the database with thefollowing initialization parameter file on the both the nodes. ECDSNOD1 & ECDSNOD2

would be created respectively on INDC1S209 & INDC1S210 nodes. The initializationparameters are mentioned as follows :

ECDSNOD1.__db_cache_size=1191182336ECDSNOD2.__db_cache_size=1191182336ECDSNOD1.__java_pool_size=16777216ECDSNOD2.__java_pool_size=16777216ECDSNOD1.__large_pool_size=16777216ECDSNOD2.__large_pool_size=16777216ECDSNOD1.__shared_pool_size=352321536ECDSNOD2.__shared_pool_size=352321536ECDSNOD1.__streams_pool_size=0ECDSNOD2.__streams_pool_size=0

*.audit_file_dest='/oracle/product/10.2.0/db_1/admin/INDCECDS/adump'*.background_dump_dest='/oracle/product/10.2.0/db_1/admin/INDCECDS/bdump'*.cluster_database_instances=2*.cluster_database=true*.compatible='10.2.0.3.0'*.control_files='+DGDATA/INDCECDS/control01.ctl','+DGDATA/INDCECDS/control02.ctl','+DGDATA/INDCECDS/control03.ctl'*.core_dump_dest='/oracle/product/10.2.0/db_1/admin/INDCECDS/cdump'*.db_block_size=8192*.db_domain=''*.db_file_multiblock_read_count=16*.db_name='INDCECDS'*.dispatchers='(PROTOCOL=TCP) (SERVICE=ECDSNODXDB)'

ECDSNOD1.instance_number=1ECDSNOD2.instance_number=2*.job_queue_processes=10*.log_archive_dest_1='LOCATION=+DGARCH/'*.log_archive_format='ECDSNOD_%t_%s_%r.arc'*.open_cursors=300*.pga_aggregate_target=2548039680*.processes=150*.remote_listener='LISTENERS_INDCECDS'*.remote_login_passwordfile='exclusive'*.sga_target=1610612736ECDSNOD2.thread=2ECDSNOD1.thread=1

*.undo_management='AUTO'ECDSNOD1.undo_tablespace='UNDOTBS1'ECDSNOD2.undo_tablespace='UNDOTBS2'*.user_dump_dest='/oracle/product/10.2.0/db_1/admin/INDCECDS/udump'

Page 58


59/99


6.1 Database creation with ASM

Invoke the DBCA as oracle user

Select the RAC option and click next.

Page 59


60/99


Select configure Automatic Storage Management and click next

Page 60


61/99


Select all nodes and click next

Page 61


62/99


Enter the password for the ASM instance as sys. Select create initialization parameterfile and click next.

Page 62


63/99


Page 63


64/99


Click on create new to create the diskgroups

Page 64


65/99


Click on OK

Page 65


66/99


Click on Create New to create the DGARCH disk group

Page 66


67/99


Click on OK

Page 67


68/99


After all the diskgroups are created, click on Finish

Page 68


69/99


Click on Yes to return to the DBCA main screen

Page 69


70/99


Select Oracle RAC database and click next

Page 70


71/99


Select create a database and click next.

Page 71


72/99


Select all and click next.

Page 72


73/99


Select custom database and click next.

Page 73


74/99


Enter the name for the RAC database. For each cluster database instance, the SID iscomprised of a common prefix for the database and a number for each instance that is

automatically generated.

Page 74


75/99


Leave the default options checked.

Page 75


76/99


Specify the database user passwords. Here the password is given as indecds.

Page 76


77/99


Select Automatic Storage Management.

Page 77


78/99


Select diskgroup name as DGDATA.

Page 78


79/99


Specify the diskgroup ARCHIVEDG for archivelogs.

Page 79


80/99


Page 80


81/99


Select all required database components and click Next.

Page 81


82/99


Click on Next.

Page 82


83/99


Page 83


84/99


Page 84


85/99


Page 85


86/99


Page 86


87/99


Page 87


88/99


Click on exit. Oracle will now start the RAC instance on both nodes.

Page 88


89/99


This completes the database creation process.

Page 89


90/99


77 Configuring TAF for the INDCECDS databaseConfiguring TAF for the INDCECDS database The (load_balance=yes) parameter instructs net protocol to proceed through the list of

listener addresses in a random sequence, balancing the load on the various listeners.

When set to OFF, instructs Net to try the addresses sequentially until one succeeds. Thisparameter must be correctly coded in your net service name (connect descriptor). Bydefault, this parameter is set to ON for DESCRIPTION_LISTs. Load balancing can bespecified for an ADDRESS_LIST or associated with a set of Addresses or setDESCRIPTIONs. If you use ADDRESS_LIST,(load_balance=yes) should be within the(ADDRESS_LIST=) portion. If you do not use ADDRESS_LIST, (load_balance=yes)should be within the (description=) portion.

(failover=on) is default for ADDRESS_LISTs, DESCRIPTION_LISTs, and a set ofDESCRIPTIONs., therefore, you do not have to specify. This is for connect-time-failover,please do not confuse it with transparent application failover (TAF).

failover_mode=): The FAILOVER_MODE parameter must be included in theCONNECT_DATA portion of a net_service_name.

The tnsnames.ora file configured w.r.t. TAF Configuration is attached the next :

INDCECDS =(DESCRIPTION =(ADDRESS = (PROTOCOL = TCP)(HOST = indc1s209-vip)(PORT = 1521))(ADDRESS = (PROTOCOL = TCP)(HOST = indc1s210-vip)(PORT = 1521))(LOAD_BALANCE = yes)(CONNECT_DATA =(SERVER = DEDICATED)(SERVICE_NAME = INDCECDS)(FAILOVER_MODE =(TYPE = SELECT)(METHOD = BASIC)

(RETRIES = 5)(DELAY = 5)

))

)

Page 90


91/99


7.1 TAF-Failover testing for oracle database and application in RAC

Before proceeding with the testing, start the listener. Check whether tnsnames.oraconfiguration is working by doing tnsping to tnsnames for all instances as below.

In indc1s209,

$ lsnrctl start LISTENER_INDC1S209

$ lsnrctl services LISTENER_INDC1S209

$ tnsping INDCECDS

In indc1s210,

$ lsnrctl start LISTENER_INDC1S210

$ lsnrctl services LISTENER_INDC1S210

$ tnsping INDCECDS

2..On indc1s209, connect to the instance:

$ sqlplus system/********@INDCECDS

3. From the session issue a select:

SQL> select host_name, thread# from v$instance;

HOST_NAME THREAD#

INDC1S209 1

4. On indc1s209 abort instance ECDSNOD1 as below

SQL > shutdown

Database unmounted

Database closed

5. Issue another select from the sqlplus session above.

SQL> select host_name, thread# from v$instance;

HOST_NAME THREAD#

INDC1S210 2

Note, the session got failed over from instance 1 (ECDSNOD1) in indc1s209 to instance 2(ECDSNOD2) indc1s210.

6. The same above testing has been done in both instances.

Page 91


92/99


Tests performed after migration & post configuration of TAF

Sr.No. Name of Test Description of Test Required Criteria Result of Test

1

The first instancefailover(ECDSNOD1)

Users should be able to see& do operation on dataeven the first instanceECDSNOD1 in indc1s209server is down

Connected a session todatabase INDCECDS.Checked the instanceconnected. Shutdown theinstance ECDSNOD1 inindc1s209. Again checked theinstance connected to which isnow the instanceECDSNOD2. Test Successful

2

The first instance failover (ECDSNOD2)

Users should be able to see& do operation on dataeven the one of theinstance (ECDSNOD2 inindc1s210 server) is down

Connected a session todatabase INDCECDS.

Checked the instanceconnected (ECDSNOD2).Shutdown the instanceECDSNOD2 in indc1s210node. Again checked theinstance connected to which isnow the instance ECDSNOD1.Test Successful

3.

Shutdown of theentire database and

starting up the same insequence

Database should comewithout any error after

migration in all 2 servers(indc1s209, indc1s210)

Database came up without anyerror. Test successful.

88 10G Release 2 RAC PRODUCT DOCUMENTATION10G Release 2 RAC PRODUCT DOCUMENTATION

8.1 What is Oracle 10G Release 2 Real Applications Clusters?

Oracle 10G Release 2 Real Application Clusters is a computing environment thatharnesses the processing power of multiple, interconnected computers. Oracle 10GRelease 2 Real Application Clusters software and a collection of hardware known as a"cluster," unites the processing power of each component to become a single, robust

computing environment. A cluster generally comprises two or more computers, or"nodes."

In Oracle 10G Release 2 Real Application Clusters (RAC) environments, all nodes concurrentlyexecute transactions against the same database. Oracle 10G Release 2 Real Application Clusterscoordinates each node's access to the shared data to provide consistency and integrity.

Page 92


93/99


Oracle 10G Release 2 Real Application Clusters serves as an important component of robust highavailability solutions. A properly configured Oracle 10G Release 2 Real Application Clustersenvironment can tolerate failures with minimal downtime.

Oracle 10G Release 2 Real Application Clusters is also applicable for many other system types.For example, data warehousing applications accessing read-only data are prime candidates forOracle 10G Release 2 Real Application Clusters. In addition, Oracle 10G Release 2 RealApplication Clusters successfully manages increasing numbers of online transaction processingsystems as well as hybrid systems that combine the characteristics of both read-only andread/write applications.

Harnessing the power of multiple nodes offers obvious advantages. If you divide a large task intosub-tasks and distribute the sub-tasks among multiple nodes, you can complete the task fasterthan if only one node did the work. This type of parallel processing is clearly more efficient thansequential processing. It also provides increased performance for processing larger workloads andfor accommodating growing user populations. Oracle 10G Release 2 Real Application Clusters

can effectively scale your applications to meet increasing data processing demands. As you addresources, Oracle 10G Release 2 Real Application Clusters can exploit them and extend theirprocessing powers beyond the limits of the individual components.

From a functional perspective RAC is equivalent to single-instance Oracle. What the RACenvironment does offer is significant improvements in terms of availability, scalability andreliability.

In recent years, the requirement for highly available systems, able to scale on demand, hasfostered the development of more and more robust cluster solutions. Prior to Oracle 10G Release2, HP and Oracle, with the combination of Oracle Parallel Server and HP Service Guard OPSedition, provided cluster solutions that lead the industry in functionality, high availability,management and services. Now with the release of Oracle 10G Real Application Clusters (RAC)

with the new Cache Fusion architecture based on an ultra-high bandwidth, low latency clusterinterconnect technology, RAC cluster solutions have become more scalable without the need fordata and application partitioning. The information contained in this document covers theinstallation and configuration of Oracle Real Application Clusters in a typical environment; a twonode HP cluster, utilizing the HP-UX operating system.

8.2 Oracle 10G Real Application Clusters Cache Fusion technology

Oracle 10G cache fusion utilizes the collection of caches made available by all nodes inthe cluster to satisfy database requests. Requests for a data block are satisfied first by alocal cache, then by a remote cache before a disk read is needed. Similarly, updateoperations are performed first via the local node and then the remote node caches in the

cluster, resulting in reduced disk I/O. Disk I/O operations are only done when the datablock is not available in the collective caches or when an update transaction performs acommit operation.

Oracle 10G cache fusion thus provides Oracle users an expanded database cache for queries andupdates with reduced disk I/O synchronization which overall speeds up database operations.However, the improved performance depends greatly on the efficiency of the inter-node messagepassing mechanism, which handles the data block transfers between nodes.

Page 93


94/99


The efficiency of inter-node messaging depends on three primary factors:

The number of messages required for each synchronization sequence. Oracle10Gs Distributed Lock Manager (DLM) coordinates the fast block transfer

between nodes with two inter- node messages and one intra-node message. Ifthe data is in a remote cache, an inter-node message is sent to the LockManager Daemon (LMD) on the remote node. The DLM and Cache Fusionprocesses then update the in-memory lock structure and send the block to therequesting process.

The frequency of synchronization (the less frequent the better). The cache fusionarchitecture reduces the frequency of the inter-node communication bydynamically migrating locks to a node that shows a frequent access pattern for aparticular data block. This dynamic lock allocation increases the likelihood of localcache access thus reducing the need for inter-node communication. At a nodelevel, a cache fusion lock controls access to data blocks from other nodes in the

cluster.The latency of inter-node communication. This is a critical component in Oracle 10GRAC as it determines the speed of data block transfer between nodes. An efficienttransfer method must utilize minimal CPU resources; support high availability as well ashighly scalable growth without bandwidth constraints.

8.3 Transparent Application failover (TAF).

The Transparent Application Failover (TAF) feature is a runtime failover for high-availabilityenvironments, such as Oracle 10G Release 2 Real Application Clusters and Oracle 10G Release 2Real Application Clusters Guard. TAF fails over and reestablishes application-to-serviceconnections. It enables client applications to automatically reconnect to the database if the

connection fails and, optionally, resume a SELECT statement that was in progress. Thereconnection happens automatically from within the Oracle Call Interface (OCI) library. Tounderstand the concept and working flow of TAF,first we need to understand the below failoverbasics.

8.3.1 Failover Basics

Failover requires that highly available systems have accurate instance monitoring or heartbeatmechanisms. In addition to having this functionality for normal operations, the system must beable to quickly and accurately synchronize resources during failover.

The process of synchronizing, or remastering, requires the graceful shutdown of the failingsystem as well as an accurate assumption of control of the resources that were mastered on thatsystem. In Real Application Clusters, your system records resource information to remote nodesas well as local. This makes the information needed for failover and recovery available to therecovering instances.

Page 94
http://download-west.oracle.com/docs/cd/B10501_01/rac.920/a96597/glossary.htm#432497http://download-west.oracle.com/docs/cd/B10501_01/rac.920/a96597/glossary.htm#432497


95/99


8.3.2 Duration of Failover

The duration of failover includes the time a system requires to remaster system-wide resourcesand the time to recover from failures. The duration of the failover process can be a relatively short

interval on certified platforms.

For existing users, failover entails both server and client failover actions

For new users, failover only entails the duration of server failover processing

8.3.3 Client Failover

It is important to hide system failures from database client connections. Such connections caninclude application users in client server environments or middle-tier database clients inmultitiered application environments. Properly configured failover mechanisms transparentlyreroute client sessions to an available node in the cluster. This capability in the Oracle database isreferred to as Transparent Application Failover

8.3.4 Transparent Application Failover

Transparent Application Failover (TAF) enables an application user to automatically reconnectto a database if the connection fails. Active transactions roll back, but the new databaseconnection, which is achieved using a different node, is identical to the original. This is trueregardless of how the connection fails.

8.3.5 Elements Affected by Transparent Application Failover

There are several elements associated with active database connections. These include:

Client/Server database connections

Users' database sessions executing commands

Open cursors used for fetching

Active transactions

Server-side program variables

Transparent Application Failover automatically restores some of these elements. For example,during normal client/server database operations, a client maintains a connection to the database sothe client and server can communicate. If the server fails, then so does the connection. The nexttime the client tries to use the connection the client issues an error. At this point, the user must login to the database again.

With Transparent Application Failover, however, Oracle automatically obtains a new connectionto the database. This enables users to continue working as if the original connection had neverfailed. Therefore, with Transparent Application Failover, a client notices no connection loss aslong as one instance remains active to serve the application

Page 95
http://download-west.oracle.com/docs/cd/B10501_01/rac.920/a96597/glossary.htm#433759http://download-west.oracle.com/docs/cd/B10501_01/rac.920/a96597/glossary.htm#433759


96/99


8.3.6 Uses of Transparent Application Failover

While the ability to fail over client sessions is an important benefit of Transparent ApplicationFail over, there are other useful scenarios where Transparent Application Fail over improves

system availability. These topics are discussed in the following subsections:

Transactional Shutdowns

Quiescing the Database

Load Balancing

Database Client Processing During Failover

Transparent Application Failover Restrictions

8.3.6.1 Transactional Shutdowns

It is sometimes necessary to take nodes out of service for maintenance or repair. For example, ifyou want to apply patch releases without interrupting service to application clients. Transactionalshutdowns facilitate shutting down selected nodes rather than an entire database. Twotransactional shutdown options are available:

Use theTRANSACTIONAL clause of the SHUTDOWN statement to remove a nodefrom service so that the shutdown event is deferred until all existing transactionsare completed. In this way, client sessions can be migrated to another node ofthe cluster at transaction boundaries.

Use theTRANSACTIONAL LOCAL clause of the SHUTDOWN statement to performtransactional shutdown on a specified local instance. You can use this statementto prevent new transactions from starting locally, and to perform an immediateshutdown after all local transactions have completed. With this option, you cangracefully move all sessions from one instance to another by shutting downselected instances transactionally.

After performing a transactional shutdown, Oracle routes newly submitted transactions to analternate node. An immediate shutdown is performed on the node when all existing transactionscomplete.

8.3.6.2 Quiescing the Database

You may need to perform administrative tasks that require isolation from concurrent usertransactions or queries. To do this, you can use the quiesce database feature. This prevents you,for example, from having to shut down the database and re-open it in restricted mode to perform

such tasks.

To do this, you can use the ALTER SYSTEM statement with the QUIESCE RESTRICTEDclause. The QUIESCE RESTRICTED clause enables you to perform administrative tasks inisolation from concurrent user transactions or queries.

Note:

Page 96
http://download-west.oracle.com/docs/cd/B10501_01/rac.920/a96597/pshavdtl.htm#7444http://download-west.oracle.com/docs/cd/B10501_01/rac.920/a96597/pshavdtl.htm#16003http://download-west.oracle.com/docs/cd/B10501_01/rac.920/a96597/pshavdtl.htm#6994http://download-west.oracle.com/docs/cd/B10501_01/rac.920/a96597/pshavdtl.htm#7001http://download-west.oracle.com/docs/cd/B10501_01/rac.920/a96597/pshavdtl.htm#20180http://download-west.oracle.com/docs/cd/B10501_01/rac.920/a96597/pshavdtl.htm#7444http://download-west.oracle.com/docs/cd/B10501_01/rac.920/a96597/pshavdtl.htm#16003http://download-west.oracle.com/docs/cd/B10501_01/rac.920/a96597/pshavdtl.htm#6994http://download-west.oracle.com/docs/cd/B10501_01/rac.920/a96597/pshavdtl.htm#7001http://download-west.oracle.com/docs/cd/B10501_01/rac.920/a96597/pshavdtl.htm#20180


97/99


You cannot open the database on one instance if the database is beingquiescedon another node. In other words, if you issued the ALTER SYSTEMQUIESCE RESTRICTED statement but it is not finished processing, you

cannot open the database. Nor can you open the database if it is already in aquiesced state.

8.3.6.3 Load Balancing

A database is available when it processes transactions in a timely manner. When the load exceedsa node's capacity, client transaction response times are adversely affected and the databaseavailability is compromised. It then becomes important to manually migrate client sessions to aless heavily loaded node to maintain response times and application availability.

In Real Application Clusters, the Transport Network Services (TNS) listener files provideautomated load balancing across nodes in both shared server and dedicated server configurations.Because the parameters that control cross-instance registration are also dynamic, Real ApplicationClusters' load balancing feature automatically adjusts for cluster configuration changes. For

example, if you add a node to your cluster database, then Oracle updates all the listener files inthe cluster with the new node's listener information.

8.3.7 Database Client Processing During Failover

Failover processing for query clients is different than the failover processing for DatabaseManipulation Language clients. The important issue during failover operations in either case isthat the failure is masked from existing client connections as much as possible. The followingsubsections describe both types of failover processing.

8.3.7.1 Query Clients

At failover, in-progress queries are reissued and processed from the beginning. This might extendthe duration of the next query if the original query required longer to complete. With Transparent

Application Failover (TAF), the failure is masked for query clients with an increased responsetime being the only issue affecting the client. If the client query can be satisfied with data in thebuffer cache of the surviving node to which the client reconnected, then the increased responsetime is minimal. Using TAF's PRECONNECT method eliminates the need to reconnect to asurviving instance and thus further minimizes response time. However, PRECONNECTallocates resources awaiting the failover event.

After failover, server-side recovery must complete before access to the datafiles is allowed. Theclient transaction experiences a system pause until server-side recovery completes, if server-siderecovery has not already completed.

You can also use a callback function through an OCI call to notify clients of the failover so thatthe clients do not misinterpret the delay for a failure. This prevents the clients from manually

attempting to reestablish connections.

8.3.7.2 Database Manipulation Language Clients

Database Manipulation Language (DML) database clients perform INSERT, UPDATE, andDELETE operations. Oracle handles certain errors and performs a reconnect when those errorsoccur.

Page 97


98/99


Without this application code, INSERT, UPDATE, and DELETE operations on the failedinstance return an un-handled Oracle error code. Upon re-submission, Oracle routes the clientconnections to a surviving instance. The client transaction then stops only momentarily untilserver-side recovery completes.

8.3.8 Transparent Application Fail over Processing DuringShutdowns

Queries that cross the network after shutdown processing completes will fail over. However,Oracle returns an error for queries that are in progress during shutdowns. Therefore, TAF onlyoperates when the operating system returns a network error and the instance is completely down.

Applications that use TAF for transactional shutdown must be written to process the error

ORA-01033 "ORACLE initialization or shutdown in progress". In the event of a failure, aninstance will return error ORA-01033 once shutdown processing begins. Such applications needto periodically retry the failed operation, even when Oracle reports multiple ORA-01033 errors.When shutdown processing completes, TAF recognizes the failure of the network connection to

instance and restores the connection to an available instance.

Connection load balancing improves connection performance by balancing the number of activeconnections among multiple dispatchers. In single-instance Oracle environments, the listenerselects the least loaded dispatcher to manage incoming client requests. In Real ApplicationClusters environments, connection load balancing also has the capability of balancing the numberof active connections among multiple instances.

Due to dynamic service registration, a listener is always aware of all of the instances anddispatchers regardless of their locations. Depending on the load information, a listener determinesto which instance and to which dispatcher to send incoming client requests if you are using theshared server configuration.

In shared server configurations, listeners select dispatchers using the following criteria in theorder shown:

1. Least loaded node

2. Least loaded instance

3. Least loaded dispatcher for that instance

In dedicated server configurations, listeners select instances in the following order:

1. Least loaded node

2. Least loaded instance

If a database service has multiple instances on multiple nodes, then the listener chooses the leastloaded instance on the least loaded node. If you have configured the shared server, then the leastloaded dispatcher of the selected instance is chosen.

8.3.9 Transparent Application Failover Restrictions

When a connection fails, you might experience the following:

Page 98


99/99


All PL/SQL package states on the server are lost at failover

ALTERSESSION statements are lost

If failover occurs when a transaction is in progress, then each subsequent callcauses an error message until the user issues an OCITransRollback call. ThenOracle issues an Oracle Call Interface (OCI) success message. Be sure to checkthis message to see if you must perform additional operations.

Oracle fails over the database connection and if TYPE=SELECT in theFAILOVER_MODE section of the service name description, Oracle also attemptsto fail over the query.

Continuing work on failed-over cursors can result in an error message

rac implementation 10gr2

Documents