0, download the following software. · # install the following package from the oracle grid media....

This note describes the installation of Oracle Database 11g Release 2 (11.2.0.1 64bit) RAC on

Oracle Enterprise Linux 5 (64bit) using VMware Server 2 with no additional shared disk devices.

After reading articles on installation of Oracle 11g Release 2 RAC on Oracle Enterprise Linux on

VMware, I narrowed down to choose Oracle Enterprise Linux as operating system and VMware

Server 2 as the virtual machine type. Some people say that 11g R2 has more memory

requirement than 11g R1 and earlier version. R2 requires at least 1.5GB for each node. Luckily I

have a computer with 6GB. I started with 2GB on each node with two nodes. It crashed on me

for 3 or 4 times either on Grid Infrastructure installation or database installations. Then I

increased the memory to 2.2GB for each node and I finally succeed in installations. One advice

is that you have to backup your virtual machine before you go to next step, which will save you

a lot of time. The beauty part of the R2 is that when you install grid infrastructure, you define in

the installer both the ASM and clusterware configuration and it installs both. I did SQL Server

2008 R2 Cluster installation with VMware Workstation 7, I wanted to use it for Oracle RAC, but it

did not work. So I have to install on VMware Server 2 instead.

0, Download the following software.

VMware Server 2.0.2

Oracle Enterprise Linux 5

Oracle 11g Release 2 Software (64bit)

Hardware Information:

Dell Inspiron 570 AMD Athlon II x3 435 Processor 2.9 GHz

Installed memory (RAM): 6.00 GB

Hard disk: 500GB

System type: 64-bit Operating System

Operating System: Windows 7 Home Premium

1, Virtual Machine Setup

Click “Create Virtual Machine” menu option

Enter the name “RAC1” for the Name and accept the standard datastore by clicking “Next”

button.

Select “Linux Operating System” and set the version to “Red Hat Enterprise Linux 5(64bit)”

Enter the required amount of memory of 2048MB

Set the disk size to “25GB” and click “Next”

Click “Add Network Adapter”

Select “Bridged” option and click “Next” button

Click the “Use a Physical Drive” link, or click the “Next” button.

Accept the DVD properties by clicking the “next” button.

Click the “Add a USB Controller” link, or click the “Next” button.

Click the “Finish” button to create the virtual machine.

Highlight the "RAC1" VM in the "Inventory" pane, then click the "Add Hardware" link in the

"Commands" section to the right.

Click the “Network Adapter” link

Click the “Finish” button.

2, Guest Operating System Installation

Click “Power On” button to start operating system installation.

Click “Skip” to skip media test.

Click “Next”

Leave default as it is and click “Next”

Choose eth0 and click “Edit”

Input IP address as below:

Choose eth1 and click “Edit” and input I.P address as below:

Input hostname as below:

Leave default as it is and click “Next”

Input password for root user.

Choose all of the choices below and click “Next”

Click “Next” to begin installation

It shows that you need to get 6 installation CDs ready and click “Continue”

The installation process may take a while to finish, in the meantime you need to change CDs of

image files from 1 to 6 one by one.

Click “Reboot” when the installation finishes.

Disable Firewall

Disable SELinux

Leave Enable kdump unchecked and click “Forward”

Set date and time here

You may create user or skip this step

Ignore this step and click “Forward”

Click “Finish” and it will reboot again.

3, Post Installtion Steps

To be consistent with the rest of the article, the following information should be set during the

installation process.

127.0.0.1 localhost.localdomain localhost

# Public

192.168.1.101 rac1.localdomain rac1


# Private

192.168.0.101 rac1-priv.localdomain rac1-priv


# Virtual

192.168.1.111 rac1-vip.localdomain rac1-vip


# SCAN

192.168.1.113 rac-scan.localdomain rac-scan

Once the basic installation is complete, install the following packages while logging as root user.

This includes the 32 bit and 64 bit version of some packages.

# From Enterprise Linux 5 DVD

cd /path/to/media/cdrom/Server

rpm -Uvh binutils-2.*

rpm -Uvh compat-libstdc++-33*

rpm -Uvh elfutils-libelf-0.*

rpm -Uvh elfutils-libelf-devel-*

rpm -Uvh gcc-4.*

rpm -Uvh gcc-c++-4.*

rpm -Uvh glibc-2.*

rpm -Uvh glibc-common-2.*

rpm -Uvh glibc-devel-2.*

rpm -Uvh glibc-headers-2.*

rpm -Uvh ksh-2*

rpm -Uvh libaio-0.*

rpm -Uvh libaio-devel-0.*

rpm -Uvh libgcc-4.*

rpm -Uvh libstdc++-4.*

rpm -Uvh libstdc++-devel-4.*

rpm -Uvh make-3.*

rpm -Uvh sysstat-7.*

rpm -Uvh unixODBC-2.*

rpm -Uvh unixODBC-devel-2.*

# For OEL, use relevant versions of the following packages from your media.

rpm -Uvh oracleasm-2.6.18-194.el5-2.0.5-1.el5.x86_64.rpm \

oracleasm-support-2.1.3-1.el5.x86_64.rpm

# Install the following package from the Oracle grid media.

Make sure the shared memory filesystem is big enough for Automatic Memory Manager to

work.

# umount tmpfs

# mount -t tmpfs shmfs -o size=3000m /dev/shm

Make the setting permanent by amending the "tmpfs" setting of the "/etc/fstab" file to look like

this.

tmpfs /dev/shm tmpfs size=3000m 0 0

If you are not using DNS, the "/etc/hosts" file must contain the following information.

127.0.0.1 localhost.localdomain localhost

# Public



# Private



# Virtual



# SCAN

192.168.1.113 rac-scan.localdomain rac-scan

Add or amend the following lines to the "/etc/sysctl.conf" file.

fs.aio-max-nr = 1048576

fs.file-max = 6815744

kernel.shmall = 2097152

kernel.shmmax = 1054504960

kernel.shmmni = 4096

# semaphores: semmsl, semmns, semopm, semmni

kernel.sem = 250 32000 100 128

net.ipv4.ip_local_port_range = 9000 65500

net.core.rmem_default=262144

net.core.rmem_max=4194304

net.core.wmem_default=262144

net.core.wmem_max=1048586

Run the following command to change the current kernel parameters.

/sbin/sysctl –p

Add the following lines to the "/etc/security/limits.conf" file.

oracle soft nproc 2047

oracle hard nproc 16384

oracle soft nofile 1024

oracle hard nofile 65536

Add the following lines to the "/etc/pam.d/login" file, if it does not already exist.

session required pam_limits.soDisable secure linux by editing the "/etc/selinux/config" file,

making sure the SELINUX flag is set as follows.

SELINUX=disabledAlternatively, this alteration can be done using the GUI tool (System >

Administration > Security Level and Firewall). Click on the SELinux tab and disable the feature.

Either configure NTP, or make sure it is not configured so the Oracle Cluster Time

Synchronization Service (ctssd) can synchronize the times of the RAC nodes. In this case we

will deconfigure NTP.

# service ntpd stop

Shutting down ntpd: [ OK ]

# chkconfig ntpd off

# mv /etc/ntp.conf /etc/ntp.conf.org

# rm /var/run/ntpd.pidIf you are using NTP, you must add the "-x" option into the following line in

the "/etc/sysconfig/ntpd" file.

OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid"Then restart NTP.

# service ntpd restart

Create the new groups and users.

groupadd -g 1000 oinstall

groupadd -g 1200 dba

useradd -u 1100 -g oinstall -G dba oracle

passwd oracle

Create the directories in which the Oracle software will be installed.

mkdir -p /u01/app/11.2.0/grid

mkdir -p /u01/app/oracle/product/11.2.0/db_1

chown -R oracle:oinstall /u01

chmod -R 775 /u01/

Login as the oracle user and add the following lines at the end of the .bash_profile file.

# Oracle Settings

TMP=/tmp; export TMP

TMPDIR=$TMP; export TMPDIR

ORACLE_HOSTNAME=rac1.localdomain; export ORACLE_HOSTNAME

ORACLE_UNQNAME=RAC; export ORACLE_UNQNAME

ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE

ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1; export ORACLE_HOME

ORACLE_SID=RAC1; export ORACLE_SID

ORACLE_TERM=xterm; export ORACLE_TERM

PATH=/usr/sbin:$PATH; export PATH

PATH=$ORACLE_HOME/bin:$PATH; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib;

export CLASSPATH

if [ $USER = "oracle" ]; then

if [ $SHELL = "/bin/ksh" ]; then

ulimit -p 16384

ulimit -n 65536

else

ulimit -u 16384 -n 65536

fi

fi

Install VMware Client Tools

Next, run the "vmware-config-tools.pl" script as the root user.

# vmware-config-tools.pl

Accept all the default settings and pick the screen resolution of your choice. Ignore any

warnings or errors. The VMware client tools are now installed.

4, Create Shared Disks

Shut down the RAC1 virtual machine. Make a directory on the host system to hold the shared

virtual disks.

On the VMware Intrastructure Web Access Console, click the "Add Hardware" link.

Click on “Hard Disk”

Click on “Create a New Virtual Disk”

Choose 8gb as Capacity and Choose location of your choice from your PC.

Check “Independent – Persistent” as Disk Mode

Click Finish to create the disk.

Repeat the previous hard disk creation steps 4 more times, using the following values:

File Name: [standard] shared/asm2.vmdk Virtual Device Node: SCSI 1:2 Mode: Independent and Persistent




At the end of the process, the virtual machine should look like this:

Edit the contents of the "/Path/To/RAC1/RAC1.vmx" file using a text editor, making sure the

following entries are present. Some of the tries will already be present, some will not.

disk.locking = "FALSE"

diskLib.dataCacheMaxSize = "0"

diskLib.dataCacheMaxReadAheadSize = "0"

diskLib.dataCacheMinReadAheadSize = "0"

diskLib.dataCachePageSize = "4096"

diskLib.maxUnsyncedWrites = "0"

scsi1.present = "TRUE"

scsi1.sharedBus = "VIRTUAL"

scsi1.virtualDev = "lsilogic"

scsi1:1.present = "TRUE"

scsi1:1.fileName = "/u01/VM/shared/asm1.vmdk"

scsi1:1.writeThrough = "TRUE"

scsi1:1.mode = "independent-persistent"

scsi1:1.deviceType = "plainDisk"

scsi1:1.redo = ""






scsi1:2.redo = ""






scsi1:3.redo = ""






scsi1:4.redo = ""






scsi1:5.redo = ""

Start the RAC1 virtual machine by clicking the "Play" button on the toolbar, then start the

console as before. When the server has started, log in as the root user so you can partition the

disks. The current disks can be seen by issuing the following commands.

Use the "fdisk" command to partition the disks sdb to sdf. The following output shows the

expected fdisk output for the sdb disk.

In each case, the sequence of answers is "n", "p", "1", "Return", "Return", "p" and "w".

Once all the disks are partitioned, the results can be seen by repeating the previous "ls"

command.

[root@rac1 dev]# ls sd*

sda sda1 sda2 sdb sdb1 sdc sdc1 sdd sdd1 sde sde1 sdf sdf1

[root@rac1 dev]#

Configure ASMLib using the following command.

Load the kernel module using the following command.

Mark the five shared disks as follows.

s

You can also use the following two commands to scan and list the disks.

5, Clone the Virtual Machine

Shutdown RAC1 and make a copy as RAC2

Start RAC2 to make sure that I.P adresses and hostname are configured as directed above and

make sure that it can ping each other.

Prior to 11gR2 we would probably use the "runcluvfy.sh" utility in the clusterware root directory

to check the prerequisites have been met. If you are intending to configure SSH connectivity

using the installer this check should be omitted as it will always fail. If you want to setup SSH

connectivity manually, then once it is done you can run the "runcluvfy.sh" with the following

command.

/u01/software/grid/runcluvfy.sh stage -pre crsinst -n rac1,rac2 –verbose

If you get any failures be sure to correct them before proceeding.

See here if you don’t know how to resolve user equivalency issues:

http://download.oracle.com/docs/cd/B19306_01/install.102/b14205/presolar.htm#BABIDAIH

Edit the /home/oracle/.bash_profile file on the RAC2 node to correct the ORACLE_SID and

ORACLE_HOSTNAME values.

ORACLE_SID=RAC2; export ORACLE_SID

ORACLE_HOSTNAME=rac2.localdomain; export ORACLE_HOSTNAME

It is good idea to make a backup of both RAC1 and RAC2 before we continue.

6, Install the Grid Infrastructure

Click the "SSH Connectivity..." button and enter the password for the "oracle" user. Click the

"Setup" button to to configure SSH connectivity, and the "Test" button to test it once it is

complete.

Click the "Identify network interfaces..." button and check the public and private networks are

specified correctly. Once you are happy with them, click the "OK" button and the "Next" button

on the previous screen.

Set the redundancy to "External", select all 5 disks and click the "Next" button.

Specify the path for Inventory Directory

If you are happy with the summary information, click the "Finish" button.

Wait while the setup takes place.

The output from the "orainstRoot.sh" file should look something like that listed below.

Output of root.sh from RAC1

Output of root.sh from RAC2:

Once the scripts have completed, return to the "Execute Configuration Scripts" screen on RAC1

and click the "OK" button.

Wait for the configuration assistants to complete.

The Configure Oracle Grid Infrastructure for a Cluster and Oracle Cluster Verification Utility

failed in this installation. I choose Skip to continue and then click the "Close" button to exit the

installer.

It is a good idea to make a backup before proceeding to the next step.

7, Install the Database

Before starting installer, I run the runcluvfy.sh script to make sure that all verifications are

passed as listed to the end of the Step 5.

This time, the parameter should be:

./runcluvfy.sh stage –pre dbinst –n rac1,rac2 –verbose

After all requirements are met, start installer.

Uncheck the security updates checkbox and click the "Next" button.

Choose Create and Configure a database

Accept the "Server Class" option by clicking the "Next" button.

Make sure both nodes are selected, then click the "Next" button.

Choose Typical install and continue

Enter "/u01/app/oracle/product/11.2.0/db_1" for the software location. The storage type should

be set to "Automatic Storage Management". Enter the appropriate passwords and database

name, in this case "RAC.localdomain".

If you are happy with the summary information, click the "Finish" button.

Wait for the installation to finish. It may take 1.5 hours to finish depending on the speed of your

computer.

Wait for DBCA to start

Run root.sh script on RAC1 and RAC2 separately.

If you have configured Enterprise Manager, you will see the URL on this screen. Write down the

URL and click Close to finish the database installations.

8, Check the Status of the RAC

There are several ways to check the status of the RAC. You can use sqlplus tool to query the

database view and The srvctl utility shows the current configuration and status of the RAC

database.

If you have configured Enterprise Manager, it can be used to view the configuration and current

status of the database using a URL like "https://rac1.localdomain:1158/em".

Now the installation of Oracle 11g Release 2 RAC is complete. Hope it helps.

0, download the following software. · # install the following package from the oracle grid media....

Documents