Oracle 9i RAC Demonstrationon CompaqTru64 Unix & TruCluster

clusters for the real world.

gilgugu.yoon@ com paq.com

A lpha S olutions C onsulting Team

Technical C onsulting & P re-sales

M ay, 2002

gilgugu.yoon@ com paq.com

A lpha S olutions C onsulting Team

Technical C onsulting & P re-sales

M ay, 2002

AgendaAgendaAgendaDescribe the Hardware and Software ComponentsDescribe the Hardware and Software Components

Describe Describe SysManSysMan Management Tool Management Tool

Describe SSI (Describe SSI (TruClusterTruCluster Single System Image) Single System Image)

Describe a deployment platformDescribe a deployment platform

Demonstrate that the Oracle Database is RunningDemonstrate that the Oracle Database is Running

Delete a Delete a TruCluster TruCluster MemberMember

Add a Add a TruCluster TruCluster MemberMember

Manage Tru64 UNIX from a Web browserManage Tru64 UNIX from a Web browser

Add a 2nd Oracle9i RAC Instance & Run the 2nd InstanceAdd a 2nd Oracle9i RAC Instance & Run the 2nd Instance

Describe Cache FusionDescribe Cache Fusion

Using the Transparent Application Facility (TAF)Using the Transparent Application Facility (TAF)

Show Show TruclusterTrucluster CFS Direct I/O CFS Direct I/O

Taking Advantage of Parallel QueryTaking Advantage of Parallel Query

AgendaAgendaAgendaBalance the Cluster WorkloadsBalance the Cluster Workloads

Expanding Data Storage On-lineExpanding Data Storage On-line

Describe the H/W and S/W ComponentsDescribe the H/W and S/W ComponentsDescribe the H/W and S/W Components

Fiber ChannelStorage Switch 8 Port

Ethernet

Alias•16.166.193.127•krutcpsds10

Member 1•16.166.193.128•Krutcpsds11(alpha1)

Member 2•16.166.193.129•Krutcpsds12(alpha2)

Domain : kr.cpqcorp.net

MA8000•HSG80 Controller x 2 (4 Ports – 200MB/Sec)• Controller Cache (512MB Cache x 2)• 36.4G Ultra3 SCSI Disk x 18

Cluster Interconnect Mode: Standard Hub ModeMemory Channel II

AlphaServe DS10 * 2EA•512 MB RAM•9 GB Disk• 1 * 466MHz CPU

CCMHB•Linecards 2ea

Software Version• Tru64 UNIX V5.1A• TruCluster Software V5.1A• Oracle 9i RAC 9.0.1.3

Demo H/W & S/W configuration

Windows 2000 16.166.195.180

DISK구성

Member System – 1AlphaServer ES45

Member System – 1AlphaServer ES45

Member System – 2AlphaServer ES45

Member System – 2AlphaServer ES45

Compaq Certified Configurations for Oracle 9iN o-S ingle P oint of Failure

Compaq Certified Configurations for Oracle Compaq Certified Configurations for Oracle 9i9iN o-S ingle P oint of FailureN o-S ingle P oint of Failure

Describe SysMan Management ToolDescribe Describe SysMan SysMan Management ToolManagement Tool

Compaq Tru64 UNIX SysManCompaq Tru64 UNIX Compaq Tru64 UNIX SysManSysMan

Click sysman station icon on laptop– Status_Monitor– AdvFs_Filesystems– Caa_application(active)– Caa_application(all)– Hardware– Mounted filesystems– Physical filesystems

# cfsmgr -v -a server/cluster/members/member2/boot_partition

Compaq Tru64 UNIX SysMan(continued)Compaq Tru64 UNIX Compaq Tru64 UNIX SysManSysMan(continued)(continued)

Right-click the alpha1 server.– SysMan Menu– Caa Management– Daily Administration– Storage Management– Configuration

Add fileset & mount fileset(Double-click the alpha1 -> Storage Management ->AdvFs Utilities)

Describe Single System ImageDescribe Single System ImageDescribe Single System Image

Traditional clusterTraditional clusterTraditional cluster

Node 1 Node 2 Node 3 Node n

Cluster/Root, /Usr,/Var

OracleExecutables

Oracle Data Files(Raw)

Cluster/Root, /Usr,/Var

OracleExecutables

Cluster/Root, /Usr,/Var

OracleExecutables

Cluster/Root, /Usr,/Var

OracleExecutables

Compaq(Integrated) clusterCompaq(Integrated) clusterCompaq(Integrated) cluster

Node 1 Node 2 Node 3 Node n

Cluster/Root, /Usr, /Var

OracleExecutables

Oracle Data Files(AdvFsFile System)

# cp ls /sbin/LS; chmod 755 /sbin/LS

Run LS on each member after executing uname–a

# rm /sbin/LS

Show CDSL by “ls –l /etc/sysconfigtab”

Choosing a deployment platformChoosing a deployment platformChoosing a deployment platform

Choosing A Deployment PlatformChoosing A Deployment Platform

A Single SMP

Scales to multipleCPUs

Doesn’t scalebeyond one node

Multiple singlepoints of failure

Users

Users

Choosing A Deployment PlatformChoosing A Deployment Platform

Failover Clustering

Fault tolerantsystems; highlyavailable

Doesn’t scalebeyond one node

Grow your DATA

Grow your USERS

Grow PROCESSINGPOWER

Choosing A Deployment Platform : OracleChoosing A Deployment Platform : OracleParallel ServerParallel Server

Users

Grow your DATA

Grow your USERS

Grow PROCESSINGPOWER

Shared CacheShared Cache

Cache FusionCache Fusion

Choosing A Deployment Platform : RealChoosing A Deployment Platform : RealApplication ClustersApplication Clusters

Users

Cache Fusion:Performance of aSHARED CACHE

Demonstrate That the Oracle Database isRunning

Demonstrate That the Oracle Database isDemonstrate That the Oracle Database isRunningRunning

Starting pointStarting pointStarting point

Start with a two-node cluster with both systemsbooted and logged-in at the root prompt.

Start cluiomon to show “direct access” or, asoracle calls it, “concurrent” I/O.

Oracle Instances shutdown.

# vmubc –cpu on each members.

# xterm&# cluiomon

Start listener on alpha1

Start the Database on alpha1

Start SQL>@inst

Start SQL>@tablespaces

Start SQL>@files

Start SQL>@q1

Show connection to listener– # netstat –na|grep EST|grep 1521 on alpha1

– Connect alpha1 with sqlplus on laptop

– # netstat –na|grep EST|grep 1521 on alpha1

Explain cluster alias function on the level of OS.– # telnet alpha without login repeatedly

Delete a TruCluster MemberDelete a Delete a TruClusterTruCluster Member Member

On the laptop, open a terminal window, connectto alpha1 using telnet over the TCP/IP network.

Delete cluster member 2.(enter no at the last question)# clu_delete_member [CTRL/C]

5 minute is enough to delete one node.

Add a TruCluster MemberAdd a Add a TruClusterTruCluster Member Member

On the laptop, open a terminal window, connectto alpha1 using telnet over the TCP/IP network.

add cluster member 2.(enter no at the last question)# clu_add_member [CTRL/C]

>> boot –fi genvmunix

Manage Tru64 Unix from a Web BrowerManage Tru64 Unix from a WebManage Tru64 Unix from a Web Brower Brower

Compaq Web-Based ManagementCompaq Web-Based ManagementCompaq Web-Based Management

Can be done from any browerCompaq Insight Management agents are runningon Tru64 UNIX.(insightd, cpq_mibs, os_midbs)System status reports can be views from the PC.Default function of OS

Start the Web Brower ofURL http://alpha1:2301/ on laptop.

Compaq Web-Based Management AgentCompaq Web-Based Management AgentCompaq Web-Based Management Agent

Click Software version info(same to #sysconfig –q on the server)

System info

Click system board

Click File System space used

Tu0 Ethernet interface

Power on messages

Add a 2nd Oracle9i RAC Instance& Run the 2nd Instance

Add a 2nd Oracle9i RAC InstanceAdd a 2nd Oracle9i RAC Instance& Run the 2& Run the 2ndnd Instance Instance

Start listener on 2nd member

SQL> @inst

SQL> @tablespaces

SQL> @files; point out same file on the clustered system

SQL> @q3

Describe Cache FusionDescribe Cache FusionDescribe Cache Fusion

Cache Fusion ArchitectureCache Fusion Architecture

Full Cache Fusion– Cache-to-cache data

shipping

– Shared cacheeliminatesslow I/O

– Enhanced IPC

Allows flexibleand transparentdeployment

Users

Shared CacheShared Cache

Cache FusionCache Fusion

Full Cache FusionFull Cache Fusion

DatabaseDatabase

Node A

Database buffers

Node B

Database buffers

RequestRequestData TransferData Transfer

Oracle9i Cache Fusion increases performance andscalability

– Data is shipped directly over high speed interconnect

– Minimize disk I/O

Cache Fusion ManagesCache Fusion ManagesInter Instance Block RequestsInter Instance Block Requests

Readers and writersaccessing instance Again access to blocksin instance B’s buffercacheAll types of blockcontention andaccessCoordination byGCS(Global CacheService) /GES(Global EnqueueService)

ReadReadRead

Request for BlockCache A

RequestRequest for Block for BlockCache ACache A

ReadReadRead

WriteWriteWrite

WriteWriteWrite

Lock StatusLock StatusBlock inBlock inCache BCache B

ReadReadRead

ReadReadRead

WriteWriteWrite

WriteWriteWrite

8i : 9iRAC == 1 : 3 (update), 1 : 4 (insert), 1.5 : 1 (delete)

Scalability:Scalability:Cache Cache FusionFusionTMTM Benefit Benefit

20 msec

.01 msec

100 msec

Block inLocal Cache

Block in RemoteCache

Block onDisk

1 msec

100x100xFasterFaster

Time to write datablock to disk

When block needed is in:

Restart the two instances to clean cacheStart the 1st instanceStart the 2nd instanceSQL> @q3Point out that this is the first time you have runthis query- which makes the SGA a cold cache.And after the initial lock sync tests between thetwo instances, all activity is on alpha1 cpu andthe alph2 cpu is idle.

Transparent Application Facility (TAF)Transparent Application Facility (TAF)Transparent Application Facility (TAF)

Transparent Application Failover(TAF)– Applications and users are automatically and

transparently reconnected to another system.

– Applications and queries continue uninterrupted.

– DML transactions are rolled back.

Connect alpha1 with db9i1.taf on laptop

Run SQL> @d:/oraclekits/Oracle/files/inst

Run SQL> @d:/oraclekits/Oracle/files/files

Run SQL> @d:/oraclekits/Oracle/files/q1

Run SQL> shutdown abort ; on alpha1

Check where the failover from alpha1 to alpha2

Show Trucluster CFS Direct I/OShow Show TruclusterTrucluster CFS Direct I/O CFS Direct I/O

CFS traditional file I/OCFS traditional file I/OCFS traditional file I/O

ClusterInterconnect

HSG80HSG80FC Switch

CFS Server

Cache

CFS Client

Cache

CFS Client

Cache

All file system I/O is served and cachedAll file system I/O is served and cached

CFS Concurrent & Cached Read without direct I/O openflag (TCS 5.1A)CFS Concurrent & Cached Read without direct I/O openCFS Concurrent & Cached Read without direct I/O openflag (TCS 5.1A)flag (TCS 5.1A)

ClusterInterconnect

HSG80HSG80FC Switch

CFS Server

Cache

CFS Client

Cache

CFS Client

Cache

Taking Advantage of Parallel QueryTaking Advantage of Parallel QueryTaking Advantage of Parallel Query

Get more servers and CPUs involved.

When the database was set up, it did not includeautomatically parallelizing queries.This is donewith the client requests.

Compaq Tru64 Unix is the only O/S that supportDAIO on “Cooked” (AdvFS) filesystem.All other Unix must use raw devices to be able toprovide DAIO(=concurrent I/O)

Start vmubc on each member

Start the cluiomon on alpha1

Connect oe/oe@alpha1 on laptop

SQL>@inst SQL>@q4 SQL>@q4point out the 2nd query is short because thecache was now warm. And alpha1 isactive,alpha2 is idle.

Parallel queryParallel queryParallel query

Start vmubc on each member

Start the cluiomon on alpha1

Connect oe/oe@alpha1 on laptop

SQL>@q5 SQL>@q5point out that the Compaq Tru64 UNIX is onlyO/S for DAIO or direct-access I/O. It’s by AdvFSfile system.

Balance the Cluster WorkloadsBalance the Cluster WorkloadsBalance the Cluster Workloads

The purpose of this demonstration is to showload balancing through the db9i.lb entry in thetnsnames.ora

– By using Oracle Enterprise Manager

– By using Oracle Workload Generator(a demotool)

Procedure– Start Oracle Enterprise Manager Console

– Login to each member as system/manager

– Show sessions on each members

– Start vmubc on each members

– Start workload generator on laptop

Expanding Data Storage On-lineExpanding Data Storage On-lineExpanding Data Storage On-line