CXtec Migration from Large SMP to Oracle RAC Cluster on Linux

Dean Bettinger

Storage Technology

Business Development Manager

CXtec Migration from Large SMP to Oracle RAC Cluster on Linux

o World-wide VAR for Network, Voice, and Data Center equipment – new and certified pre-owned.

o equal2new®: 10-step certification process, 99.51% reliability rating and lifetime warranty

o CABLExpress® cables, including fiber jumpers. Also new and equal2new® Fibre Channel switches & HBAs.

o $135M annual sales, 6 consecutive years on VAR 500 (#231), ISO 9001:2000 certified.

o 300+ employees.o Great Place To Work – last 4 years in top 25.

CXtec Company Overview

o ERP o Oracle Applications eBusiness Suite 11.5.9o Sun E4500, 12 x 400MHz CPU, 12G RAMo Oracle Database 9i, named user licenseo Running out of CPU cycles and RAM

o Web Siteo Sun 450, 400MHz CPU, 4G RAM for our web

siteo Oracle Database 9i, per CPU license,

unlimited userso Day old copy of production data replicated

from ERP

Where We Were - 2004

o Business desire to bring live transaction data to our web site, especially to our Extranet for customers

o Business / IT desire to leverage Oracle for everything

o IT developer desire to simplify our development environmento Avoid complexity of live synchronization

between inside & outside databaseso Cease daily replication of day-old data to

Extraneto Utilize web services from Extranet to access

up to the moment live data

Challenges

o Migrate internal Oracle database licensing from named user to per CPU, unlimited userso Can now serve 5,000+ Extranet users

directly. Avoided $800 per user -- $4Mo Can now leverage Oracle database for all

application development: ERP, Extranet, Intranet for all employees, not just ERP users

o Eliminate Oracle database from our DMZo Corporate web site and Extranet will now use

web services to get data from the one true source

Database Licensing Migration

o Oracle charges $40K per CPU licenseo Sun CPUs were 900MHz max and we figured we

needed 12 CPUs -- $480Ko New Sun enterprise-class server -- $350Ko Sun SPARC and Sun Solaris – out the window!

o CXtec had been a long-time committed customer of Sun, and we loved its reliability

o Solaris was favored operating system due to features and stability, but Sun had abandoned Solaris x86.

Ouch!

o In Intel we trust!o Scale CPU speeds faster than CXtec growso 3GHz Xeon much faster than 900MHz SPARCo Desire to move to commodity 2 socket

serverso Goodbye Sun, Dell here we come!

o Decision point – How do we get to 12GHz?o Single Quad socket server?

o Simple, but hardware more expensiveo Two Dual socket servers in a RAC cluster?

o Complex, but commodity servers, resiliency

The Way Forward

o Raw device mapping?o Not visible to SysAdmins, complexity borne by

DBAo Oracle ASM not yet available

o Cluster file system?o Oracle Cluster File System – OCFS1

o Not ready for prime timeo Sistina GFS (now a Red Hat product)

o Single node lock managero PolyServe Matrix Server (now an HP product)

o Distributed lock manager, BCF reference

Mitigating RAC Complexity

o We settled on PolyServe’s Database Utility for RACo Common mount points on all servers

o SysAdmin friendlyo Simple for DBAo Shared Oracle HOME

o Distributed Lock Managero No single point of failure

o Context Sensitive Symbolic Linkso Oracle Disk Manager (ODM) integration

PolyServe Matrix Server

Database Server9iRAC

Dababase Server9iRAC

Oracle Apps9iAS

Web Apps9iAS Java Edition

DiscovererAS 10g Enterprise Edition

Brocade 2800Fibre Channel Switch

Brocade 2800 Fibre Channel Switch

Xiotech Magnitude 3D

Cisco 3508 GbECluster Interconnect

Core Network Stack – Nortel 5510 switches

Corporate LAN – Nortel 450 switches

`

Middle Tier

Servers

EndUsers` `

Oracle RAC

Cluster

1st GenerationRAC Cluster

2nd GenerationRAC Cluster

Database Server10gRAC Dababase Server

10gRAC

Oracle AppsAS 10g

Web AppsAS 10g Java Edition

DiscovererAS 10g Enterprise Edition

Qlogic 5200Fibre Channel Switch

Qlogic 5200Fibre Channel Switch

Corporate LAN – Cisco 6509 Core switch

`

Middle Tier

Servers

EndUsers` `

Oracle RAC

Cluster

Compellent Storage Center

Cisco SFS 300110G InfiniBand

Switch

Cisco SFS 300110G InfiniBand

Switch

ClusterInterconnect

o Improved Performanceo Batch job take minutes instead of hourso Dmart loading window dramatically reduced

o Improved End-User Productivityo Most application response now sub-second

o Improved IT Productivityo Single database for all applications. Simple!

o Live Extranet information for Customerso Enhanced Reliability

o Run on 2 nodes, 3rd standby node

Results

o Startup Costs - $137Ko $20K for 3 Dell 2 socket 3GHz Xeon servers,

8GBo $24K for PolyServe Matrix Filesystem, DBUo $93K Oracle License Migration to RAC

o Annual Maintenance Commitmento +$16K / year

Economic Analysis

o Cost Avoidance - $670Ko $350K for a new Sun Enterprise-Class SMP

servero $320K in Oracle CPU licensing for slow Sun

CPUs

o Annual Maintenance Cost Avoidanceo $112K on Sun Server and Oracle Licenses

o Bottom Lineo $1,017,075 Cumulative 5 Year Net Benefito ROI 473%, Payback Period – 2 Months

Economic Analysis

o Cisco / TopSpin SFS 3001 InfiniBand switcheso 12 ports of 10G Full-Duplex InfiniBando 2 port of 2G Fibre Channel

o For dual pathing to two Fibre Channel switches

o Cisco HCA (Host Channel Adapter) cards for serverso 2 ports of 10G Full-Duplex InfiniBando For dual pathing to two InfiniBand switches

o 10x bit rate of Gigabit Etherneto 1/10th latency of Gigabit Etherneto 1/10th CPU utilization compared to Gigabit

Ethernet

InfiniBand Cluster Interconnect

o Storage Virtualization on steroidso Boot from SANo Snapshots and Viewso Enterprise Managero Thin Provisioningo Automated Tiered Storageo Fast Tracko Thin Importo Windows File Space Recovery

Compellent Storage Center

o ???

Questions & Answers