database cloud server - · pdf fileoracle database servers, and are not limited to the oda....

Database Cloud Server Introduction and Technical Overview

Vendita February 2017 DCS-ito-1.02


DCS-ito-1.02 © 2017 Vendita, All Rights Reserved

Abstract

This paper introduces the Vendita Database Cloud Server, a product that features Oracle Database software, pre-installed and pre-configured on IBM® Power Systems™ servers with POWER8™ processors.

It is the first product of its kind for IBM Power® Systems hardware, and builds on the success of other appliance offerings for Oracle Database.

In this document, the features of the Vendita Database Cloud Server are presented, along with the rationale behind the selection of various technologies that comprise the Database Cloud Server.

Audience

The intended audience for this document is technical professionals who have a working understanding of database server software and concepts, server hardware and server operating systems.

Resources

A comprehensive glossary is provided at the end of this document to define terms and concepts covered. These terms and concepts are industry standard, and cover both database software and servers, as well as server hardware and operating systems.



Introduction

In its most simplistic form, the Database Cloud Server consists of Oracle Database running on an IBM Power Systems server, where the operating system is IBM AIX® Unix®.

The Database Cloud Server uses IBM hardware virtualization, known as PowerVM™, allowing multiple servers to be created or instantiated on a single frame (Power Systems server). The Database Cloud Server features technologies from both IBM and Oracle Corporation. When designing an appliance to run Oracle Database using IBM hardware, there are technologies available from both companies that provide the capabilities needed in a database appliance, requiring technical decisions to be made. In this document, these technical decisions are explored and explained in detail. The Database Cloud Server was developed to provide similar deliverables and capabilities available from Oracle Engineered Systems, but at reduced cost and complexity.

Oracle Engineered Systems and most database servers are built on “classic,” commonly used technical architectures. The Database Cloud Server features a high performance technical architecture that surpasses these architectures by providing a set of deliverables that are unattainable from database servers based on classic technical architectures. These deliverables include superior capability within the areas of scalability, reliability and cost containment.

In this document, we will review each of these deliverables and discuss how they are achieved.



Review of Typical Database Server Architectures

In this section, several typical architectures of database servers will be presented.

These architectures will be presented with logical diagrams that depict

items existing in the operating system domain, as well as items that exist in the server hardware domain.

The purpose of this review is to present commonly used database servers

in terms of features, strengths and weaknesses.

Architecture 1 - Oracle Database Server with OS File System

Figure 1 (below) represents the typical architecture for database servers running Oracle Database (RDBMS). This architecture is the simplest architecture possible for a database server, and uses a file system provided by the operating system to store database files.

Operating system-based files systems are sometimes called a “cooked file system”. There are components that may not be clear in this diagram: NIC and SAS adapter. NIC stands for Network Interface Card, and is used for the server to connect to a local area network. The SAS adapter provides connectivity to the internal storage.

This architecture uses a file system managed by the operating system. Typical file systems used are journaled file system (JFS), ext3, xfs and NTFS.



Figure 1 – Basic Oracle Database server architecture

There are many weaknesses with the above architecture related to

reliability, scalability and cost.

Scalability Issues

To increase physical memory, more or larger memory modules must be

installed. Adding additional CPU capacity with this architecture is nearly impossible, since it would require changing the CPU. Adding internal disks to the server can increase storage capacity. This requires opening the server and installing additional disks, in addition to requiring an outage for installation.

Stability

The above hardware architecture contains many single points of failure,

especially with regards to the NIC and SAS adapter. Often this architecture is deployed on lower cost, commodity hardware that is more prone to failure.



Cost Containment

The above hardware architecture can initially be inexpensive to deploy,

because commodity hardware can be used. However, because it is not easily scaled, Oracle customers tend to

oversize hardware to avoid performance issues. This often results in a system with low overall utilization, which further lowers the value attained from investments made in Oracle Database licenses.

Architecture 2: Oracle Database Appliance

The Oracle Database Appliance (ODA) is a low cost, appliance-based database server with three configurations available. No further customization is possible from these three configurations.

The architecture of ODA is shown in figure 2. The architecture consists of

two servers accessing shared storage. The architecture provides a number of improvements on architecture 1, reviewed in the previous section, by leveraging Oracle technologies.



Figure 2 – Oracle Database Appliance architecture

In the next sections we review two pivotal Oracle technologies utilized in

the ODA. These technologies are widely used in architectures for running Oracle Database servers, and are not limited to the ODA.

Oracle Automatic Storage Management

Oracle Automatic Storage Management, commonly abbreviated to ASM,

is a file system designed specifically for Oracle Database files, and replaces



the operating system “cooked” file system. ASM provides a number of capabilities that are vast improvements over operating-based file systems. ASM has a block level structure designed for database files, and more efficiently processes read and write requests from the database.

ASM provides double or triple mirroring without compromising performance, in addition to facilitating superior workload balancing over multiple physical volumes.

One of the most important features of ASM is clustered file system

capability. A clustered file system is needed to support clustering at the database level. This clustering capability is covered in the next section. Note that ASM can be used without Oracle Real Application Clusters, and this is recommended for all databases regardless of whether it is used.

Oracle Real Applications Clusters

Oracle Real Application Clusters, commonly referred to as Oracle RAC,

allows the creation of a clustered database. An ASM clustered file system is used to access a shared database that is visible to all node servers on the database cluster. The ODA uses Oracle RAC to create a clustered database server using two servers. This provides High Availability (HA) capability, allowing for one of the two servers to fail or be taken offline for patching while the surviving server continues to run and service database users.

ODA Installation and Setup

Oracle Grid Infrastructure is the Oracle software product that provides

ASM and Oracle RAC. It must be installed separate from the Oracle Database software. Oracle Grid Infrastructure can be installed to provide ASM only, and this is a common practice.

The ODA is provisioned from a single software bundle that includes the Oracle Grid Infrastructure and Oracle Database software. This installation of the software bundle is simple enough that most Oracle Database administrators can perform this task. Aside from installing the software bundle, no further involvement is need by information technology personnel, besides network engineers to assign network addresses to the ODA.

Having reviewed the most relevant aspects of the ODA, an examination of core deliverables is provided in the next three sections.



Scalability

The ODA enables scalability by generally providing more processor cores

than most customers would ever need to use, with 36 total Intel® cores. However, to fully utilize more than 18 cores for a single database, Oracle RAC must be licensed. The cost of licensing is $23K per CPU, or $11.5K per CPU core. With this cost, to fully license Oracle RAC alone on the ODA would cost $414K. The three different configurations available for the ODA are all related to memory capacity. The ODA features 256G of memory per server, and can be expanded to 512G or 768G per server. The ODA supports the addition of a storage expansion shelf that allows storage to be increased as needed. The storage expansion shelf must be purchased separately.

The ODA provisioning bundle includes only the most current version of Oracle Database software. If ODA users must use older versions of Oracle Database software, then those versions must be installed after the provisioning process is completed. This is a limiting factor of the ODA as, in this instance, the installation can be arduous, especially if Oracle RAC and ASM are to be used.

As a database server, the ODA has two operational modes. It can function as two stand-alone enterprise database servers, or one clustered database server. Beyond those two modes, no additional servers can be provisioned on the ODA. The disadvantages of this are many-fold. No isolation between servers for different databases can be achieved. If one database becomes oversubscribed, its performance problems could impact the performance of other databases. Operating system patches and customizations will impact all databases on the ODA, since they all run on a common server. By having all databases on a single server, security risks increase, since a DBA or systems administrator who performs administrative work on the ODA will have access to all of its databases.

Stability

As was previously mentioned, the ODA has two operational modes, as two stand-alone enterprise database servers, or one clustered database server. As a clustered database server, the ODA contains fewer single points of failure than the architecture in figure 1, and can tolerate the failure of either of the two servers. When the ODA is used as two stand-alone enterprise database servers, these benefits are lost.

One of the benefits of ASM is double or triple mirroring of physical disks. This mirroring is performed in a manner that does not impact storage



performance. Appendix “B” contains a discussion on double versus triple mirroring.

The ODA lacks redundancy with regards to disk controllers used to access its internal storage. If the disk controller fails, then the ODA will experience catastrophic failure.

Cost Containment

The ODA achieves cost containment by limiting the personnel needed for

provisioning. Normally a systems administrator and storage administrator must be involved with provisioning a database server with the complexity of the ODA. Because the ODA can have relatively large amounts of storage, many databases can be consolidated on it.

Since Oracle RAC is required for the ODA to utilize all processor cores, additional licensing costs occur. Moreover, Oracle RAC is an additional layer of technical complexity requiring a different set of skills that DBAs must master. This can require additional training for DBAs, and more effort to support, since there is an additional layer of Oracle technology that must be monitored and patched.

Architecture 3: Oracle RAC with SAN-Based Storage

In this section the architecture used for running Oracle RAC with a storage server is reviewed. This architecture has the same features as the ODA, reviewed in architecture 2, in that Oracle RAC and ASM is used. Figure 3 shows architecture 3.

Note that the architecture features two physical servers. These servers could be either rack-mounted servers, or servers located in a blade center. Moreover, there could be more than two severs in the database server cluster.

From a hardware standpoint, this architecture is a departure from the

ODA, and in the next four sections, these differences are explored.

Private Network for Oracle RAC

In the middle of figure 3 are network interface cards with connectivity to a

private network. Oracle RAC requires the private network for communications between node computers. This network must not be on a shared switch, and must not have any traffic other than inter-node communications for Oracle RAC.



HBA

Hardware Bus Adapter, or HBA, is a card required for a server to

communicate with a SAN. (SANs are described in the next section.) Each server must have at least one HBA card, but often multiple HBA

cards are used to avoid an outage due to a single HBA failure.

SAN

Storage Area Network, or SAN, is a special type of network switch which

the purpose of allowing communications between servers and a storage server. (The SAN is shown in the middle of the architecture).

The majority of SANs are based on the Fibre Channel Protocol. Brocade

is the best known manufacturer of Fibre Channel SAN equipment.

Storage Server

The storage server is a large array of physical disks shared amongst

multiple servers. The storage server allows disks to be added as needs arise for more storage.



Figure 3 – Two-node Oracle RAC with storage server



Storage Server

The storage server provides the storage used for the clustered file system required for Oracle RAC. There are a number of companies who manufacture storage servers, including EMC, NetApp and IBM. These storage servers vary in size and capability, and in some cases are large enough to fill an entire 42U rack. In many circumstances, storage servers are shared among several other storage intensive enterprise platforms, including mail servers, virtual desktop environments and other transaction database servers. An examination of architecture 3 core deliverables is provided in the next three sections.

Scalability

Architecture 3 is highly scalable, allowing for additional Oracle RAC nodes

to be installed if additional processing capacity is required. The ability to add additional Oracle RAC nodes is often referred to as horizontal scaling and has long been promoted by Oracle as one of the benefits of Oracle RAC. However, adding Oracle RAC nodes is not a trivial activity, and most DBAs are not familiar with the steps required to add an additional Oracle RAC node.

Storage can be added to the storage server with relative ease, if additional disk space is required.

Stability

Architecture 3 is inherently stable because of Oracle RAC and a high availability hardware architecture that tolerates the failure of most hardware components without incurring an outage on the database cluster. However, most Oracle RAC installations do not have excess processing capacity. If a node computer fails, the database provides a degraded service level. This is usually due to the high cost of the Oracle Database server and Oracle RAC licenses.

The stability delivered by Oracle RAC is, however, limited. Because architecture 3 uses a shared storage resource, it is vulnerable to performance issues from over-subscription issues on the storage server or SAN. This occurs when an inappropriate number of enterprise applications and users of those applications are using the storage server. The effect of these over-subscription issues is that the read/write rate for the storage server is not capable of supporting all enterprise applications. It is also possible for the SAN to become a bottleneck, if the SAN cannot support the throughput required for all enterprise applications. Diligent and judicious storage administrators can often prevent over-subscription issues with proper management practices.



Cost Containment

Architecture 3 offers no advantage with regards to cost containment. To

support Oracle RAC, a private network is needed, requiring extra network interface cards, as well as a separate network switch. Even more expensive is the hardware needed to support the clustered storage. SANs, HBAs and storage servers are all costly. Moreover, a large number of information technology professionals with different skill sets are required to support this architecture. This includes storage administrators to support the storage server and SAN, network engineers to support to private network switch and systems administrators to maintain each of the database servers.

As previously mentioned, licensing for Oracle RAC is costly and if a

system based architecture 3 needs to be expanded, the cost for additional Oracle RAC licenses and additional hardware can be staggering.



Architecture 4: Oracle Exadata

The Oracle Exadata is the final architecture that will be reviewed in this document. The Oracle Exadata architecture is shown in figure 4. The Oracle Exadata architecture has attributes of architecture 2 and 3, and is essentially a melding of these two architectures. Like architecture 3, the Oracle Exadata can have more than two Oracle RAC nodes. Figure 4 shows a two-node Oracle Exadata architecture to allow for easy reference between previous architectures.

Like the Oracle Database Appliance shown in architecture 2, the Oracle Exadata has dedicated storage that is not on a shared storage server. The Oracle Exadata also uses Oracle ASM and Oracle RAC, similar to the ODA. The Oracle Exadata differs from the ODA by using a storage server with a SAN. This is similar to architecture 3, however the Oracle Exadata does not use Fibre Channel-based SAN equipment. Instead, the Oracle Exadata uses InfiniBand, which is also used for the private network required by Oracle RAC. InfiniBand is a networking technology that has an extremely high throughput at 40Gb/s and ensures that neither SAN throughput nor Oracle RAC private network throughput will impact performance.

The Oracle Exadata uses a proprietary storage server designed specifically for Oracle Database workloads. This storage server was originally designed for decision support system workloads, in that it is very adept at processing queries for large quantities of data. The Oracle Exadata has several different sizes, based on the amount of rack space consumed. These sizes are 1/8th rack, ¼ rack, ½ rack and full rack.



Figure 4 – Oracle Exadata architecture for a two-node system



Having reviewed the most relevant aspects of the architecture 4, an examination of its core deliverables is provided in the next three sections.

Scalability

Like architecture 3, the Oracle Exadata is highly scalable, allowing for

additional processing capability to be brought online as needed and providing horizontal scaling capability. Recall that adding Oracle RAC nodes to architecture 3 is a daunting task.

On the Oracle Exadata, the addition of Oracle RAC nodes is relatively easy, since the hardware needed for this expansion is already installed and available. Storage can be added to the storage server with relative ease, if additional disk space is required. The Oracle Exadata is essentially a closed platform with regard to hardware additions, so there is no option to add a Fibre Channel HBA, for example, to access a common storage resource. Support for running multiple versions of Oracle Database is also not supported on Oracle Exadata. Some enterprise software products that use Oracle Database specify which Oracle Database release their product can use. If the product uses a release of Oracle not supported on Oracle Exadata, then the database cannot be run on Oracle Exadata.

Stability

Like architecture 3, the Oracle Exadata benefits form the high availability

aspects of Oracle RAC. Because the Oracle Exadata does not use a shared storage server, the storage server will never compromise performance caused by the over subscription issues that can plague architecture 3. The Oracle Exadata is designed to minimize single points of failure from an architectural standpoint, and has redundancy in storage, connectivity to storage and other infrastructure features, such as power.

In most regards, the Oracle Exadata offers a high level of stability and has virtually not deficiencies in this area.

Cost Containment

While the Oracle Exadata has few issues with scalability and stability, it

has weaknesses with regards to cost containment. Since the Oracle Exadata has extreme high performance, it is capable of hosting a large number of databases, a feature which Oracle claims reduces costs by allowing customers to consolidate database server workloads onto one Oracle Exadata.



While this is a plausible argument, it does not take into account the limitations regarding the Oracle Exadata’s inability to run different versions of Oracle Database.

The Oracle Exadata is costly to license from an Oracle standpoint. Oracle Database and Oracle RAC are required, along with licenses for the storage server. The storage server licensing is based on the number of spindles in the storage server. More storage on an Oracle Exadata storage server requires a higher license costs for the storage server.

The Oracle Exadata cannot be installed by end users, and requires specialized personnel to bring it online in customer data centers. Many customers are not comfortable applying all of the disparate patches to firmware, operating system and the storage server that the Oracle Exadata needs from time to time, and must use contract labor for this support.

While the Oracle Exadata does not need involvement from customer storage admins and systems administrators, any cost saving from this are overshadowed by these ongoing support costs.



Database Cloud Server

Having reviewed four of the most commonly used architectures for Oracle Database servers and examined them with regards to scalability, stability and cost containment, this section introduces the Vendita Database Cloud Server and how it addresses the shortcomings of the four architectures.

The four architectures previously reviewed perform well in some of the key deliverables, but have deficiencies in others. The Database Cloud Server is based on a winning architecture that includes the architecture’s strengths, and also addresses the deficiencies. This is made possible by leveraging and exploiting the best technologies of the IBM and Oracle stack. In the sections that follow, these technologies are explored and discussed in terms of the deliverables of scalability, stability and cost containment.

PowerVM The architecture of the Database Cloud Server leverages IBM PowerVM virtualization, which is a venerable virtualization technology that is exclusive to IBM Power Systems servers. PowerVM has some similarities to traditional virtualization products, like VMWare, but has benefits that far eclipse traditional virtualization products, that make it more versatile and powerful. PowerVM serves as an arbiter of server infrastructure for the Power Systems servers, and provides different methodologies for allocating resources to servers. PowerVM supports the creation of dynamic logical partitions (DLPARs), a hardware partitioning virtualization technology. DLPARs allow hardware resources on a physical server (frame) to be divided up and dedicated to individual DLPARs. This allows the creation of multiple servers on one frame that are entirely separate with regards to storage, CPU and memory. Figure 5 (below) depicts a common approach to allocating multiple DLPARs from a set of server resources.



Figure 5 – Allocation of server resources by PowerVM

In figure 5, the Power S824 server has 24 CPU cores available, and 2500G of memory. The database hosted on the Power S824 has a production database instance that uses 12 CPU cores, and is allocated 750G of memory. The same Power S842 server also hosts two other instances of the production database, one for quality assurance testing and one for testing. Quality assurance database instances are used for testing changes before they are moved to production. Quality assurance databases need to replicate the behavior of the production database server closely, and are usually setup to have 30-40% of the resources of production. The quality assurance instance has five CPU cores, and 200G allocated to it to allow testing of production processes. It is not necessary to have the same quantity of CPU cores and memory as production, just enough to ensure that production processes can be tested in a reasonable amount of time. The test instance has two CPU cores and 100G allocated to it. In test instances of databases, there are sometimes very few users. For example, in a test instance, a DBA might only be testing patches, or developers might be working on code changes. The ability to have separate partitions that act as individual database servers on one frame does not exist on architectures 1-4. CPU cores and memory can be easily changed for any DLPAR, so this capability has great advantages in terms of scalability. In figure 5, if the PROD database is expected to have an unusually large workload, CPU cores can be reduced for QA, and these can be combined with available unused cores to add processing capability to PROD.



In architectures 1-4, Oracle RAC provides similar scalability, but comes at a high cost in terms of licensing, and also imposes a layer of technical complexity due to Oracle RAC. PowerVM also supports a special type of DLPAR known as a virtual I/O server, or VIO server. The VIO server eliminates the need for physical resources to be dedicated to particular DLPARs, and allows resources to be pooled and shared among servers. This is particularly useful for non-critical servers, such as test or development servers. CPU cores, memory, storage, and network adapters can all be virtualized. Figure 6 (below) shows an example of pooled resources being used in the system previously examined with a test, QA and production system. CPU and memory resources have been allocated to shared pools instead of dedicated pools.

Figure 6 – Pooled CPU and memory capability of PowerVM

In this scenario a pool of five CPU cores services the test and QA servers. Previous TEST had two CPU cores and QA had five CPU cores. By pooling CPU resources, we have conserved two CPU cores. A similar benefit is realized with memory, where 50G of memory has been conserved by creating a shared memory pool. DLPARs can connect to multiple VIO servers for access to these resources, providing redundancy to all of the resources.



Figure 7 (below) shows an example of two VIO servers providing redundant paths to storage adapters and network interface cards (NICs).

Figure 7 – Redundant VIO servers with storage adapters and NICs

With the system shown in figure 7, a failure of a storage adapter or NIC, or an entire VIO server can occur without causing a disruption of service.

Cloud Architectures with PowerVM

Cloud architectures require an infrastructure that supports the creation of

multiple servers on a repeatable basis that can have different requirements with regards to resources required, operating system versions and, in the case of database servers, different versions of Oracle Database software. This capability is ostensibly impossible with architectures 1-4, and is a major limitation of Oracle Engineered Systems like the Oracle Database Appliance and the Oracle Exadata. The ability to support diverse and varied cloud architectures is a native capability for the Vendita Database Cloud Server, and is the origin of its name.

Consider the example database architecture shown in figure 8 (below).



Figure 8 – Separate database servers with varied architecture

The database architecture shown has a number of Oracle Databases running on different operating systems at different versions of Oracle Database, all on different servers.



Like most organizations, this company needs to consolidate workloads of database servers to reduce the number of physical servers in their data center.

Moreover, the company needs to upgrade many of its databases to the latest version of Oracle Database software, which is 12c, release 1. Performing an Oracle version upgrade at the same time as a server migration is very challenging, and the company wants to avoid this. If Oracle Engineered Systems were used for this project, a migration to Oracle Database 12c would be required.

The Database Cloud Server eliminates this complication because both Oracle Database 12c and older versions of Oracle Database, like 11g R1 or 11g R2, can be installed on the same server. This allows databases to be moved to a Database Cloud Server DLPAR without requiring an Oracle Database version upgrade. Since Oracle Database 12c is also installed on the DLPAR, the customer can perform a database version upgrade at the most opportune time.

Another advantage of the Database Cloud Server is the POWER8 processor. With POWER8, fewer processor cores are needed than on an Intel® x86 Linux® architecture.

For the purposes of the migration in this example, a two to one reduction is used.

Figure 9 (below) shows the migration capability of the Database Cloud

Server for the databases shown in Figure 8.



Figure 9 – Migration strategy for scattered database architecture

The migration in Figure 9 involves consolidating all databases onto a single 4U Database Cloud Server. Each of these database server migrations is reviewed in detail below.



ERP Database – This database is currently on an older version of Oracle Database that is no longer supported by Oracle. There is an urgent need to migrate this database to a supported version. The target server has the current version of Oracle Database installed, as well as the latest version of Oracle Database, version 12c R1.

A migration from the old server to a new server on a DLPAR occurs, without changing the version of Oracle Database. After sufficient testing and preparation, a migration from Oracle Database 11g R2 to 12c R1 can occur on the server.

Note that the cores required on the Database Cloud Server are 50% less than the previous server.

CRM Database and EDW Database – These two databases are similar to the ERP database, although the ERP database is on a more recent version of Oracle Database. Both databases are migrated at their current version, and after sufficient testing and validation, migrated to Oracle Database 12c R1.

UTIL Database – This database contains utility databases that include Oracle RMAN repository databases, Oracle Enterprise Manager repository databases, and a repository database for a production scheduling system.

These databases have low requirements in terms of systems resources, and are good candidates to use pooled resources, like CPUs. All of these databases can be moved to their own database server, because of the economic advantage of using shared resources.

SECURE Database – This database contains sensitive information and, for compliance reasons, needs to be on a separate server. Like the UTIL database, it has low resources requirements and can use shared resources.

The database runs on a very old version of Oracle Database and needs to be upgraded to take advantage of new security features. The old version of Oracle Database, 10g R2, is installed on the new server, along with the latest version, 12c R1. This allows the same migration paradigm used on other servers to be implemented on this server.

LEGACY EDW – This database contains a number of Oracle Databases used for historical reporting. The data contained in these databases is from systems that have long been decommissioned and replaced by the current ERP and CRM system.

This database is seldom used, but still needs to be available for occasional queries against the historical data. Because the version of Oracle Database on this server is old and the server has 12 cores, the Oracle support costs are much too expensive.

To eliminate the annual Oracle support costs on this server, the data tables (about 75 total) will be migrated to MariaDB, the open source version of MySQL. The performance of MariaDB is adequate to support any future



reporting needs. This strategy of migrating off of Oracle Database completely eliminates the annual support costs of the legacy server. As can be seen in the example, the ability of the Database Cloud Server to provide a cloud architecture is unequalled by any of the systems in architectures 1-4.

Much of this capability is due to PowerVM. In addition to PowerVM, other technologies are utilized in the Database Cloud Server and are covered in the sections that follow.

Oracle ASM

Similar to architectures 2-4, Oracle ASM is utilized on the Database Cloud Server for database files, because of the superior performance it delivers for Oracle Databases. Oracle ASM is used in conjunction with the file management facilities available in IBM AIX®, including the logical volume manager and the SAS array manager.

IBM AIX®

IBM AIX® is the IBM version of Unix® that can be used on IBM Power Systems servers. IBM AIX® has capabilities superior to other operating systems, such as Red Hat Linux®, due to its longevity and heritage that draws on IBM mainframe technology.

The benefits of AIX® are explored in the next three sections.

Security

AIX® is renowned for the strength of its security, and has fewer security

issues identified by the NSA than any other Unix® operating system.

Stability

Linux®-based servers have crashes due to “kernel panics” where the

entire Linux® operating system crashes, causing a server reboot. Kernel panics are due, in part, to the overall architecture of Linux®, based

on a hybrid monolithic kernel. AIX® does not share these design deficiencies and, as a result, does not

experience crashes at the same frequency as Linux®.

Monitoring



AIX® features a rich set of monitoring utilities, including a more capable version of Unix®/Linux® “top,” known as “topas” and “nmon”, a utility developed for AIX® that has been compiled for other operating systems.

AIX® Made Easy

Linux® has gained acceptance with systems administrators and database

administrators because it includes standard features, like the BASH Unix® shell, and utilities, such as gzip, curl, wget and Python.

The Database Cloud Server features “AIX® Made Easy”, a bundle of preferred utilities that bridge the gap between Linux® and AIX®. The Database Cloud Server also includes “rlwrap”, a utility that adds last command recall to Oracle command line tools, like SQLPlus, lsnrctl and RMAN.

IBM Active Memory Expansion

IBM Active Memory Expansion (AME) is a technology for expanding a DLPAR’s effective memory capacity. AME employs memory compression technology to transparently compress in-memory data, which allows more data to be placed into memory and thus expand the memory capacity of configured systems.

AME is a feature of IBM AIX®, and has been used with Oracle Database with excellent results, effectively doubling the amount of memory available to a server.

Technologies Wrap Up

The Database Cloud Server benefits from the careful selection and utilization of technologies from IBM and Oracle. The confluence of these technologies results in an appliance for running Oracle Databases with superior capabilities.



Database Cloud Server Deliverables

The scalability, stability, and cost containment aspects of architectures 1-4 were examined previously in this document, and are now covered for the Database Cloud Server.

Scalability

The Database Cloud Server excels in the area of scalability in a variety of ways. The Database Cloud Server is available on two different server hardware platforms, the IBM Power S824 and the IBM Power E850. The Power S824 supports two, 12-core POWER8 processors, and the Power E850 supports four, 12-core POWER8 processors. The Power S824 is appropriate for many Oracle Database workloads, and when a substantial number or cores is needed for many servers with large core counts, the Power E850 is a good fit. PowerVM allows for multiple logical partitions, making it possible for multiple database servers to be created, instead of forcing all databases together on a single server. PowerVM allows processor cores, memory and storage to be easily increased for a DLPAR as needed. The effort required for this is far easier and faster than when using Oracle RAC. For lower priority servers like test and quality assurance databases, system resources, such as processor cores and memory, can be pooled, allowing the resources to be conserved for high priority workloads. Advanced Memory Expansion also allows server memory to double, eliminating the need to add memory to a DLPAR if it should become memory starved. IBM Power System servers also supports Red Hat Linux® and can be used for hosting other databases, including MongoDB and MariaDB, in addition to analytic products including SAP HANA.

Stability

Because all storage is local to the Database Cloud Server, the oversubscription issues encountered with systems that use community storage are completely avoided.

The Database Cloud Server uses Oracle ASM to achieve triple mirroring for critical databases, without compromising performance. The Database Cloud Server features redundant storage interfaces and network interface cards. IBM hardware is far more reliable than typical x86-based hardware, so outages due to hardware failures can be avoided. The Database Cloud Server also delivers stability through its monitoring facilities available to maintain performance.



The solution leverages the same database monitoring features used for Oracle Engineered Systems (Oracle Enterprise Manager) to maintain optimum Oracle Database performance. The Hardware Management Console (HMC) provides comprehensive and detailed performance for monitoring DLPARs and virtual I/O servers. IBM hardware includes IBM’s renowned support, and all server hardware features self-diagnostic capabilities unoffered by any other hardware platforms. Should a hardware or operating system issue occur, IBM support is notified and will dispatch support personnel to contact the affected data center.

Cost Containment

Cost containment is an area of strength for the Database Cloud Server. The Database Cloud Server helps keep expenses down by avoiding the required use of Oracle RAC. As stated previously in this document, licensing costs for Oracle RAC are a major reason for high prices associated with Oracle Engineered Systems. Many organizations focus on server consolidation to reduce costs, by consolidating many physical servers onto virtualized environments. The Database Cloud Server delivers server consolidation with its virtualization capability available through PowerVM. The Database Cloud Server allows older versions of Oracle Database to be installed, which in turn allows migrations to occur without requiring an upgrade to Oracle Database 12c. The ability to “dial in” only the resources needed for a logical partition conserves processor cores and memory so that these resources can be used to create other servers and stand in reserve for critical databases. The Database Cloud Server does not require a SAN or storage server, which eliminates those additional costs items.

And from a time and effort standpoint, support for the Database Cloud Server is minimal, since less assistance is needed from systems administrators or storage administrators. This allows them to work on other tasks.

Agility

Agility is a deliverable absent from Oracle Engineered Systems and most database server architectures. As a result, agility is a focus point in this document.

Systems that have agility can change quickly in response to business needs, and Database Cloud Server can provide agility because of some of the capabilities already discussed, such as the ability to create multiple DLPARs and the ability to quickly add resources to DLPARs.



Zero Installation Time

Perhaps the most significant reason that the Database Cloud Server

delivers agility is because of its extraordinarily short installation time. Before being shipped to a customer data center, the Database Cloud

Server is fully configured and tested at an integration center. At the integration center, customer network infrastructure is simulated; allowing customers the option to perform pre-shipment testing on the Database Cloud Server, including running application severs on virtualized environments, or running test queries to validate database performance.

When the Database Cloud Server arrives at the customer data center, there are no operating system or software installation tasks. The Database Cloud Server simply needs to be racked, connected to network and power cables and powered on to be ready for service.

No other engineered system has this capability.



Conclusion

The Vendita Database Cloud Server represents the next evolutionary step in appliances for running Oracle Databases. It includes all of the benefits of key Oracle technologies, like ASM, and also uses IBM technologies like PowerVM to deliver unique capabilities.

The Database Cloud Server has a high level of versatility and can be used by a wide range of Oracle Database clients, including:

▪ Customers already running Oracle Databases on IBM Power Systems technology and want to upgrade their database server to a newer technology with expanded capability.

▪ Customers with a wide array of Oracle Database servers in their data centers, who want to consolidate workloads.

▪ Customers running Oracle Database on Linux® and want the improved stability, security and performance that AIX® delivers on IBM Power Systems hardware.

In each of the cases above, the Database Cloud Server delivers superior capability over competitors in terms of scalability, stability, cost containment and agility.



Glossary The following glossary contains terms used within this publication.

Term Technology Definition ACFS Oracle ASM Clustered File System

Advanced Memory Expansion

IBM A feature of AIX® that allows memory to be compressed up to 2X with the addition of physical memory.

AIX® IBM AIX® is IBM’s version of Unix®. AIX® is an abbreviation for Advanced Interactive eXecutive.

ASM Oracle Automatic Storage Management – A file system designed specifically to store database files in a more efficient manner. Used in place of a cooked file system.

Bare Metal General A term for a server with no virtualization, where the server operating system interacts directly with physical entities, like network cards, memory and storage.

Brocade General Brocade Corporation is the most significant manufacturer of SANs and associated technologies.

Cooked File System

General Any of the file systems maintained by an operating system, including JFS, ext3, NTFS and MS DOS.

Core General Refers to a CPU core, a single element of processing

in a CPU. Different CPU families have different core counts. The POWER8 processor has eight cores per CPU. Intel® server processors typically have two cores per CPU.

Correlated failure General A failure resulting from a common manufacturing defect affecting products manufactured at a similar time at a common manufacturing facility, or at different facilities using identical manufacturing processes. See appendix B for the impact of correlated failures on storage.

CPU General Central Processing Unit – Refers to the POWER or

Intel® processor that performs computation within a server.

Dynamic Logical Partition (DLPAR)

IBM A server created and maintained on a Power Systems server. DLPARs are similar to, but not the same as servers or guest partitions created with other virtualization technologies, like VMWare.

Exadata Oracle Oracle Exadata is one of Oracle’s Engineered Systems designed to run Oracle RDBMS. The Oracle Exadata features the Oracle Exadata storage server.

EXT3 / EXT4 General File systems commonly used by Linux®-based servers.

Fabric General Refers to the internal connection structure of a SAN, which resembles a textile fabric. Fabrics support a multitude of paths between connection points.

FCOE General See Fibre Channel Over Ethernet

Fibre Channel General Common protocol used for connecting servers to storage, uses fiber fiber optic cables instead of copper for increased performance.

Fibre Channel Over Ethernet

General An implementation is Fibre Channel using Ethernet instead of fiber fiber optic cables.



File System General A file system is the low level organization of data on a disk. File systems are maintained by operating systems in most cases. Oracle Grid Infrastructure has a file system designed specifically for databases.

Frame General Refers to a server and everything it contains. Product examples include the IBM S824, p570 and E850.

FSP IBM Flexible Service Processor – A standalone computer included with every IBM Power Systems-based server that manages resource allocation and control of servers on a frame. The FSP provides an interface to an IBM HMC, or allows stand-alone operation with IVM.

Grid Infrastructure Oracle Oracle software that includes RAC, ASM and ACFS.

HA General See High Availability

HBA General Hardware Bus Adapter – A server interface card with dedicated processing for connecting to a SAN.

High Availability General High Availability is a capability afforded by a hardware architecture to tolerate failures of individual hardware components without loss of service.

HMC IBM Hardware Management Console – A stand-alone x86-based computer used to manage one of more Power Systems servers. Servers managed by a HMC are referred to as Managed Servers. The HMC offers a rich user interface, simplifying setup and maintenance.

IBM Techline IBM A team within IBM that assists with the procurement of IBM Power Systems processor-based hardware.

Infiniband General Protocol for connecting servers to storage and for server-to-server communication. Used by the Oracle Exadata.

Integrated Virtualization Manager (IVM)

IBM This is a technology that allows management of a frame without an HMC. A frame configured to use IVM is an unmanaged server. Once a frame is configured for IVM, it’s unchangeable to a managed server.

IVM IBM See Integrated Virtualization Manager.

JFS / JFS2 General Journaled File System – A file system commonly used by IBM AIX® servers, maintained by the operating system. An example of a “cooked file system.”

Managed Server IBM An IBM server managed by an HMC. The HMC maintains settings that are stored in the FSP.

Miscellaneous Equipment Specification (MES)

IBM When hardware is added to an IBM system, IBM’s ordering system issues a MES number that can be referenced by a HMC to ease and simplify the installation of hardware or features.

Network Attached Storage (NAS)

General A stand-alone storage server that can be accessed with a variety of storage protocols, including Fibre Channel.

N-port Virtualization (NPIV)

General A term associated with SAN technology

ODA Oracle See Oracle Database Appliance

OEL Oracle Oracle Enterprise Linux®

Oracle Database Appliance

Oracle The Oracle Database Appliance is one of Oracle’s Engineered Systems designed to run Oracle RDBMS.

POWER IBM POWER® is the family of CPUs used in IBM Power Systems servers. POWER is an abbreviation for Performance Optimization With Enhanced RISC.



PowerHA IBM PowerHA ® is high availability technology available through IBM for POWER processor-based servers.

PowerVC IBM PowerVC™ is a management console based on OpenStack, currently under development by IBM. PowerVC provides an easier method of managing server resources, and provisioning DLPARs.

PowerVM IBM Virtualization technology used by IBM POWER processor-based servers. Allows the creation of VIO Servers and supports the creation of a media repository.

Real Application Clusters (RAC)

Oracle Oracle clustered database technology. It is included with the installation of Oracle Grid Infrastructure

RDBMS Oracle Relational Database Management System.

RISC General Reduced Instruction Set Computer – This refers to a particular class of CPUs, and is the opposite of Intel® CPUs, which have complex instructions. RISC processors typically have more cores per CPU, and more threads of execution per core.

SAN General A network switch designed specifically for storage products, allowing connectivity from multiple servers to a shared storage product.

SAS General Serial Attached SSCI – Most common type of drive used in servers featuring a rotating spindle.

SEA IBM Shared Ethernet Adapter

SR-IOV General Single Root I/O Virtualization – This technology is the basis for self-virtualizing I/O boards that allows the devices to manage virtualized resources. The advantage of SR-IOV for the Database Cloud Server is that individual ports can be assigned to DLPARs without assigning the entire adapter to a DLPAR.

SSD General Solid State Drive - A drive that uses circuitry in place of a rotating spindle. SSDs are high performance, but also come at a high cost.

Unmanaged Server

IBM An IBM Power Systems series server that uses IVM, and is not managed by a HMC.

VET Code IBM A code entered through the HMC or FSP to enable IBM PowerVM for a period of time. PowerVM cannot be used on a POWER server unless a VET code is provided. VET codes are managed through IBM Techline.

VIO Server IBM Virtual I/O Server – A special DLPAR to establish connectivity to physical resources, like storage, memory, CPU cores and network adapters, and provide logical version of these resources to AIX® DLPARs.

Virtualized I/O IBM I/O presented to a DLPAR that is logical, not physical, entirely created and maintained by a VIO Server.

World Wide Port Number

General WWPN - A unique ID for a port number inside a SAN.

Zero Installation Time

Vendita Refers to the exceptionally rapid installation time of the Database Cloud Server



Appendix A - Double vs. Triple Mirroring

The decision to use double or triple mirroring is often confusing to database administrators and systems administrators. Using triple mirroring is costly in that more drives are required and performance is impacted if a traditional RAID technology like RAID 5 is used. However, for databases, triple mirroring can be attractive, because of the criticality of uptime and dire consequences of experiencing multiple drive failures.

Google performed testing on drives as part of an extensive study to better understand the probability of drive failures. The study concluded that the probability of a drive failing, regardless of age, is 8%.

Other studies have indicated a higher failure rate when multiple drives fail due to a correlated failure. Correlated failures occur when drives from the same manufacturer all expect a common manufacturing defect. Academic studies performed on correlated failures indicate that when triple mirroring is used, the probability of surviving a series of correlated failures for a drive array is more than 96%. However for drive arrays that only have double mirroring, this probability drops to below 75%. Based on this data, for critical systems, triple mirroring is an imperative in order to minimize vulnerability to correlated failures. For more information on correlated failures, please refer to the white paper "Using device diversity to protect data against batch-correlated disk failures” by Jehan-François Pâris, Univeristy of Huston, and Darrell D. E. Long, University of California at Santa Cruz.



Appendix B - Reference Documents and Publications

The publications in this appendix are readily available from IBM through the Internet. Enter the title of the book in any Internet search tool and it can be easily located.

The following IBM Redbook is an introduction to two IBM Power Systems servers, and provides an excellent introduction to the Power Systems hardware family. “IBM Power Systems S814 and S824 Technical Overview and Introduction”

PowerVM is a key feature of Power Systems series machines, and is covered in the Redbook shown below. “IBM PowerVM Virtualization Introduction and Configuration“

IBM Integrated Virtualization Manager is a feature of Power Systems servers that allows DLPARs to be created and managed without the IBM HMC. The Redbook shown below is the official IBM guide to IBM IVM. “Integrated Virtualization Manager for IBM Power Systems Servers”



Appendix C - Web Resources

IBM System Planning Tool

The IBM System Planning Tool (SPT) is a browser-based application that helps you design system configurations. It is particularly useful for designing a system based on existing performance data, or based on new workloads.

System plans generated by the SPT can be deployed on the system by the Hardware Management Console (HMC) and Integrated Virtualization Manager. The SPT is available to assist the user in system planning, design and validation, and to provide a system validation report that reflects the user's system requirements while not exceeding system recommendations.

Document Note • Intel®, Intel® logo, Intel® Inside, Intel® Inside logo, Intel® Centrino, Intel® Centrino logo, Celeron, Intel® Xeon, Intel® SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel® Corporation or its subsidiaries in the United States and other countries. • Linux® is a registered trademark of Linus Torvalds in the United States, other countries, or both. • Java® and all Java®-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates. • UNIX® is a registered trademark of The Open Group in the United States and other countries.