hp bladesystem

Technical white paper

HP BladeSystem with Symantec NetBackup and HP StoreOnce Reference architecture and best practices for backup and recovery

Table of contents Executive summary .............................................................................................................................................................. 3

Solution overview .................................................................................................................................................................. 4

Technology overview ............................................................................................................................................................ 5

HP BladeSystem ................................................................................................................................................................ 5

HP BladeSystem c7000 Enclosure ..................................................................................................................................... 6

HP Onboard Administrator ............................................................................................................................................... 7

HP Virtual Connect ............................................................................................................................................................ 7

HP ProLiant BL460c Gen9 Server Blade ......................................................................................................................... 8

HP OneView ....................................................................................................................................................................... 9

HP 3PAR StoreServ 7450 All-flash Storage array ..................................................................................................... 11

HP StoreOncekey features and benefits ................................................................................................................ 12

HP StoreOnce 4700 ....................................................................................................................................................... 13

HP StoreOnce integration with Symantec NetBackup via OST................................................................................. 15

Symantec NetBackup anatomy .................................................................................................................................... 17

Installation and configuration of Symantec NetBackup 7.6 ......................................................................................... 19

Configuring Catalyst stores on HP StoreOnce 4700 and then integrating them into NetBackup ....................... 20

Contents of a Catalyst store explained ....................................................................................................................... 27

Catalyst stores configured in this investigation ......................................................................................................... 28

Symantec NetBackup Oracle RAC 12c integration ......................................................................................................... 29

Test Bed architecture overview ........................................................................................................................................ 29

Solution components .................................................................................................................................................... 30

Optimizing backup and recovery for Oracle RAC 12c on HP BladeSystem with Symantec NetBackup .............. 32

NetBackup tuning guidelines ........................................................................................................................................ 34

Backup configuration ..................................................................................................................................................... 35

Click here to verify the latest version of this document

Technical white paper | HP BladeSystem with Symantec NetBackup and HP StoreOnce

2

Configuring Symantec NetBackup with Oracle ............................................................................................................... 35

Investigation results .......................................................................................................................................................... 43

Backup use cases ........................................................................................................................................................... 43

Restore from HP StoreOnce Backup ................................................................................................................................ 43

Recovery use cases ........................................................................................................................................................ 44

Recovery best practices for virtualized Oracle RAC 12c ........................................................................................... 44

Symantec NetBackup VMware integration: Protecting your VMs ................................................................................ 45

VMware backup configuration ...................................................................................................................................... 48

NetBackup VMware resource tuning ........................................................................................................................... 52

NetBackup VMware Recovery use cases: VMs and individual files .......................................................................... 54

Summary ............................................................................................................................................................................. 59

NetBackup licensing ........................................................................................................................................................... 59

StoreOnce licensing ........................................................................................................................................................... 60

Appendix ASample RMAN backup scripts used in this investigation ....................................................................... 60

Appendix BSample RMAN recovery scripts used in this investigation .................................................................... 66


3

Executive summary

Applications that are core to the success of your business are often classified as mission critical or business critical. Their requirement for zero-downtime operations results in stringent service-level agreements (SLAs) to achieve and maintain high levels of performance, availability, reliability, and serviceability. Normally, addressing these requirements requires the application to be deployed upon bare-metal operating environments, but advancements in hypervisor technologies now let you deploy these applications as virtualized workloads, for added scalability and rapid deployment.

Many organizations rely on virtualization to improve security and meet compliance requirements, increase data center flexibility, simplify deployment and management, improve operational efficiencies, and lower the total cost of ownership (TCO). To fully embrace and reap these benefits, adopting an integrated end-to-end solution can deliver the agility needed to accommodate the current needs and future growth requirements that mission-critical applications demand. Equally important, an integrated technology stack can enable you to expand, contract, scale up, scale down, scale out, or scale in to address infrastructure allocation requirements as workloads change. To meet these needs, HP has created a converged infrastructure platform using HP BladeSystem and HP OneView. Together it delivers a single infrastructure and single management platform with automation for rapid delivery of service and rock-solid reliability with federated intelligence. HP BladeSystem is a modular infrastructure platform that converges compute, storage, fabric, management and virtualization to accelerate operations and speeds delivery of applications and services running in physical, virtual, and cloud-computing environments.

When it comes to mission-critical transactional and analytical workloads, Oracle databases are chief among the applications driving these workloads. More importantly, many major applications rely on Oracle database architectures within the application stack. Deploying a solid database architecture, virtually or physically, is a key success indicator that can mean the difference between leading and following your competition.

A solid database architecture can make the difference between competitive differentiation and simple comparative parity. Competitive organizations establish aggressive recovery-point objectives (RPOs) and recovery-time objectives (RTOs) to minimize data loss and ensure application recovery and reliability. They choose primary infrastructure and data protection strategies that must deliver application-consistent backups, application-restartable recoveries, user-defined service levels, single-point-of-failure eliminations, along with the ability to maximize resource utilization. Finding these requirements in a non-integrated solution is possible, but the long-term application lifecycle costs are often much greater in the end.

In this paper, we will examine both a mission-critical application-streaming backup and recovery as well as a virtual machine (VM)-based backup and recovery leveraging the Symantec NetBackup features such as, Traditional Oracle RMAN scripting integrated with NetBackup scheduling and catalogs, Full VADP integration with VMware, and Symantec NetBackup Accelerator to speed up File system and VMware backups. All of these advanced data protection features can be supported on HP StoreOnce using HP StoreOnce Catalyst backup targets.


4

Solution overview

A well-designed data management strategy does not use a one-size-fits-all approach to data protection. A combination of tactics tailored to the target application provides the best protection and SLA adherence. Following that principle, the environment described below takes different approaches for critical and generalized workloads. Critical applications, such as Oracle database needing deep integration with the data management system are protected using the Symantec NetBackup RMAN scripting integration. This technique relies on a process of linking the Oracle Server software with the NetBackup API library installed by NetBackup for Oracle. Oracle uses this library when it needs to write to or read from the devices that NetBackup media manager supports. For generalized virtualized workload, the Symantec integration with VMware vSphere Storage APIs for Data Protection (VADP) along with Symantec NetBackup Accelerator provides a fast/unified protection and recovery vehicle for all VMs, with fast recovery times. None of this would be possible if it were not for the tight integration of HP StoreOnce with the advanced features offered by Symantec OpenStorage interface (OST) and the HP StoreOnce Catalyst backup target type. Using this technology, HP StoreOnce can offer:

Source side deduplication

NetBackup controlled replication between HP StoreOnce targets using Storage Lifecycle policies

Asynchronous expiry dates of different copies of data on different HP StoreOnce units

Support for Symantec Granular recovery technologyrecovery options can be browsed directly on the HP StoreOnce unit before recovery

Support for Symantec Targeted AIR (auto image replication) on HP StoreOncea must for improved disaster recovery (DR)

Support for NetBackup Accelerator on StoreOnce for VMware virtual machines and OS file systems Full backups at the speed of incrementals

Improved StoreOnce backup target device reporting through the OST interface

For more details on Symantec NetBackup Integration with Oracle, see NetBackup integration with Oracle.

For more details on Symantec NetBackup integration with VMware including NetBackup Accelerator, see NetBackup Integration with VMware.

Figure 1. Schematic overview of the solution


5

The diagram on page 4 shows the overall solution set up, all the VMs are configured in the highly scalable HP BladeSystem, the datastores for these VMs utilize an HP 3PAR disk array. The set up simulates a typical mid-range customer with Oracle databases and other applications running on several Windows VMs. The NetBackup Master server and media server are also virtualized. The real world transaction load capability of the two Oracle databases is simulated using the HammerDB Utility. On the StoreOnce 4700 backup appliance, we have created three backup targets, one each for the two Oracle databases and another one for the Windows 2012 VMs 813 backups. This follows the best practice of a backup target for different data types. We also utilize the source and target side deduplication capabilities of HP StoreOncefor the Oracle databases, we found the additional CPU load of performing deduplication on the database server itself increased the CPU load, and this would affect transactional performance, so instead the deduplication is performed on the StoreOnce unit itself (target-based deduplication). With the Windows 2012 VM backups, we are using Symantec NetBackup Accelerator on VMs, which greatly reduces the volume of data transferred in the backup and then goes on to produce a synthesized full, in this case the deduplication is performed on the Windows 2012 VMs (source-side based deduplication)this then reduces the load on the HP StoreOnce unitso we can architect the best of both worlds.

Technology overview

HP BladeSystem HP BladeSystem is a modular infrastructure platform that converges compute, storage, fabric, management and virtualization to accelerate operations and speeds delivery of applications and services running in physical, virtual, and cloud-computing environments. The unique design of the HP BladeSystem c-Class helps reduce cost and complexity while delivering better, more effective IT services to end users and customers.

Only HP BladeSystem delivers a whole new experience for IT with the Power of Oneone infrastructure, one management platform to help customer reduce the need for multiple management tools, streamline processes and eliminate common sources of errors to speed the delivery of services. As the single software-defined management platform, HP OneView delivers industry leading innovation such as proactive health monitoring built on a federated architecture to streamline operations and maximize availability while delivering 23 percent lower TCO over other bladed architectures.

HP BladeSystem with HP OneView delivers the Power of Oneone infrastructure, one management platform. Only the Power of One provides leading infrastructure convergence, the security of federation, and agility through data center automation to transform business economics by accelerating service delivery while reducing data center costs. As a single software-defined platform, HP OneView transforms how you manage your infrastructure across servers, storage, and networking in both physical and virtual environments.

The HP BladeSystem brings together the best HP innovations and latest industry standards into one design to address some of the toughest challenges of todays data centersefficiency, availability, and speed of service delivery. The HP BladeSystem offers unique differentiators versus other blade choices that can improve efficiency for customer at a data center level.


6

HP BladeSystem c7000 Enclosure

The HP BladeSystem c7000 Enclosure represents an evolution of the entire rack-mounted infrastructure, consolidating and repackaging featured infrastructure elementscomputing, storage, networking, and powerinto a single infrastructure-in-a-box that accelerates data center integration and optimization.

The HP BladeSystem enclosure infrastructure is adaptive and scalable. It transitions with your IT environment and includes modular server, interconnect, and storage components. The enclosure is 10U high and holds full-height and/or half-height server blades that may be mixed with storage blades, plus redundant network and storage interconnect modules. The enclosure includes a shared high-speed NonStop passive midplane with aggregate bandwidth for wire-once connectivity of server blades to network and shared storage. Power is delivered through a passive pooled-power backplane that enables the full capacity of the power supplies to be available to the server blades for improved flexibility and redundancy. Power input is provided with a very wide selection of AC and DC power subsystems for flexibility in connecting to data center power.

You can populate a BladeSystem c7000 Enclosure with these components: Server, storage, or other option blades

Interconnect modules (four redundant fabrics) featuring a variety of industry standards including:

Ethernet

Fibre Channel

Fibre Channel over Ethernet (FCoE)

InfiniBand

iSCSI

Serial Attached SCSI (SAS)

Hot-plug power supplies supporting N+1 and N+N redundancy

BladeSystem OA management module

Figure 2. HP BladeSystem c7000 enclosure


7

HP Onboard Administrator The Onboard Administrator centralizes c-Class infrastructure management. Together with the enclosures HP Insight Display, the Onboard Administrator has been designed for both local and remote administration of HP BladeSystem c-Class components and provides the following capabilities:

Wizards for simple, fast setup, and configuration

Highly available and secure access to the HP BladeSystem infrastructure

Security roles for server, network, and storage administrators

Automated power and cooling of the HP BladeSystem infrastructure

Agent-less device health and status

Thermal Logic power and cooling information and control

Figure 3. HP Onboard Administrator for the BladeSystem c7000 chassis

HP Virtual Connect HP Virtual Connect technology provides wire-once, change-ready connectivity that is simple, flexible, and secure. This technology is a key element of HP Converged Infrastructure, providing a better way to connect your virtualized environment to the network core. Rather than tying profiles to specific server blades, you create a profile for each of the bays in the c7000 enclosure; Virtual Connect then maps physical LAN or SAN connections to these profiles, allowing you to manage connectivity without involving LAN or SAN administrators. In addition, if a server blade were to fail, you could move its associated profile to a bay containing a spare blade, thus restoring availability without needing to wait for assistance.

HP Virtual Connect FlexFabric technology supports the convergence of traffic based on different network protocols, allowing you to split a 20 Gb network connection into four variable partitions. The benefits of FlexFabric technology include the ability to replace multiple low-bandwidth physical NIC ports with a single port, lower management burden, fewer NICs and interconnect modules, and lower power and operational costs.

Key benefits of Virtual Connect: Servers are change-ready. Move, add or change servers without affecting the LAN and SAN

Reduce cables without adding switches to manage

Standards-based compatibility with other brands of Data Center networking infrastructure

Pre-configure network connections for blade enclosure bays before servers are installed for easy, drop-in installation

Figure 4. HP Virtual Connect FlexFabric-20/40 F8


8

HP Virtual Connect FlexFabric 20/40 F8 Modules are the simplest, most flexible way to connect virtualized server blades to data or storage networks, HP Virtual Connect FlexFabric 20/40 F8 Modules eliminate up to 95 percent1 of network sprawl at the server edge using one device that converges traffic inside enclosures and directly connects to external LANs and SANs. Utilizing Flex-20 technology with Fibre Channel over Ethernet (FCoE) and accelerated iSCSI, these modules converge traffic over the industrys first high-speed 20 Gb connections to servers with HP FlexFabric Adapters (HP FlexFabric 20 Gb 2-port 630FLB and 630M Adapters). Each redundant pair of Virtual Connect FlexFabric modules provides eight adjustable downlink connections (six Ethernet and two Fibre Channel, or six Ethernet and two iSCSI, or eight Ethernet) to dual-port 20 Gb FlexFabric Adapters on servers. Up to eight uplinks are available for connection to upstream Ethernet (up to 40GbE) and Fibre Channel switches. Virtual Connect FlexFabric modules avoid the confusion of traditional and other converged network solutions by eliminating the need for multiple Ethernet and Fibre Channel switches, extension modules, cables, and software licenses. In addition, Virtual Connect wire-once connection management is built-in enabling server adds, moves, and replacements in minutes instead of days or weeks.

HP ProLiant BL460c Gen9 Server Blade Designed for a wide range of configuration and deployment options, the HP ProLiant BL460c Gen9 Server Blade provides the flexibility to optimize your core IT applications with right-sized storage for the right workload for a lower TCO.

The BL460c Gen9 Server Blade uses Intel Xeon E5-2600 v3 processors with up to 70 percent performance gains over the previous generation, plus enhanced HP DDR4 SmartMemory offering up to a 33 percent performance increase. It also offers flexible storage controller options, 12 Gb/s SAS, 20 Gb FlexibleLOM NICs, and USB 3.0 on the internal connector. All of this is managed by HP OneView, the converged management platform that accelerates IT service delivery and boosts business performance.

Figure 5. HP ProLiant BL460c Gen9 Server Blade

Superior value across different workloads: Delivers the right performance, scalability, and economics for the converged data center in the new era of compute at the

lowest cost, fastest time to value with latest innovations.

Provides the flexibility to optimize your core IT applications, with right-sized storage for the right workload lowering TCO, all managed by HP OneView the converged management platform that accelerates IT service delivery and boosts business performance.

1 HP internal calculations comparing the number of hardware components of traditional infrastructure vs. HP BladeSystem with two Virtual Connect FlexFabric

modules, June 2013.


9

Increased performance in the data center: The HP ProLiant BL460c Gen9 Server Blade delivers up to 70 percent performance increases with Intel Xeon

E5-2600 v3 processors.

HP DDR4 SmartMemory at 2133 MHz (up to 512 GB) and 35 percent lower power consumption than 1.5 V DDR3 at the same bin speed.

Optional HP Smart Array P244br storage controller with 1 GB Flash Backed Write Cache (FBWC) DDR3 at 1866 MHz improves storage performance for demanding workloads.

More versatile than ever before: The HP ProLiant BL460c Gen9 Server Blade delivers a flexible embedded storage controller options (HP Smart Array

P244br, HP H244br Smart Host Bus Adapter, or the HP Dynamic Smart Array B140i Controller) for increased deployment flexibility.

Every BL460c Gen9 Server Blade includes USB 3.0, future optional dual microSD, and future optional M.2 support for a variety of system boot options at the best price.

Both Unified Extensible Firmware Interface (UEFI) and Legacy BIOS modes available for increased configuration and deployment versatility.

Transforming business economics by accelerating service delivery: HP OneView provides the Power of One a single comprehensive view of the data center, managing hardware, software,

firmware, and drivers.

HP Integrated Lights-out (iLO) software features server lifecycle management advancements including iLO Federation, which remotely manages groups of servers at scale with built-in rapid discovery of all iLOs, group configurations, group health status, and ability to determine iLO licenses.

HP Smart Update Manager powered by iLO Federation technology for faster firmware updates.

Industry leading serviceability: HP Technology Services delivers confidence, reducing risk and helping customers realize agility and stability.

Provides consulting advice to transform and modernize your infrastructure; services to deploy, migrate and support ProLiant servers and education to help you succeed quickly.

Offers worldwide availability, service and support for customers who have multiple data centers in multiple countries.

HP OneView HP OneView is converged management that eliminates infrastructure complexity with automation simplicity. This modern management architecture is designed to accelerate your IT operations for managing servers, storage, and network resources.

The HP OneView design is:

Converged, with an innovative architecture that delivers a unified and consistent management experience across servers, storage, and networking.

Software-defined, providing software-based control, infrastructure mapping, and a user-centric approach to ensure rapid, repeatable, and reliable operations at lower costs.

Automated, working as an intelligent hub to streamline the delivery of IT services and to speed the transition to IT-as-a-Service and to the hybrid cloud.

Convergence cuts in half the number of tools required to learn, manage, deploy, and integrate infrastructure. The innovative architecture delivers simplified and consistent management across servers, storage, and networking. A single, open management platform supports multiple generations of HP DL servers, HP BladeSystem, HP 3PAR storage, and HP ConvergedSystem. Integration solutions also allow you to provision and manage lifecycles within familiar consoles like VMware vCenter Server and Operations Manager, Microsoft System Center, and Red Hat Enterprise Virtualization.


10

Software-defined approaches to systems management create get it right repeatability every time to help you prevent unplanned outages caused by human error or device failure. Profiles and groups capture your best practices and policies to help you increase productivity and enable compliance and consistency. You can also manage this infrastructure programmatically using powerful APIs build on industry standards such as REST. These APIs are easily accessible from any programming language and SDKs are provided for interfaces, Windows PowerShell, and Python scripts.

Automation can streamline your delivery of IT services and speed your transition to Infrastructure-as-a-Service (IaaS) and hybrid cloud delivery. Using HP OneView as an intelligent hub provides you a closed-loop automation with consistent APIs, data model, and state-change message bus. Your virtualization administrators can automate control of HP compute, storage, and networking resources using VMware vCenter or Microsoft System Center without having detailed knowledge of each device. The result: tasks, processes, and projects are accomplished faster and with more consistency than the older patchwork approaches to management.

These innovations in HP OneView can reduce your OPEX and improve your business agility. HP OneView is your converged management foundation to free your resources for new business initiatives, whether that is lights-out automation or enabling infrastructure for a hybrid, heterogeneous cloud. Efficiently transition from your current HP and third party infrastructure, tools, and processes to your vision of IT-as-a-Service using HP OneView.

HP OneView simplification through convergence Leveraging the power of HP management through one interface

Intelligent Provisioning

Array Configuration Utility

Integrated Lights-Out 4 (iLO 4)

HP Smart Update Manager

HP Systems Insight Manager

Virtual Connect Manager/VCEM

Onboard Administrator

HP 3PAR array management

Figure 6. HP OneView dashboard


11

HP 3PAR StoreServ 7450 All-flash Storage array HP 3PAR StoreServ 7450 is an all-flash based storage array that combines accelerated performance with all the enterprise-class, tier-1 features and functionality expected for a mission-critical application environment. This flash-optimized architecture relies on several unique HP 3PAR StoreServ innovations:

Mesh-Active architecture: Fine-grained virtualization and system-wide striping.

Purpose-built HP 3PAR StoreServ ASIC: Supports mixed workloads with extremely high performance levels.

HP 3PAR Adaptive Read and Write: Matches host I/O size reads and writes to flash media at a granular level to avoid unnecessary data reads and writes to reduce latency.

Autonomic cache offload: Reduces cache bottlenecks by automatically changing the frequency at which data is offloaded from cache to flash media based on utilization rate.

Multi-tenant I/O processing: Enables performance improvement for mixed workloads by breaking large I/O into smaller chunks.

A unique suite of persistent technologies power HP 3PAR StoreServ 7450 Storage in delivering high availability and tier-1 resiliency to performance-critical applications including:

HP 3PAR Persistent Cache: Preserves service levels, so they are not impacted by unplanned component failures.

HP 3PAR Persistent Ports: Allows non-disruptive upgrades without relying on multi-pathing software and without initiating failover.

HP 3PAR Peer Persistence: Ability to federate storage across data centers without being constrained by physical boundaries.

HP 3PAR StoreServ Data at Rest Encryption: Protects data from both internal and external security breaches.

Flash-based media failure reconstruction: This enables the system to provide consistent performance levels even under situations of flash media failure.

HP 3PAR Remote Copy software enables low recovery time objectives (RTOs) and zero data-loss recovery point objectives (RPOs) with complete distance flexibility.

Thin-deduplication increases capacity

Efficiency without compromising performance inline deduplication lowers flash cost by saving capacity up front without the need for any storage pools or post-processing.

Avoids unnecessary writes to flash media extending its life Integration with Peer Motion increases storage efficiency across systems.

HP 3PAR StoreServ storage is also backed by the Get 6-Nines Guarantee, which stands behind the ability of all quad-node and larger HP 3PAR StoreServ Storage systems to deliver 99.9999 percent data availability.2

Figure 7. Front view of the HP 3PAR StoreServ 7450 All-flash

2 Subject to qualification and compliance with the HP 3PAR Get 6-Nines Guarantee Program Terms and Conditions, which will be provided by your HP Sales or

Channel Partner representative.


12

Solid state drives The HP 3PAR StoreServ 7450 All-flash array offers following types of SSDs in either SFF or LFF profile.

HP 3PAR StoreServ M6710 480GB 6G SAS SFF(2.5in) MLC Solid State Drive

HP 3PAR StoreServ M6720 480GB 6G SAS LFF(3.5in) MLC Solid State Drive

HP 3PAR StoreServ M6710 480GB 6G SAS SFF(2.5in) cMLC Solid State Drive

HP 3PAR StoreServ M6720 480GB 6G SAS LFF(3.5in) cMLC Solid State Drive

HP 3PAR StoreServ M6710 920GB 6G SAS SFF(2.5in) MLC Solid State Drive

HP 3PAR StoreServ M6720 920GB 6G SAS LFF(3.5in) MLC Solid State Drive

HP 3PAR StoreServ M6710 1.92TB 6G SAS SFF(2.5in) cMLC Solid State Drive

HP 3PAR StoreServ M6720 1.92TB 6G SAS LFF(3.5in) cMLC Solid State Drive

For more info: hp.com/h20195/v2/GetDocument.aspx?docname=c04111384

HP StoreOncekey features and benefits HP StoreOnce is a dedicated backup appliance with deduplication available in a range of models from 1 TB to 2240 TB. Some models are available as VMs and others are available as physical hardware appliances. HP StoreOnce offers significant savings in storage costs because of deduplication and allows customers to keep pace with increasing data growth. A further advantage of deduplication is that it enables low-bandwidth replication allowing for offsiting of data in a very cost-effective and efficient way. Each StoreOnce unit can support any mix of VTL NAS (CIFS and NFS) and HP StoreOnce Catalyst storesto allow it to function in the widest range of environments/usage models and with the widest range of software. One of the most powerful offerings is the combination of HP StoreOnce Catalyst stores and Symantec NetBackup OST.

For more details on product ranges and compatibility, visit HP StoreOnce overview on hp.com

HP StoreOnce Compatibility Guide

Industry-leading, scale-out architecture to meet enterprise requirements The scale-out architecture allows you to grow as your business needs dictate and not be limited by technology or vendor constraints. Choose capacity points available through virtual backup solutions or dedicated appliances that start small and allow you to add in virtual capacity, shelves, or nodes as needed.

With a range of capacity points from 1 TB to 2240 TB, HP StoreOnce Backup suits all requirements from small remote offices to enterprise data centers with centralized monitoring through HP Reporting Central.

HP StoreOnce Deduplication Deduplication works by examining the data stream as it arrives at the storage appliance, checking for small blocks of data that are identical and removing redundant copies. If duplicate data is found, a pointer is established to the original set of data as opposed to actually storing the duplicate blocks, removing, or deduplicating the redundant data. The key here is that the data deduplication is being done at the block3 level to remove far more redundant data than deduplication done at the file level where only duplicate files are removed. HP StoreOnce uses data compression prior to storing data. What makes HP StoreOnce deduplication technology unique is its variable length chunking algorithmwhich accommodates minor changes in the backup stream layout and the fast matching algorithm using HP Sparse Index technology developed by HP Labs.

Data deduplication is especially powerful when it is applied to backup, since most backup data sets have a great deal of redundancy. The amount of redundancy will depend on the type of data being backed up, the backup methodology, and the length of time the data is retained.

3 Block is sometimes referred to as segment in other deduplication technology.


13

HP StoreOnce provides virtual tape (VT), network-attached storage (NAS), or StoreOnce Catalyst target devices for data protection applications. Interfaces can be via a network connection or FC. Figure 6 shows the basic components of the StoreOnce appliance. The actual storage medium is hard disk and these are arranged in a RAID 6 configuration with an enterprise-class HP-designed RAID controller. Data is written across all disks in the RAID. RAID 6 prevents data loss in case of two hard disk failures. RAID disks in current StoreOnce appliances are either 2 TB or 4 TB serial-attached SCSI (SAS) disk drives.

HP StoreOnce deduplication is also used to move backups to other HP StoreOnce appliances in a bandwidth-efficient manner. This enables customers to move backups to another physical location often using a WAN connection with no human intervention. In the event of a total site loss, the data is still safe at the DR site and systems can be quickly restored. Because of the tight OpenStorage Technology (OST) interaction with NetBackup using the HP StoreOnce Catalyst backup target type, this replication to another site can be actually controlled by NetBackup itself using Storage Lifecycle policies. Replication to another NetBackup domain is further enhanced through the use of targeted Auto Image Replication (AIR) integration, which allows automated catalog imports that drastically reduce DR recovery times.

Figure 8. StoreOnce Architecture overview

HP StoreOnce 4700 HP StoreOnce has been designed to cater to the needs of all types of customers from entry level to large scale enterprises. HP StoreOnce Backup systems deliver scale-out capacity and performance to keep pace with shrinking backup windows, reliable DR, simplified protection of remote offices, and rapid file restore to meet todays SLAs. The models vary by capacity and connectivity protocol and customers can start out by purchasing a single HP StoreOnce base unit/couplet, and then expand with additional couplets and expansion shelves.

Note In all cases, actual backup performance is dependent upon configuration, data set type, data change rate, compression levels, number of data streams, number of devices emulated and number of concurrent tasks, such as housekeeping or replication.


14

Table 1. StoreOnce configurationoptions and features

HP StoreOnce 4700 specification

Form factor 4U scalable rack

Total capacity (RAW) Up to 192 TB4

Total capacity (usable) Up to 160 TB4

Data retention with deduplication (20:1) 3.2 PB4

Maximum number of source appliances per target appliance (fan in)

50

Write performance (aggregated VTL) 7.6 TB/hr4

Read performance (aggregated VTL) 9.0 TB/hr4

Catalyst performance 22 TB/hr4

Targets for backup applications HP StoreOnce Catalyst, virtual tape library (VTL), and NAS

Device interfaces 4x 8 GB FC, 2x 10 Gb Ethernet, 4x 1 Gb Ethernet

Disk drives 2 TB, SAS 7200 rpm, 3.5-inch

Number of disk drives 12 (min), 12 x 8 (max), hardware RAID 6

Maximum number of StoreOnce Catalyst, VTLs, and NAS backup targets (combined) on StoreOnce 4700

50

Maximum number of cartridges emulated 204, 800

Replication Supports data replicationReplication is automatic and appliances may function as both replication targets and sources simultaneously with licensing only being required for appliances acting as a target. Replication of data can occur between VTL and NAS devices created on StoreOnce appliances and StoreOnce VSAs.

To provide sizing assistance for which HP StoreOnce model to choose, use the HP Storage Sizer.

Key parameters required for accurate sizing are data volumes, retention periods required, and an assessment of data change rates.

4 These values assume infinite performance hosts and measure maximum ingest rate. Deduplication ratios assumes small data change rate and long

retention period.


15

HP StoreOnce integration with Symantec NetBackup via OST In this particular integration, we will be using the HP StoreOnce Catalyst target OST device type because it offers the following benefits with NetBackup OST:

1. Media server deduplicationSaves on network bandwidth on Ethernet and FC 2. ISV controlled replication (duplication in Symantec terminology)Including multi-hop replication 3. Quota setting possible (capacity management) 4. All copies of data are known to NBUAllows secondary copies to be promoted to primary copies in DR scenarios 5. Much better DR recovery times(No need to import) if using Symantec AIR (Automated Image Replication) 6. Supports Federated Catalyst stores on StoreOnce 6500 (large teamed Catalyst stores for easier management) 7. Support for Symantec Granular Recovery Technology (GRT)Single item recovery from a snapshot 8. Support for Symantec NetBackup AcceleratorFull backups at the speed of incrementals

All the configurations that are supported, including under what OSs can be found in the compatibility guide at hp.com/go/ebs. We will use Catalyst over IP in this scenario.

HP StoreOnce supports different backup target types.

1. VTLover iSCSI or FCtradition tape based emulation 2. NAS (CIFS/NFS) disk-based backup 3. HP StoreOnce Catalysta new intelligent API-based backup target capable of integrating much more tightly with

Symantec NetBackup. The HP StoreOnce plugin for Symantec NetBackup is an OST (OpenStorage Technology) plugin that is installed on the NetBackup Media Server and the plugin is freely downloadable from hp.com at the following URL: h20392.www2.hp.com/portal/swdepot/displayProductInfo.do?productNumber=StoreOnce

4. The integration allows innovative features such as distributing deduplication media server and StoreOnce backup target appliance for enhanced performance, Catalog replication between domains for faster DR, Symantec Granular Recovery Technology (GRT) support for individual file recovery and Symantec NetBackup Accelerator support for superior incremental backup

In this HP BladeSystem solution, we have chosen to use catalyst because we want to offer customers the option of media server based deduplication (on the Oracle RAC nodes) and we also want to take advantage of NetBackup Accelerator for VMware VMs. Symantec media server code is installed on the Oracle RAC nodes and a media server dedicated to the VM backups, along with the HP OST Plug-in v3.1 Symantec NetBackup Client software is installed on the VMs 17. Let us now look at the HP StoreOnce Catalyst implementations in this investigation.

On the Oracle RAC servers, we have enabled catalyst in high bandwidth mode. Which means the deduplication load is performance on the StoreOnce unit itselfbecause we do not want to add additional CPU load to these Oracle RAC servers that could potentially impact the transactional throughput during backup.

On the Windows VM backups using the media server mediawe have implemented Low Bandwidth Catalyst backupwhere the correctly sized media server performs the deduplication processthis along with using Symantec NetBackup Acceleratorreduces the network traffic for these backups to an absolute minimum, reduces data storage requirements and speeds up overall throughput.


16

Figure 9. Oracle RAC backups use high bandwidth catalyst implementation in this investigation

In the above scenario, a high bandwidth catalyst store is provisioned which means all the deduplication is taking place on the StoreOnce 4700allowing for more Oracle transactions during backup. HammerDB is a commercially available tool for creating transaction loads on Oracle databases so we can simulate the typical backup performance under real-world load conditions.

The diagram below shows the Low Bandwidth Catalyst implementation used in this investigation along with NetBackup Accelerator. The NetBackup Accelerator functionality is included in the NetBackup client software installed on each of the Windows 2012 VMs and seeks to intelligently map which blocks have changed between backups. The media server media contains the HP StoreOnce OST Plugin and the HP StoreOnce Catalyst store on the StoreOnce 4700 that has been configured in Low Bandwidth Catalyst mode. This means the deduplication work is done mainly on the Windows 2012 Media Server VM, which has been sized correctly to accommodate this load. This usage model is sometimes known as source side deduplication and the deduplication load on the StoreOnce unit is reduced because it is taking place on the media server instead.

Figure 10. Windows VM backups use Low Bandwidth Catalyst implementation along with Symantec NetBackup Accelerator


17

Symantec NetBackup anatomy Figure 11. Symantec NetBackup anatomy

The above diagram shows the basic components in a NetBackup Domain.

Master serverWhere the main Catalog information is kept, the master server is also the central administration server.

Media serverA data mover in our scenario where each Oracle RAC nodes are configured with NetBackup Media Server software for Linux. The media server Media used for the Windows 2012 backups is running NetBackup Media Server software for Windows 2012. We are using Catalyst over IP so the transport to the StoreOnce unit from the media servers is 10GbE.

Media Manager Storage unitis a logical storage device that can be tape, virtual tape, disk or OpenStorage unit.

Basic disk/Advanced disk storage unitSupport for CIFS or NFS StoreOnce Backup targets.

OpenStorage disk poolThe correct term for how catalyst is supported in NetBackup using the process of: Storage Server > Disk Pool > Storage Unit.

NetBackup clientsAny host that requires data to be backed up needs to have the NetBackup client software loaded (NetBackup Client software is included in the media server software install by default), in our environment the NetBackup Client software only needed to be loaded on the six Windows VMs we were backing up (installed by default with media server software on Oracle RAC nodes). The client software we installed on the six Windows VMs performs the integration with VADP VMware Backup API and gives us support for NetBackup Accelerator for VMs. Loading the client software on the VMs allows us to restore directly to the VM and also enables application state capture for SQL, Exchange, and SharePoint.

HP StoreOnce OST PluginsThe way HP StoreOnce interfaces with Symantec OST API. These must be installed on every server acting as media server to support HP StoreOnce Catalyst backup targets. So the Oracle RAC servers had the Linux based OST plugin loaded and the Windows 2012 media server had the Windows OST Plugin installed. h20392.www2.hp.com/portal/swdepot/displayProductInfo.do?productNumber=StoreOnce


18

The screenshot below shows how all the VMs are distributed across different ESXi hosts and the virtualized NetBackup Master server and media server were installed (NetBackup-0, NetBackup-1) along with various HP management software for virtualized environments.

This screenshot below is a visual through vSphere Client of the virtualized environment we are backing up.

4 node RAC clusterBender 14RH Linuxalso NetBackup media servers with HP OST plugin installed.

3 node RAC clusterRAC1node 13RH Linuxalso NetBackup media servers with HP OST plugin installedno particular reason for 3 nodesjust convenience.

NetBackup 0 master serverVirtualized on Windows 2012 R2.

NetBackup 1 media server (virtualized)Used to backup VMs windows 16 using NetBackup Accelerator.

Windows 16Windows 2012 Virtualized servers with NetBackup Client loaded.

The Minimum revision of NetBackup to support Windows 2012 R2 as a master/media server is v7.6.0.3 which is what was used in this investigation.


19

Installation and configuration of Symantec NetBackup 7.6

The configuration of the NetBackup environment took place as follows:

It is of key importance when installing Symantec NetBackup to always use FQDN names 1. Install NetBackup on a separate master server and separate media server both running Windows 2012 R2 2. On the master server, add the new media server to the additional servers list of the master server 3. Install the Linux media server software on RAC Cluster called Bender nodes 1, 2, 3, 4 and RAC cluster called

RAC1node 1, 2, 3

4. Before writing to a storage unit, link the Oracle Server software with the NetBackup API library installed by NetBackup for Oracle. Oracle uses this library when it needs to write to or read from the devices that NetBackup media manager supports. Link the Oracle RMAN using the linking script that NetBackup provides. e.g., for Linux run /usr/openv/netbackup/bin/oracle_link. Consult Symantec NetBackup for Oracle Administrators Guide for more details symantec.com/business/support/index?page=content&id=DOC6481

5. Install HP OST Plugin Version 3.1 to all media servers (Windows and Linux)no cost download from h20392.www2.hp.com/portal/swdepot/displayProductInfo.do?productNumber=StoreOnce

6. Install the latest device mapping files from Symantecthese enable the advanced functionality such as NetBackup Accelerator. These are no cost and downloadable from: For UNIX: symantec.com/docs/TECH216417 For Windows: symantec.com/docs/TECH216416

7. On all the VMware VMs we installed the NetBackup Client because we wanted the ability to be able to restore single files directly to the VM

8. Symantec NetBackup offers various tuning parameters by means of creating touch files; the complete list is shown below. For catalyst implementation, the two parameters of interest are:

SIZE_DATA_BUFFERS_DISK

NUMBER_DATA_BUFFERS_DISK

Set:

SIZE_DATA_BUFFERS_DISK and NUMBER_DATA_BUFFERS_DISK to 262144 (256K) and 30 respectively (Default) on the Oracle servers.

But on media server called media that is doing all the VM backups set:

SIZE_DATA_BUFFERS_DISK and NUMBER_DATA_BUFFERS_DISK to 512 and 262144 (256K) and found we obtained better throughput.

You have to be careful because the Oracle media servers are the Oracle application servers as wellso wed not want to starve the application of resources during backup. The media server however is dedicated just to backup of the Windows VMs and we can use more and larger size buffers.


20

The full range of NetBackup tuning parameters is shown below:

On Windows systems, the above parameters can be located in \Program Files\Veritas\NetBackup\db\config

For UNIX/Linux systems, the parameters are in the locations below by creating the specific touch files

echo 262144 > /usr/openv/NetBackup/db/config/SIZE_DATA_BUFFERS

echo 256 > /usr/openv/NetBackup/db/config/NUMBER_DATA_BUFFERS

echo 1048576 > /usr/openv/NetBackup/db/config/SIZE_DATA_BUFFERS_DISK

echo 512 > /usr/openv/NetBackup/db/config/NUMBER_DATA_BUFFERS_DISK

echo 262144 > /usr/openv/NetBackup/db/config/SIZE_DATA_BUFFERS_FT

echo 16 > /usr/openv/NetBackup/db/config/NUMBER_DATA_BUFFERS_FT

echo 128 > /usr/openv/NetBackup/db/config/CD_NUMBER_DATA_BUFFERS

echo 524288 > /usr/openv/NetBackup/db/config/CD_SIZE_DATA_BUFFERS

touch /usr/openv/NetBackup/db/config/CD_WHOLE_IMAGE_COPY

echo 180 > /usr/openv/NetBackup/db/config/CD_UPDATE_INTERVAL

echo 1500 > /usr/openv/NetBackup/db/config/OST_CD_BUSY_RETRY_LIMIT

echo 1048576 > /usr/openv/NetBackup/NET_BUFFER_SZ

echo 1048576 > /usr/openv/NetBackup/NET_BUFFER_SZ_REST

The Symantec NBU 7.6 Tuning Guide can be found here symantec.com/business/support/index?page=content&id=doc7449

At this stage we assume the reader is familiar with configuring policies, adding media servers to the master server, deciding which media servers can access which storage units etc. If not, please read the Symantec NetBackup Administration Guide. symantec.com/business/support/index?page=content&id=DOC6452

Configuring Catalyst stores on HP StoreOnce 4700 and then integrating them into NetBackup Below are the steps to configure StoreOnce Catalyst store to operate in a low or a high bandwidth mode.

1. Login to the HP StoreOnce GUI and navigate to the StoreOnce Catalyst/storeshere you can see all the stores we created for this investigationlets create one more to show the integration process. Click Create in the top right hand corner.


21

We will create a Catalyst store called TEST and will make it Low Bandwidth by setting both transfer policies to Low Bandwidth (if we wanted High Bandwidth, we would set both transfer policies to High Bandwidth).

Click Create in the bottom right hand corner.

The store is created (see below) HP StoreOnce offers selective client (media server) access to Catalyst stores. This is explained in more detail in the HP StoreOnce and Symantec NetBackup integration guideavailable through HP Pre-sales on request. For now we will focus on the integration with NetBackup and have set No Client access permission setting in the Permissions tab.

We will need the IP address of the Catalyst store to configure it into NetBackupthis 4700 has 3 IP address, the 10GbE address is 172.28.6.10 and can be found as shown below in the StoreOnce GUI.


22

Now let us turn to NetBackupNetBackup has a construct as shown in Figure 12the Catalyst store we have created interfaces through a construct known as a Storage server in NetBackup, the storage server can be split into several Disk Pools which in turn can create different Storage units which are then accessible via a backup policy.

Figure 12. Symantec OST stack

We first created and added the Storage server, from credentials in the left hand navigation. Right-click new on the Storage servers in the Left Hand Navigation pane to create a new storage server as shown below.


23

Click Next and select OpenStorage from the drop down list of NetBackup supported disk-based backup targets, click Next.

The next screen is importanthere we enter the IP address of the 10GbE data network on our StoreOnce unit along with the unique Storage server type (as installed via the OST plugin)note the syntax hp-StoreOnceCatalyst. Then we select a media server that can initially access this storage server (we can add more later) and finally we must enter some credentials. If we are using Client access permission checking, these must align with the Clients and passwords we have created on the StoreOnce. In this example, we did not enable Client access permission checking on StoreOnce. Click Next.


24

We can now select other media servers, which are allowed to access this storage server. In this TEST example case, it is all the RAC Cluster nodes in both Bender and RAC1. Click Next and the Storage server is created and we are automatically sent to the Disk Pool creation Wizard.


25

Click Next and choose the type of Disk Pool to createin our case hp-StoreOnceCatalyst, click Next.

Select the appropriate storage server then click Next.

The Disk Pool Wizard has discovered all the Logical Storage Units (LSUs) on the Storage serverwe can see the new one TEST, which we have created. Click Next and give the Disk Pool a name as shown below.


26

In this example, we have decided to limit the number of streams to this Disk Pool to 24, as this was determined as the sweetspot for maximum throughput. Click Next and Disk Pool is created and the Storage Unit Wizard is automatically enabled.


27

Provide a name for the storage unit and again set a value for the maximum number of concurrent streams allowed to access this Storage unit. Click Next and the Storage Pool is created successfully and can be used in Backup Policies as seen below.

The above example has shown the creation of a TESTSTORAGEUNIT for demonstration purposes. In the main investigation, we created separate Catalyst stores for the Bender and RAC1 Oracle clusters as well as separate Storage unit for the redo transaction logs. The Storage units are then referenced in the Policies, which control the backup Scheduling and advanced features.

Contents of a Catalyst store explained In the Overall Solution diagram (Figure 1), you can see we created some High bandwidth and Low bandwidth catalyst stores as the backup targets for the solution. This section explains what the contents of these Catalyst stores actually looks like when viewed through the StoreOnce GUI.

It is important to understand what a Catalyst store looks like as opposed to a VTL or NAS-based backup target and the format with which Symantec NetBackup writes to a Catalyst store. In the StoreOnce GUI, select Catalyst stores and then select the Catalyst store you want to inspect (in this case BenderBackupHB) from the top part of the window. Select the data Jobs tab and you will see all the entries in the Catalyst store. The contents of a Catalyst store are known as Items and the item name is assigned by NetBackup; F1 means fragment one and if multiple fragments are required (there will be more) and each item is accompanied by a unique NetBackup header. You can clearly see in the example below the different


28

items that have come from the different media servers as part of our load balanced backup. The Client identifier is also clearly listed as an attribute as well as the throughput being achieved and data volumes written.

Click on an individual item and more information is displayed in the bottom pane. In the highlighted item, a deduplication ratio of 64.8 is being achieved. Bandwidth saving is 0 percent because this is a high bandwidth store we have createdto reduce the deduplication load on the Oracle application servers/media servers.

Catalyst stores configured in this investigation Table 2. Catalyst store names configured on StoreOnce 4700

Catalyst store name Usage Comments

BenderBackupHB Used for Bender Backups (1 node or 4 node) HB = High Bandwidth Catalystused in mainstream testing

BenderBackupLB Used for experimenting with Low Bandwidth Catalyst Backups

LB = Low Bandwidth Catalyst

BenderRedoLogsHB Separate Catalyst store for archive redo logs Archive redo logs are transient and do not deduplicate well so they were configured to use a separate store to enhance dedupe ratios of Oracle data files store (BenderBackupHB)

RAC1BackupHB Used for RAC backups (1 node or 3 node) N/A

RAC1BackupLB Used for experimenting with Low Bandwidth Catalyst Backups

N/A

RAC1RedoLogsHB Separate Catalyst store for archive redo logs N/A

Accelerator Used for all the Windows 2012 R2 VM backups Again different data types demand a separate datastore

You can also view the deduplication ratios through this GUI; for our Oracle RAC databases we were getting typically 8:1 and for our Windows VM backups with NetBackup Accelerator, the deduplication ratio was around 10:1 over a period of a week of full backups with either HammerDB creating data change on the Oracle servers or HP Create Data creating data change on the Windows VM machines. Deduplication ratios vary according to the data change rate and retention period.


29

Symantec NetBackup Oracle RAC 12c integration

For more details on Symantec best practices for Oracle RAC backups, visit: symantec.com/business/support/index?page=content&id=HOWTO89685

Symantec NetBackup provides a policy mechanism for setting up backup definitions, these contain various Policy typeslike VMware and Oracle, along with Backup Schedules, Storage Units to be used (backup targets), and any special features for that particular Policy Type such as backup configuration parameters for Oracle. When it comes to Oracle, Symantec NetBackup offers intelligent backup policy (automated scripting, but only for single Oracle instance) and Legacy policy (Symantec script templates) for Oracle RAC clusters.

For customers with non-RAC (single instance) Oracle environments, the Oracle Intelligent Policies (shown later in this document) offer an easy-to-use interface to the backup and restore of non-RAC Oracle databases. This is shown for the sake of completeness because some HP BladeSystem customers may not use Oracle RAC and so can take advantage of the Oracle Intelligent Policies. Oracle Intelligent Policies generate RMAN scripts to use at run time, and so eliminates the need for creating scripts on the Oracle client system.

Test Bed architecture overview

Figure 13. HP BladeSystem, Oracle RAC, and Symantec NetBackup Test Bed architecture overview

The Test Bed consisted of one HP BladeSystem connected to HP 3PAR StoreServ 7450 and HP StoreOnce 4700 connected via a 10GbE network. Two 10GbE links were bonded to provide redundancy and a large pipe. Nine VMs were created on four of the HP BladeSystem blades. Seven VMs were dedicated to the two Oracle clusters; across the four ESXi hosts on the HP BladeSystem. Each Oracle VM has 16 virtual CPU (vCPU) and 96 GB of memory. One VM is used for Symantec NetBackup Master server and another for NetBackup media server for VM backups and these NetBackup servers have 16 vCPU and 32 GB of memory provisioned. The two Oracle databases and VMware datastores reside on the HP 3PAR StoreServ 7450.


30

Solution components This solution includes the following key components:

HP BladeSystem tested configuration Table 3. HP BladeSystem hardware for the Test Bed

Component Purpose

One HP BladeSystem c7000 Enclosure Enclosure to host blades and Virtual Connect modules

Two HP Virtual Connect FlexFabric-20/40 F8 Virtual Connect module for Ethernet and SAN connectivity

Six HP ProLiant BL460c Gen9 Server Blades Server blade to host two Oracle RAC 12c 4-node clusters, Simpana application, and HammerDB application running as VMs

One HP 3PAR StoreServ 7450 Storage for Oracle database and datastores for VM

Two HP StoreFabric SN6500B 16GB 96/48 FC SAN switches

FC switches for SAN connectivity between servers and HP 3PAR

HP StoreOnce 4700 Target for Oracle database backup

Two HP FlexFabric 5930-24G switches.

Two HP 5130-24G EI switches

10GbE top-of-rack switches Ethernet switches

Software configuration Oracle RAC 12c configured on VM running Red Hat Enterprise Linux 6.5

Oracle Automated Storage Management (ASM)

Oracle Recovery Manager (RMAN)

Symantec NetBackup 7.6.0.3 Master server configured on Windows 2012 R2

Symantec NetBackup 7.6.0.3 Media server configured on Windows 2012 R2

Symantec Media server installed on All Bender and RAC1 nodes so they could act as media servers

Symantec Client software installed on all VM machines to be backed up

HammerDB 2.16 configured on VM running Red Hat Enterprise Linux 6.5

HP 3PAR OS version must be 3.1.3 and later

HP StoreOnce 4700 release 3.12 featuring reporting central and phone home capabilities

Symantec NetBackup Ops Center (reporting add on) was also installed to prove it works in a fully virtualized environment but was never used in producing reports

Storage configuration The HP 3PAR StoreServ 7450 hosted the two Oracle databases of 1 TB each. To accomplish this layout, the HP 3PAR StoreServ 7450 storage array is used to allocate and present storage pools as described table 4 once allocated, the storage objects are presented to the VMware vSphere environment. The storage is allocated and provisioned using a thick format to enhance I/O performance during the workload-testing phase of this solution.


31

Table 4. Storage configuration for the Test Bed

Contents Size Provisioning Notes

Oracle data files 1 TB Thick RAID 5

Oracle redo logs 500 GB Thick RAID 5

VM OS, binaries specifications

150 GB Thick RHEL 6.5, Oracle 12c (16 core and 96 GB memory)

NetBackup 7.6.0.3 Master server and Media server

200 GB Thick Windows 2012 R2

HammerDB 50 GB Thick RHEL 6.5, HammerDB 2.16

HP StoreOnce 4700 configuration The StoreOnce 4700 for this evaluation was configured with two disk shelves. Each disk shelf has fourteen 2 TB disk drives. The maximum throughput of a fully configured StoreOnce 4700 without Catalyst is 7.6 TB/hr. The unit we tested had a maximum throughput of 1.5 TB/hr due to the limited number of disk drives (28). Performance is affected by parallel stream count; more streams gives higher performance. The StoreOnce unit included the following:

2x 10GbE bondedCatalyst backup target

4x 1GbEManagement

2x 8 Gb FCUnused

StoreOnce software version 3.12 featuring new reporting central and Phone home capabilities

Workload HammerDB is an open source database load testing and benchmarking tool for Oracle and other databases.

HammerDB is used to load the data into the two Oracle databases. Transactions were run against the databases during backup to simulate a customer production environment. The load on the system would be qualified as medium and would consume 45 percent of the CPU. The daily data change rate is estimated between 1.8 percent and 2 percent.

Figure 14. Database data load using HammerDB


32

Figure 15. HammerDB transaction running against the Oracle database

Optimizing backup and recovery for Oracle RAC 12c on HP BladeSystem with Symantec NetBackup

StoreOnce tuning guidelines Make use of the HP StoreOnce Sizing tool to size your StoreOnce solution. It is available at:

h30144.www3.hp.com/SWDSizerWeb/default.htm.

Always ensure that the appliance software in your HP StoreOnce Backup System is fully up-to-date. Software upgrades also contain all the necessary component firmware upgrades.

Where possible, group backups of like data types to the same destination device (Share/VTL/Catalyst Store)This can help optimize deduplication ratios.

An informed decision should be made as to whether High Bandwidth Catalyst or Low Bandwidth Catalyst stores should be usedThis mainly depends on the CPU capabilities of the application servers and/or dedicated media servers.

Run multiple backup streams in parallel to improve aggregate throughput for a StoreOnce appliance.

Use blackout windows and replication windows to ensure that the appliance is not concurrently performing backup, replication, housekeeping, and offload to tape operations. This can keep system performance consistent throughout the backup period.

Configure multiple Ethernet ports in a network bond to achieve increased available network throughput.

Identify and resolve other performance bottlenecks in your backup environment such as slow clients and media agents.

Experiment with the ISV tuning parameters (in this case, the number and size of data buffers) to obtain best throughput being mindful of application resources that also may be required from the same server.


33

Networking The recommendation is to configure the StoreOnce with bonded 10GbE connections where possible to allow increased throughput. Multiple Clients and Media Agents can write to the StoreOnce simultaneously reducing the total backup window required.

The number of streams required to get best throughput really depends on the performance of the hosts supplying the data. Maximum ingest can be reached and a fairly low stream counts if the host can supply data fastotherwise the stream count increase to attain the maximum ingest rate on the StoreOnce unit. Table 5 shows some best case throughput rates.

Table 5. Maximum supported streams and devices

StoreOnce Devices Streams Low Bandwidth Catalyst maximum throughput5, 6

VSA 4 16 600 GB/hr

2700 8 48 2.63 TB/hr

4500 32 128 10.8 TB/hr

4700 50 192 15.2 TB/hr

4900 50 320 17.05 TB/hr

6500 50 320/node 126.4 TB/hr (8 nodes)

Replication When using StoreOnce Catalyst with replication, you can configure NetBackup to actually replicate the Catalyst stores to

another site for DR purposes. This is achieved through the Symantec NetBackup concept of Storage Lifecycle policies. See symantec.com/business/support/index?page=content&id=HOWTO73205.

More information on StoreOnce replication and its configuration is available at: hp.com/go/storage/docs.

An example of Symantec NetBackup controlling StoreOnce replication is shown belowthe first operation is the backup and the second operation is the Duplication (Catalyst copy) between NetBackup systems in the same NetBackup Domain to another StoreOnce unitall this is controlled by Symantec NetBackup itselfso it is aware of all copies of the backupsthis makes for much more efficient recovery from disasters because there is no need for time consuming import operations. To replicate catalyst stores between StoreOnce devices in different NetBackup domains and additional NetBackup feature (also supported on HP StoreOnce) called Targeted Auto Image Replication (AIR) is required.

5 These figures are correct at the time of publication, but are subject to change with differing software versions. 6 These figures are headline performance figures, generated with extremely high performance clients to show raw StoreOnce ingest performance.


34

NetBackup tuning guidelines

Number of clients A larger number of clients allows for a greater aggregate throughput to StoreOnce, shrinking the backup window required. Please see table 5 for supported streams and total throughput on each StoreOnce platform; this should be considered during the sizing exercise.

Size and number of buffers to be used in NetBackup (media server) This has been explained previously.

Media Agent sizing The Media Agent is involved in all data operations between the clients and StoreOnce; it needs to be capable of supporting sufficient concurrent operations. When using Catalyst Low Bandwidth, care must be taken to understand the extra load required on the media/application serveras a general rule of thumb, use the formula shown below.

Figure 16. HP StoreOnce Catalyst best practice-backup server requirements

Oracle tuning parameters During the course of this investigation, we managed to improve restore performance by 10 percent by tuning Oracle buffers using the sqlplus command shown below. The default values were both zero.

alter system set _backup_ksfq_bufcnt=1;

Oracle Buffer Tuning for restores SQL> alter system set _backup_ksfq_bufsz=131072; oracle.com/technetwork/database/features/availability/s316928-1-175928.pdf

Oracle/Symantec Backup types Symantec NetBackup can run various backup types to protect Oracle data: Full, Cumulative Incremental, and Differential Incremental. Regular Full backups can deduplicate very efficiently and allow immediate restores. Online full backups place a significant load on the client as it streams the entire data set for each backup.

For Oracle database, a Schedule Policy containing regular differential incremental backups and less frequent full backups can often result in excellent balance of performance, StoreOnce disk use, and client load. The downside to such a policy is that restoring may require the restore of a full backup and subsequently several differential incremental restores in sequence.


35

Backup configuration For the test configuration, a cycle of full (weekly) and incremental (daily) backups were scheduled. Each cycle represents a week and day of transactions and growth in the Oracle database. The daily data rate change was between 2 percent and 3 percent.

Full backups were scheduled weekly during low activity. Nightly incremental backups were scheduled to run in between the full backups. Oracle Block Change Tracking was enabled for an optimized incremental backup. Oracle archive redo log backups were configured to run automatically with the full and incremental backups to enable point-in-time recovery. Because of the way Symantec NetBackup uses RMAN scripts, the incremental backups were configured as shown below:

Configured to run daily (under Schedule tab)

Uses a modified script/bender-full-archive-inc.sh, which uses the new RMAN Parameter INCREMENTAL=1 to set up the backup as an incremental

Under these circumstances, only the data file blocks that have changed and archive redo logs generated under HammerDB load are backed up instead of the whole database.

Configuring Symantec NetBackup with Oracle

Symantec provides two main methods of protecting Oracle database in a streaming mode utilizing the RMAN (Recovery manager feature) integrated into Oracle itself. One method is called Oracle Intelligent Policies (automated scripting generation) or Legacy policy type (custom scripted using starting templates supplied by Symantec). Depending on which method is used, the contents of the Backup Selections tab in the NetBackup Policy GUI may vary. Furthermore, the Intelligent Policies are designed more for single instance Oracle protection and are not supported with Oracle RAC. For the sake of completeness of this white paper, we will show both methods in case customers want to use the Oracle Intelligent Policy method for single instance (non RAC) Oracle implementations.

Both methods start off the sameselecting the Oracle policy type in the policy Wizard, along with the NetBackup storage unit to be used for the backups (in this case BenderBackupHBSU).


36

On the Schedules tab, we can set the window when the Oracle intelligent policy could run along with the type of backup; media multiplexing should be set to 1 (only used on physical tape).

Selecting Instances tab, we can select instance or instance groups or select Clients for use with scripts or templates. Selecting instances or instance groups forces this to become an Oracle Intelligent Policy type.

Once the instance is selected, the Oracle Intelligent policies causes the Backup Selections tab to be populated as shown below to select different components for backup.


37

And also for Oracle Intelligent policies, the final Oracle tab controls various backup settings.

These parameters input through the GUI are used to populate the RMAN script that Symantec Intelligent Oracle Policies will use. Below is an explanation of the various parameters. In the manual scripting method, we will move all these parameters into the RMAN scripts.

Number of parallel channelsControls the number of streams sent to the StoreOnce unit. This is calculated by taking total number of data files and dividing by the Files per set value. In this example, Bender 1 consists of approx. 120 data files.

Number of open filesThis is the number of files open in memory at any one time for NetBackup/RMAN to access. This is a compromiseif filesperset = 1 the database is backed up with data files in the exact same order every time meaning we get a good dedupe ratio, but we starve the NetBackup/RMAN backup engine of data so the performance is slower. A compromise value of five is chosen to keep the backup engine supplied with data whilst not decreasing the dedupe ratio too much. The five files in memory can be picked out in any order so the pattern of sent to StoreOnce is not guaranteed to be the same every time.

Files per backup setHow many database files are packaged into a backup piece.

Archived redo logsGenerally these are backed up well to enable a complete or point-in-time (incomplete) database recovery. The archive logs backup can take a long time to backup if only a single channel is enabled, so we have enabled 16 channels for archive redo log backups. With Oracle Intelligent policies, the archive redo logs and database files are sent to the same storage unitwhich means dedupe ratio could be reduced. A better practice is to send database files to one Catalyst storage unit and archive redo log files to a different Catalyst storage unit (this can be done in the Legacy scripting method described later).

User specified file formatsThese again are translated into an RMAN scripting command that names the various Oracle files to suit the customers requirements.


38

A recovery catalog is a database schema used by RMAN to store metadata about one or more Oracle databases. Typically, you store the catalog in a dedicated database. A recovery catalog provides the following benefits:

A recovery catalog creates redundancy for the RMAN repository stored in the control file of each target database. The recovery catalog serves as a secondary metadata repository. If the target control file and all backups are lost, then the RMAN metadata still exists in the recovery catalog.

A recovery catalog centralizes metadata for all your target databases. Storing the metadata in a single place makes reporting and administration tasks easier to perform.

A recovery catalog can store metadata history much longer than the control file. This capability is useful if you have to do a recovery that goes further back in time than the history in the control file. The added complexity of managing a recovery catalog database can be offset by the convenience of having the extended backup history available.

Some RMAN features function only when you use a recovery catalog. For example, you can store RMAN scripts in a recovery catalog. The chief advantage of a stored script is that it is available to any RMAN client that can connect to the target database and recovery catalog. Command files are only available if the RMAN client has access to the file system on which they are stored.

A recovery catalog is required when you use RMAN in a Data Guard environment. By storing backup metadata for all primary and standby databases, the catalog enables you to offload backup tasks to one standby database while enabling you to restore backups on other databases in the environment.

Because RAC is an Enterprise implementation of Oracle, we decided to use a Recovery Catalog in our data protection strategy for the reasons shown above. If we had chosen not to use Recovery Catalog, we would have added an extra line to the RMAN scripts in Appendix A.

Add line

RMAN> backup current controlfile; Removed the reference to RECOVERY_CATALOG_STR=rman/cat@cat Oracle Intelligent policies are not RAC Aware and with the Bender & RAC1 RAC Clusters, we decided to use the more traditional Legacy Oracle RMAN scripting methodfavored by Oracle DBAs the world over.

With the Oracle Legacy backup scripted method supported in NetBackup, we can script the solution using RMAN to have every node in the RAC cluster contributing to the backup (so called load balanced backup)which means the backups will run fasterbut transactions may run slower whilst the backup is proceeding. Using Oracle Intelligent policies, the number of parallel channels option is basically the number of streams the database is delivered to the backup target in, for instance.

The Legacy scripting method utilizes the NetBackup process bphdb to run the script as root on the database node. From this node the script connects to all of the nodes in the cluster through Oracle Network Services and the progress is reported through the Symantec Activity monitor. For Restore using the Legacy method or Oracle Intelligent Policiesthere is no invocation from the Symantec GUI, instead the restore script has to be run from one of the RAC nodesbut again the progress is reported through the Symantec Activity monitor.


39

Below, we see the Policy tabs for the Oracle Legacy scripted method for Oracle RAC backups using a two policy approach.

Policy 1bender-full-archive-run starts the RMAN script which in turn calls the Bender_2nd_Policy using the NB_ORA_POLICY feature in RMAN that allows Symantec NetBackup parameters to be passed into the RMAN script. See Appendix A for details. We will now work through the GUI/RMAN script process for Legacy scripted backups, which are the best way of backing up Oracle RAC Clusters with Symantec NetBackup. On the Attributes tab we select the Oracle Policy type and BenderBackupHBSU storage unit.

The Schedules tab in Policy bender-full-archive-run is standard but is split into AUTOMATIC and Default-Application backups. This is normal and cannot be changed for this Policy type. When your schedule runs, it activates what is in effect a user backup on the client which then runs its script and that script uses the default application backup schedule to actually run. Without it, the backup will not run/work. Note that the job also takes its retention period from the default application schedule and not your schedule.


40

In the Clients tab for bender-full-archive-runinstead of selecting instances as we would for an Oracle Intelligent Policy backup, select Select Clients for use with scripts or templates and we enter the vip hostname (failover address) of the RAC clusterin this example bender 1-vip as shown below.

On the Backup Selections tab, we point to an RMAN script location as shown below (the script is derived from templates supplied by Symantec).

At this point, it is advisable to look at the highlighted parts of the sample scripts in Appendix A for this solutionthey are the actual scripts used.

Note in particular the references in the scripts in Appendix A to Bender_2nd_Policy, which is passed into the RMAN script through the tight integration or Oracle API and Symantec Librariesthis is a key part of the installation process.

Bender_2nd_Policy has been defined through NetBackup but linked to bender-full-archive-run via the RMAN script itself.

In the Bender_2nd_Policy tabs, we have the following:

Attributes tab as expected


41

In the Schedules tabwe have integrated both aspects of our RMAN script, which runs both Database backup and archive redo log backups under the titles of BENDER_FULL and BENDER_LOGSwhich have to match what is in the RMAN script.

The benefit of doing this (using two different schedules) is that we can also override the policy storage unit and use a different one for this schedule. If we click on BENDER_LOGS, we can see for the archive redo logs we are using Policy Storage BenderRedoLogsHBSU.

In the Clients tab of Bender_2nd_Policy, we select select clients for use with scripts and templates we want a load balanced backup across all four bender RAC nodesso all are specified as clients in Bender_2nd_Policy.


42

Finally in Bender_2nd_Policy, there is nothing in the Backup selections tab because the RMAN Script is being called by Bender-full-archive-run policy and Bender_2nd_Policy is only receiving the backup data.

And so that is how we configure Legacy Oracle Backups in Symantec NetBackup. The screenshot below shows a completed job.

Note How each client (RAC node) is acting as its own media server and backing up six streams of data with the six streams of

redo logs.

How although we started the job by running policy bender-full-archive-runthe policy doing all the work is Bender_2nd_Policy, which is called from within the RMAN script shown in Appendix A.

The BENDER_FULL and BENDER_LOGS schedules both running one after the other.

Note In the above screenshot, the two policies which are linked together by the RMAN script those are Bender-full-archive-run and Bender_2nd_Policy. Also, note all four bender nodes partaking in the backup (load balanced) supplying six streams of data each to the BenderBackupHBSU Catalyst target. The same process was used for the RAC1 Cluster.

You can also see in the above screenshot that schedule BENDER_LOGS also runs on each nodethis is because Oracle redo logs are produced on all nodes whereas the Oracle database can be run only on selected nodes as the database files are accessible for all the nodes.


43

Investigation results

Backup use cases Three Oracle backup use cases were investigated whilst the HammerDB loading was runningto simulate the real world. The backups also included the archive redo logsas is also typical in the real world. The throughput values below were calculated by Total Data Transferred divided by Total time taken as reported by NetBackup. Because not all data files are the same size and the archive logs are small and single stream, the throughput values do not reflect the Peak sustained throughput observed using the Reporting central feature in HP StoreOnce. These values are shown in brackets (). Each Database had its own catalyst High Bandwidth store created on StoreOnce.

1. An online streaming full backup was taken utilizing a single Oracle RAC node with 24 RMAN channels allocated for the backup. The average throughput for this case was 1.43 TB/hr (3.96 TB/hr peak sustained).

2. Spreading the workload over all four Oracle RAC nodes for Bender database, increased the average throughput to 2.2 TB/hr (6.84 TB/hr sustained), while also spreading the CPU demands of the backup across all four Oracle RAC nodes.

3. Finally, two parallel database backups (Bender and RAC1) were run to the StoreOnce, with 12 RMAN channels allocated for each database. The average throughput for both databases was 2.5 TB/hr (7.92 TB/hr) peak sustained.

Table 6 shows the per

hp bladesystem

Documents