Virtual Infrastructure for SAP Landscape Enabled
by EMC CLARiiON CX4-480 and VMware Distributed Resource Scheduler
A Detailed Review
EMC Information Infrastructure Solutions
Abstract
This white paper describes the design, deployment, and validation of an SAP landscape in a VMware virtualized environment incorporating VMware® Distributed Resource Scheduler (DRS) as the load-balance technology on EMC® CLARiiON® CX4.
December 2010
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and
VMware Distributed Resource Scheduler—A Detailed Review 2
Copyright © 2010 EMC Corporation. All rights reserved.
EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.
For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.
VMware is a registered trademark of VMware, Inc. All other trademarks used herein are the property of their respective owners.
Part number: H8048
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and VMware Distributed Resource Scheduler—A Detailed Review
3
Table of Contents
Executive summary ........................................................................................................................... 5 Overview ....................................................................................................................................... 5 Product solution............................................................................................................................. 5 Key results ..................................................................................................................................... 5
Introduction........................................................................................................................................ 6 Overview ....................................................................................................................................... 6 Purpose ......................................................................................................................................... 6 Scope ............................................................................................................................................ 6 Audience ....................................................................................................................................... 6 Terminology ................................................................................................................................... 7
Key components ............................................................................................................................... 8 Introduction .................................................................................................................................... 8 EMC CLARiiON CX4 ..................................................................................................................... 8 VMware vSphere ........................................................................................................................... 8 SAP ERP IDES ............................................................................................................................. 8 HP LoadRunner............................................................................................................................. 8
Physical architecture ......................................................................................................................... 9 Overview ....................................................................................................................................... 9 Physical environment .................................................................................................................... 9 Environment profile ..................................................................................................................... 10 Hardware resources .................................................................................................................... 10 Software resources ..................................................................................................................... 11
Installation and configuration .......................................................................................................... 12 Introduction .................................................................................................................................. 12 Installing VMware vSphere ......................................................................................................... 12 Configuring the VMware cluster .................................................................................................. 13 Installing SAP systems ................................................................................................................ 21 Configuring SAP systems ........................................................................................................... 23 Configuring LoadRunner ............................................................................................................. 24
Design and validation ...................................................................................................................... 26 Introduction to design and validation........................................................................................... 26 Test methodology ........................................................................................................................ 26 Test scenarios ............................................................................................................................. 27 Test procedure ............................................................................................................................ 31 Results analysis .......................................................................................................................... 32
Conclusion....................................................................................................................................... 38 Summary ..................................................................................................................................... 38 Key points .................................................................................................................................... 38
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and
VMware Distributed Resource Scheduler—A Detailed Review 4
References ...................................................................................................................................... 39 White papers ............................................................................................................................... 39 Product documentation ............................................................................................................... 39 Other documentation ................................................................................................................... 39
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and VMware Distributed Resource Scheduler—A Detailed Review
5
Executive summary
Overview Effective distribution of an SAP workload during heavy use periods such as month-
end close, peak seasonal demands, or nightly batch processing is challenging for many SAP technical teams.
This white paper describes a method to significantly improve SAP response times during peak workloads by leveraging the automation and simplicity of VMware® Distributed Resource Scheduler (DRS).
Product solution
A common and realistic customer environment in the EMC lab is used to validate the benefits of using VMware DRS for SAP.
This solution demonstrates how DRS can dynamically and efficiently manage the resource allocation across physical servers to maintain or improve the SAP system performance.
DRS affinity rules are used to comply with service level agreements for the simulated SAP business processes.
This solution does not provide a comprehensive guide to every aspect of an SAP business-ready solution.
Key results A “before and after” comparison demonstrates that DRS in an SAP landscape
significantly increases the operational efficiency for SAP applications. Specifically, it provides the following benefits:
• Automated real-time response to changing performance conditions increases the operational efficiency of SAP applications.
• DRS balances hosts with CPU utilization at maximum to prevent VMware ESX® server CPU resource bottlenecks and to provide improved SAP response times.
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and
VMware Distributed Resource Scheduler—A Detailed Review 6
Introduction
Overview This solution uses an SAP ERP system landscape consisting of a development
system, a quality assurance system, and a production system in a VMware cluster environment. DRS is enabled in VMware vCenter™.
This solution demonstrates:
• Starting an SAP production system in a VMware cluster environment
• Enabling DRS to nondisruptively distribute the CPU load in the VMware cluster when the SAP processes remain running on all production instances
• Meeting the resource compliance requirements while improving hardware utilization in a three-system SAP landscape
Purpose The purpose of this solution is to validate the use of VMware DRS to aggregate
computing capacity across a collection of servers. Logical resource pools are used in the context of a typical SAP system landscape to intelligently allocate available resources among the virtual machines based on pre-defined rules that reflect business needs and changing priorities.
Scope The scope of this solution is bound by the following objectives:
• Validate that SAP load balancing can be complemented by DRS to manage system resource allocation across physical servers automatically and efficiently.
• Demonstrate that DRS with affinity rules can meet the compliance requirements. The following policies are set as a recommended practice for the SAP environment:
− Dedicated resource to the production system cannot be shared with non-production systems.
− Servers hosting non-production systems can offer unallocated resources to the production system.
Audience This white paper is intended for SAP administrators and system administrators who
are involved in planning, architecting, or administering a virtualized environment with EMC® CLARiiON® storage and VMware vSphere™ infrastructure. The audience should have basic knowledge of SAP administration and VMware resource management.
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and VMware Distributed Resource Scheduler—A Detailed Review
7
Terminology This section defines the technical terms used in this document.
Term Definition
DEV DEV is the abbreviation for the SAP development system.
QAS QAS is the abbreviation for the SAP quality assurance system (test system).
PRD PRD is the abbreviation for the SAP production system.
vCPU vCPU represents the virtual CPU that is set within the virtual machine and links to a physical CPU core.
Datastore Datastore is a VMware vSphere storage location for virtual machine files.
ESX ESX is a VMware hypervisor architecture built on compatible physical servers. It provides the foundation for building and managing a virtualized IT infrastructure.
Virtual machine Virtual machine is a tightly isolated software container that can run its own operating systems and applications as if it were a physical computer.
VMDK VMDK stands for virtual machine disk format files; it is a datastore descriptor.
VMFS VMFS is a VMware virtual machine file system. It is a cluster file system that enables multiple VMware ESX hosts to concurrently access a single VMDK file.
VMware cluster VMware cluster is a collection of ESX/ESXi hosts and associated virtual machines with shared resources and a shared management interface.
Resource pool A resource pool is a logical abstraction for flexible management of resources. Resource pools can be grouped into hierarchies and used to hierarchically partition available CPU and memory resources.
VMware DRS VMware Dynamic Resource Scheduler (DRS) aggregates compute resources across many clusters and dynamically allocates them to virtual machines based on business priorities, reducing management complexity through automation.
VMware vMotion VMware vMotion® technology uses the complete virtualization of servers, storage, and networking to move an entire running virtual machine instantaneously from one server to another.
Affinity rule Two types of affinity rule are used in VMware DRS cluster: A “virtual machine to host” affinity rule specifies whether the members of a selected virtual machine DRS group can run on the members of a specific host DRS group. A “virtual machine to virtual machine” affinity rule specifies whether the selected individual virtual machines should run on the same host or separate hosts.
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and
VMware Distributed Resource Scheduler—A Detailed Review 8
Key components
Introduction This section briefly describes the key solution components. For details on all of the
components that make up the architecture, see the Environment profile section in this paper.
EMC CLARiiON CX4
The EMC CLARiiON CX4 series deliver industry-leading innovation in midrange storage with the fourth-generation CLARiiON CX™ storage platform. The unique combination of flexible, scalable hardware design and Advanced Storage Efficiency products enables the CX4 series system, powered by Intel Xeon processors, to meet the growing, diverse needs of today's midsize and large enterprises.
CLARiiON CX4 can automatically balance performance and capacity by optimizing storage investment and addressing data's dynamic requirements.
VMware vSphere
VMware vSphere is the foundation for a dynamic, self-optimizing IT infrastructure. It is a robust, production-proven virtualization layer that abstracts processor, memory, storage, and networking resources into multiple virtual machines. VMware vSphere allows enterprises to dramatically reduce hardware and operating costs by sharing resources across a virtual environment.
SAP ERP IDES SAP ERP IDES is the "Internet Demonstration and Evaluation System" in the ERP
Central Component (ECC) system and represents a model company. It consists of an international group with subsidiaries in several countries. IDES contains application data for various business scenarios that can be run in SAP systems. The business processes in the IDES system are designed to reflect real-life business requirements and have access to many realistic characteristics. IDES uses easy-to-follow business scenarios to show the comprehensive functions of the ECC system. The focal point of IDES, however, is not the functionality itself, but the business processes and their integration.
HP LoadRunner Hewlett-Packard (HP) LoadRunner can help you prevent application performance
problems in production by detecting bottlenecks before a new system or upgrade is deployed. This software enables you to measure end-to-end performance, diagnose application and system bottlenecks, and tune for better performance—all from a single point of control. The integrated load test, performance test and application stress test features help you reduce the costs and time required to test and deploy new applications and systems into your production environment.
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and VMware Distributed Resource Scheduler—A Detailed Review
9
Physical architecture
Overview This section identifies and briefly describes the technology and components that are
used in the environment. The environment consists of VMware vSphere servers, which contain the entire infrastructure required to operate an SAP ERP 6.0 EhP4 testing bed, including the central instance, dialog instances, LoadRunner controller, and LoadRunner load generators. SAP ERP 6.0 EhP4 is hosted on a Windows/SQL Server platform. EMC CLARiiON provides flexible storage with high availability while ensuring adequate performance to the entire SAP system.
Physical environment
Figure 1 illustrates the overall physical environment of this solution. A customer’s environment may vary depending on the customer’s existing hardware and hardware preferences. Also, the customer’s environment would not include the servers or software used to simulate user load as in the EMC lab.
Figure 1. Physical architecture
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and
VMware Distributed Resource Scheduler—A Detailed Review 10
Note Figure 1 actually illustrates the environment design in the CPU overcommitted scenario. The CPU non-overcommitted scenario uses the same design with one more dialog instance (four dialog instances). For more details about the CPU overcommitted and non-overcommitted scenarios, refer to the Test scenarios section.
Environment profile
This solution is validated with the environment profile described in Table 1.
Table 1. Environment profile
Profile characteristic Quantity/Type/Size
SAP application SAP ERP 6.0 EhP4
SAP architecture The following SAP systems are included:
• PRD with CI+DB, DI x 3 (4*)
• DEV with CI+DB
• QAS with CI+DB, DI x 1
Storage CLARiiON CX4-480
Database type, size, and source 250 GB x 3 MSSQL, 2 GB data files with the IDES content
Virtualized platform VMware vSphere
SAP workload simulator HP LoadRunner
Note The CPU overcommitted test scenarios are conducted with three dialog instances while the CPU non-overcommitted scenarios use four dialog instances.
Hardware resources
Table 2 and Table 3 list the hardware resources used to validate this solution.
Table 2. Server configuration
Equipment Quantity Configuration
ESX server for SAP systems
4 • CPUs: 2 x quad-core, 2.4 GHz
• Memory: 36 GB
• HBAs: 2 x 8 G
• NIC: 1 Gb Ethernet
ESX server for HP LoadRunner
1 • CPUs: 4 x six-core, 2.4 GHz
• Memory: 128 GB
• HBAs: 2 x 8 G
• NIC: 1 Gb Ethernet
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and VMware Distributed Resource Scheduler—A Detailed Review
11
Table 3. SAN configuration
Equipment Quantity Configuration
CLARiiON
1 • Array type: CLARiiON CX4-480
• FLARE® release: 30
• Guest OS and LoadRunner disks - 4 x 300 GB FC 15k (RAID 1_0)
• SAP production disks - 9 x 300 GB FC 15k (RAID 5 8+1) for DB - 4 x 300 GB FC 15k (RAID 1_0) for logs
• Read Cache: 2,048 MB for each SP • Write Cache: 2,450 MB for each SP
SAN switch 1 FC
Software resources
Table 4 lists the software resources used to validate this solution.
Table 4. Software resources
Software Version
VMware vSphere 4.1 Build 258902
VMware vCenter 4.1 Build 258902
VMware Guest OS Windows Server 2008 Enterprise Edition SP1, x64
Microsoft SQL Server SQL Server 2008 Enterprise Edition SP1, x64
SAP application SAP ERP 6.04 / NW 7.01 IDES
HP LoadRunner 9.5
EMC CLARiiON FLARE 30.004
EMC PowerPath®/VE 5.4 SP1
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and
VMware Distributed Resource Scheduler—A Detailed Review 12
Installation and configuration
Introduction The following installation and configuration were used to create the test environment.
Additionally, some key considerations are also listed below:
• A VMware cluster was configured with DRS enabled or disabled to create a base resource aggregation.
• Resource pools were configured within the cluster to facilitate resource assignment to SAP systems based on its business requirements.
• Affinity rules were configured to group virtual machines and ESX hosts or to set different virtual machines apart.
• SAP systems and LoadRunner systems were configured appropriately to meet the requirement of a real business load simulation.
Installing VMware vSphere
Installing VMware vSphere 4.1 within the context of this solution involves the following steps:
Step Action
1 Install VMware ESX 4.1 on all servers.
2 Install VMware vCenter 4.1.
3 Apply licenses for ESX 4.1 and vCenter 4.1.
4 Provision the storage space for the SAP systems according to Table 5.
Table 5. Storage LUNs configuration
Drive Purpose LUN Datastore type
RAID type
C: O/S and Swap WIN_PRD_OS VMFS RAID 1_0 (2+2)
E: SAP executables WIN_PRD_BIN1 VMFS RAID 5 (8+1)
F: Temp DB WIN_PRD_TEMPDB VMFS RAID 5 (8+1)
G: Log WIN_PRD_LOG VMFS RAID 1_0 (2+2)
H: DB Data Volume 1 WIN_PRD_SAPDATA1 VMFS RAID 5 (8+1)
I: DB Data Volume 2 WIN_PRD_SAPDATA2 VMFS RAID 5 (8+1)
J: DB Data Volume 3 WIN_PRD_SAPDATA3 VMFS RAID 5 (8+1)
K: DB Data Volume 4 WIN_PRD_SAPDATA4 VMFS RAID 5 (8+1)
Note The DEV and QAS systems share the same configuration pattern with the production system in terms of the O/S and Swap disk.
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and VMware Distributed Resource Scheduler—A Detailed Review
13
For detailed information about installing VMware vSphere 4.1, refer to ESX and vCenter Server Installation Guide 4.1.
Configuring the VMware cluster
Follow these steps to configure the VMware cluster:
Step Action
1 Create a VMware cluster and then ensure that DRS is disabled at this moment. Note Generally, DRS is disabled by default.
2 Add the four ESX servers that are dedicated to SAP to the cluster you created in step 1.
3 Configure the resource pool:
a. As shown in Figure 2, create a new resource pool on the top level of the cluster. b. As shown in Figure 3, create sub-resource pools for the SAP production system. c. As shown in Figure 4, create sub-resource pools for the SAP non-production system.
4 Create DRS groups according to Table 6.
5 Configure the affinity rules:
a. As shown in Figure 5, create the affinity rules. b. As shown in Figure 6, define the placement rule for the production central instance and database instance. c. As shown in Figure 7, define the placement rule for the production dialog instances. d. As shown in Figure 8, define the placement rule for the non-production systems. e. As shown in Figure 9, configure the virtual machine affinity rule for the non-production systems.
Note For detailed information about the purpose of using affinity rules in this solution, refer to the Scope section.
Table 6. DRS groups configuration
DRS group Group type Virtual machine/Host contained
PRD Central System
Virtual machine DRS groups SAPPRD
PRD Dialog Instances
Virtual machine DRS groups SAPPRDD01, SAPPRDD02, SAPPRDD03
Non-Production Systems
Virtual machine DRS groups SAPDEV, SAPQAS, SAPQASDI
PRD Central Host Host DRS groups SAPESX05
PRD DI Host Host DRS groups SAPESX06
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and
VMware Distributed Resource Scheduler—A Detailed Review 14
Non-Production Host
Host DRS groups SAPESX07 SAPESX08
Note DRS groups are configured according to the Scope section of this white paper.
Figure 2 illustrates the configurations of the new resource pool created in the cluster.
Figure 2. SAP resource pool configuration
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and VMware Distributed Resource Scheduler—A Detailed Review
15
Figure 3 illustrates the configurations of the sub-resource pools created for the SAP production system.
Figure 3. Production resource pool configuration
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and
VMware Distributed Resource Scheduler—A Detailed Review 16
Figure 4 illustrates the configurations of the sub-resource pools created for the SAP non-production system.
Figure 4. Non-production resource pool configuration The SAP parent resource pool contains all CPU and memory resources of the cluster. The rate of CPU and memory resources assigned to the sub-resource pools is calculated by comparing the sum of virtual machine resources in the relevant (production or non-production) group to the total resources.
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and VMware Distributed Resource Scheduler—A Detailed Review
17
Figure 5 illustrates the affinity rules created for this solution.
Figure 5. Overview of affinity rules There are four affinity rules configured in the testing environment.
• The PRD CI+DB placement rule specifies that the virtual machine of the production central system must run on its own ESX server. This rule is defined because the production central system requires substantial system resources, and sometimes it may take up all the resources that are available on a single ESX server.
• The PRD DI placement rule specifies that the virtual machines of the production dialog instances cannot be moved to the ESX server hosting the production central system. This rule is defined to avoid resource contention between the production dialog instances and the production central system.
• The DEV and QAS set apart rule specifies that the virtual machines of the DEV central system and the QAS central system cannot coexist on the same ESX server. This rule is defined to avoid resource contention between the DEV central system and the QAS central system.
• The non-production placement rule specifies that the virtual machines of the non-production systems cannot be moved to the ESX servers hosting the production system. According to the business priority, this rule is defined to ensure that the non-production instances will not take up the production system’s resources.
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and
VMware Distributed Resource Scheduler—A Detailed Review 18
Figure 6 illustrates the definition of the placement rule for the production central instance and database instance.
Figure 6. PRD CI+DB placement rule
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and VMware Distributed Resource Scheduler—A Detailed Review
19
Figure 7 illustrates the definition of the placement rule for the production dialog instances.
Figure 7. PRD DI placement rule
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and
VMware Distributed Resource Scheduler—A Detailed Review 20
Figure 8 illustrates the definition of the placement rule for the non-production systems.
Figure 8. Non-production placement rule
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and VMware Distributed Resource Scheduler—A Detailed Review
21
Installing SAP systems
Installing SAP systems in the context of this solution involves the following steps:
Step Action
1 Install Windows Server 2008 as the operating system for an SAP instance on a virtual machine.
2 Install SQL Server 2008 after the operating system installation.
3 Repeat Steps 1 and 2 to create seven virtual machines.
4 Install SAP systems as defined in Table 7.
Figure 9 illustrates the virtual machine affinity rule for the non-production systems.
Figure 9. Non-production virtual machine rule
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and
VMware Distributed Resource Scheduler—A Detailed Review 22
Table 7. SAP systems configuration
Host name of the virtual machine
Usage Instance number
Placement
SAPPRD PRD CI + DB 00 SAPESX05
SAPPRDD01 PRD DI 01 SAPESX06
SAPPRDD02 PRD DI 02 SAPESX06
SAPPRDD03 PRD DI 03 SAPESX06
SAPDEV DEV CI + DB 00 SAPESX07
SAPQAS QAS CI + DB 00 SAPESX08
SAPQASDI QAS DI 01 SAPESX08
The details of vCPU and memory settings for each SAP instance are described in the Design and Validation section; they may vary depending on the test scenario specification.
After you finish installing both VMware vSphere and SAP systems, the configuration in the vSphere client should resemble Figure 10.
Figure 10. vSphere configuration For detailed information about installing SAP systems, refer to Master Guide for SAP ERP 6.0.
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and VMware Distributed Resource Scheduler—A Detailed Review
23
Configuring SAP systems
Configuring the SAP production system Follow these steps to configure the SAP production system:
Step Action
1 Configure the logon group to include all dialog instances.
Note The central instance and the database instance are installed on the same virtual machine. In order to ensure that the system resources are dedicated to the database instance, the central instance is not included in the logon group so that it generates less workload and consumes less system resources.
2 Set up VMware virtual machines according to SAP and VMware guidelines listed in the References section.
3 Adjust the SAP instance profile according to Table 8 and Table 9 so that the system can handle high volume parallel workload.
4 Prepare the HP LoadRunner scenario to generate SAP specific heavy workload.
Table 8. CI system profile
Parameter Value
sapgui/user_scripting True
zcsa/db_max_buftab 20,000
zcsa/table_buffer_area 90,000,000
rsdb/cua/buffersize 9,000
abap/buffersize 600,000
rsdb/ntab/irbdsize 9,000
rsdb/ntab/ftabsize 90,000
rsdb/ntab/entrycount 30,000
rdisp/wp_no_dia 30
rdisp/wp_no_btc 3
rdisp/wp_no_enq 1
rdisp/wp_no_vb 14
rdisp/wp_no_vb2 8
em/largepages Yes
Table 9. DI system profile
Parameter Value
em/largepages Yes
zcsa/db_max_buftab 30,000
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and
VMware Distributed Resource Scheduler—A Detailed Review 24
zcsa/table_buffer_area 250,000,000
zcsa/presentation_buffer_area 4,950,000
abap/buffersize 600,000
rsdb/ntab/irbdsize 9,000
rsdb/ntab/ftabsize 90,000
rsdb/ntab/entrycount 60,000
rsdb/obj/large_object_size 16,384
rsdb/obj/max_objects 4,000
rsdb/obj/buffersize 8,192
sapgui/user_scripting True
rdisp/wp_no_dia 40
rdisp/wp_no_btc 5
Configuring the SAP non-production systems For SAP non-production systems, follow the SAP Master Guide to install the systems and adopt system profiles by default.
Configuring LoadRunner
The LoadRunner configuration used in this case is outlined in Table 10.
Table 10. LoadRunner configuration
Instance type Instance number
Instance setting
Controller 1 2 vCPUs, 16 GB memory
Generator 10 2 vCPUs, 10 GB memory
The following workload profiles are used in this case:
• Material Master Data Maintenance
This process covers a material master data maintenance scenario, which includes the creation of a material with basic and extended information, a goods receipt, pricing condition configuration, and 100 information records for this material.
• Human Resource Data Maintenance
This process covers a human resource master data maintenance scenario, which includes the action of hiring employees in a given organization.
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and VMware Distributed Resource Scheduler—A Detailed Review
25
• Order to Cash transaction
This process covers a sell-from-stock scenario, which includes the creation of a customer order with six line items and the corresponding delivery with subsequent goods movement and invoicing.
In this solution’s tests, each LoadRunner generator uses the same user number setting. The details of how workload profiles are configured on generators depend on the test scenarios. For detailed information about the test scenarios, refer to the Test scenarios section.
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and
VMware Distributed Resource Scheduler—A Detailed Review 26
Design and validation
Introduction to design and validation
This section provides the detailed information of solution design and validation procedures. The following considerations are taken into account prior to the solution implementation:
• A given LoadRunner scenario with the variables of user number, generator, and script type provides an identical workload to an SAP system. Each group of testing between baseline and DRS-enabled scenario shares exactly the same LoadRunner scenario in order to keep the testing environment consistent.
• CPU usage of a virtual machine transparently reflects the system workload. CPU usage of an ESX server reflects the resource consumption at the cluster-wide level, which is the key metric that DRS uses as the threshold to trigger vMotion.
• The SAP average response time is the most important high-level key performance indicator (KPI) for customers to determine the system efficiency. By comparing this KPI, you can observe the performance between different setups.
• An SAP logon group is configured to distribute the workload evenly across the dialog instances. Because the central instance and database instance share the same server, the central instance is excluded from the logon group in order to minimize the performance impact on the database instance.
Test methodology
In general, a typical SAP landscape consists of three systems: the development system (DEV), the quality assurance system (QAS), and the production system (PRD). The production system assigned by its business role usually requires more hardware resources to process mass user requests. During peak times, such as month-end, demands on computing power may become much higher than normal. This leads to a severe resource imbalance across the VMware cluster, regarding its utilization.
DRS is introduced to address the problem of resource imbalance in the cluster. It rebalances the CPU and memory resources by automatically moving virtual machines over ESX hosts. When the production system requires additional CPU capacity, DRS moves the virtual machines with the dialog instances from the original ESX server to another ESX server, which has free CPU and/or memory resources, in the same VMware cluster, depending on the demand. Likewise, when the workload in the non-production systems increases and the production system’s load decreases, DRS can move dialog instances of the production system back to the original ESX host.
How DRS influences the SAP system depends on the CPU and memory settings of individual dialog instances of the production system. There are two types of configuration in terms of resource assignment: overcommitment and non-overcommitment. According to SAP, memory overcommitment is not recommended as it impacts the system performance. The best practice for memory settings is to make the reservation equal to its general setting. It is because the SAP kernel does not release memory after allocating it. For detailed information about this behavior, refer to SAP Note 1056052.
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and VMware Distributed Resource Scheduler—A Detailed Review
27
The following scenarios are used for CPU settings:
• Non-overcommitment
In this profile, the number of vCPUs configured for all virtual machines on a single ESX server does not exceed the total number of physical CPU cores.
• Overcommitment
In this profile, the number of vCPUs configured for all virtual machines on a single ESX server exceeds the total number of physical CPU cores.
Both these CPU setting profiles for the production system were applied in this solution validation to test the DRS functionality.
Test scenarios The following test scenarios were conducted to verify how DRS takes effect among
different CPU and memory settings and various workload profiles.
Table 11 lists the VMware settings for the non-production SAP systems that were applied throughout the test scenarios.
Table 11. VMware settings for non-production systems
Virtual machine vCPU Memory (GB) ESX server
SAPDEV 4 16 SAPESX07
SAPQAS 4 16 SAPESX08
SAPQASDI 2 8 SAPESX08
For the production system, the virtual machine placement and specification varied according to the scenario type.
CPU non-overcommitment Group 1 - Single ESX approach This scenario was designed to investigate how DRS functions in a scale-in (simulated) SAP landscape. A scale-in landscape means that the virtual machines of all SAP instances are consolidated on a single ESX server, in order to produce severe imbalance across cluster. Table 12 lists the virtual machine placement and specification in this scenario.
Table 12. Virtual machine placement and specification Virtual machine vCPU CPU
reservation Memory Memory reservation ESX server
SAPPRD 4 4 GHz 16 GB 16 GB SAPESX05
SAPPRDD01 1 1 GHz 3 GB 3 GB SAPESX05
SAPPRDD02 1 1 GHz 3 GB 3 GB SAPESX05
SAPPRDD03 1 1 GHz 3 GB 3 GB SAPESX05
SAPPRDD04 1 1 GHz 3 GB 3 GB SAPESX05
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and
VMware Distributed Resource Scheduler—A Detailed Review 28
Note The total capacity of a single ESX server is eight CPU cores and 36 GB memory.
Table 13 lists the LoadRunner configuration applied in this scenario.
Table 13. LoadRunner configuration
Scenario User Number
Material Master (MM) Data Maintenance 14
Human Resource (HR) Data Maintenance 1 14
HR Data Maintenance 2 14
Order to Cash 1 14
Order to Cash 2 14
Order to Cash 3 14
Total 84
Note Each scenario in this table was running on an independent LoadRunner generator.
Figure 11 shows the DRS migration threshold setting.
Figure 11. DRS migration threshold setting The DRS migration threshold allows you to specify which recommendations are generated and then applied (in fully automated mode) or shown in vCenter (in manual mode).
You can use the slider to set the DRS migration threshold to a value on a scale of five. Moving the slider towards the Aggressive end makes the migration more responsive to resource imbalance so that the DRS is most likely to be triggered. Moving the slider towards the Conservative end makes the migration less responsive to resource imbalance so that small resource imbalance is ignored.
In this test, SAPESX05 had the highest workload while SAPESX06 had no workload. This resulted in severe imbalance of CPU utilization. In order to avoid vMotion being triggered too early, the threshold was set to one level lower than Aggressive.
For the rest of test, the threshold was set to Aggressive so that DRS can respond as early as possible.
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and VMware Distributed Resource Scheduler—A Detailed Review
29
Group 2 - Double-ESX approach
This scenario was designed to investigate how DRS functions in a scale-out (real) SAP landscape. A scale-out landscape means deploying the virtual machines of the SAP system on two or more ESX servers. Therefore, the SAP system can use more resources to get better performance. This scenario is more close to a real customer’s environment.
Table 14 lists the virtual machine placement and specification applied in this scenario.
Table 14. Virtual placement and specification
Virtual machine vCPU
CPU reservation (GHz)
Memory(GB)
Memory reservation (GB)
ESX server
SAPPRD 8 N/A 30 30 SAPESX05
SAPPRDD01 2 1 7 7 SAPESX06
SAPPRDD02 2 1 7 7 SAPESX06
SAPPRDD03 2 1 7 7 SAPESX06
SAPPRDD04 2 1 7 7 SAPESX06
Note Total capacity of a single ESX server is eight CPU cores and 36 GB memory.
Table 15 lists the LoadRunner configuration applied in this scenario.
Table 15. LoadRunner configuration
Scenario User number
MM Data Maintenance 38
HR Data Maintenance 1 38
HR Data Maintenance 2 38
Order to Cash 1 38
Order to Cash 2 38
Order to Cash 3 38
Total 228
Note Each scenario in this table was running on an independent LoadRunner generator.
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and
VMware Distributed Resource Scheduler—A Detailed Review 30
Figure 12 shows the DRS migration threshold setting applied in this scenario.
Figure 12. DRS migration threshold setting
CPU overcommitment This scenario is designed to investigate how DRS functions in a scale-out (real) SAP landscape when the CPU resource is overcommitted on the ESX server that hosts production dialog instances.
Table 16 lists the virtual machine placement and specification applied in this scenario.
Table 16. Virtual machine placement and specification
Virtual machine vCPU
CPU reservation (GHz)
Memory(GB)
Memory reservation (GB)
ESX server
SAPPRD 8 N/A 30 30 SAPESX05
SAPPRDD01 4 1 8 8 SAPESX06
SAPPRDD02 4 1 8 8 SAPESX06
SAPPRDD03 4 1 8 8 SAPESX06
Note The total capacity of a single ESX server is eight CPU cores and 36 GB memory. Therefore, the CPU resource is overcommitted on the specific ESX server (SAPESX06).
Table 17 lists the LoadRunner configuration applied in this scenario.
Table 17. LoadRunner configuration
Scenario User Number
MM Data Maintenance 40
HR Data Maintenance 1 40
HR Data Maintenance 2 40
Order to Cash 1 40
Order to Cash 2 40
Order to Cash 3 40
Total 240
Note Each scenario in this table was running on an independent LoadRunner generator.
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and VMware Distributed Resource Scheduler—A Detailed Review
31
Figure 13 shows the DRS migration threshold setting applied in this solution.
Figure 13. DRS migration threshold setting
Test procedure In each scenario, the following steps were run to collect the results. A comparison of
the results between the baseline and the DRS test scenarios was performed to investigate how DRS benefits SAP systems in each scenario.
Prerequisites:
• Adjust virtual machine CPU and memory settings as specified in each test scenario.
• The duration of each LoadRunner scenario is set to 1 hour.
• The interval of starting users in LoadRunner is set to 8 seconds.
Baseline test procedure
Step Action
1 Disable DRS in the VMware cluster.
2 Start all SAP instances, including DEV, QAS, and PRD in the landscape.
3 Use the corresponding LoadRunner profile as specified in each test scenario to start loading the business volume in the SAP production system.
4 Record the statistics of the CPU usage on ESX servers and virtual machines across the cluster during the test.
5 Record the SAP performance statistics through ST03 after the test is completed.
DRS test phase 1 - PRD DI migrated out Step Action
1 Enable DRS in the VMware cluster and set the Automation Level to Fully automated. Then, apply the migration threshold setting as specified in each test scenario.
2 Use the corresponding LoadRunner profile as specified in each test scenario to start loading the business volume in the SAP production system.
3 Record the statistic of CPU usage on ESX servers and virtual machines across the cluster during the test.
4 Record the SAP performance statistics through ST03 and the DRS history information through the vSphere client right after the test is completed.
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and
VMware Distributed Resource Scheduler—A Detailed Review 32
DRS test phase 2 - PRD DI migrated back
Step Action
1 Remove the workload on the PRD system.
2 Deliver a very high workload to the DEV and QAS systems.
3 Record the DRS history information after the test is completed.
Results analysis
This section presents and analyzes the results of all test scenarios.
Group 1 results analysis in the CPU non-overcommitted scenario
As shown in Table 18, PRD DI04 was moved from SAPESX05 to SAPESX06 after DRS was enabled.
Table 18. Virtual machine placement in this scenario
ESX server SAP instances on virtual machines (DRS disabled)
SAP instances on virtual machines (DRS enabled)
SAPESX05 PRD CI+DB PRD DI01, DI02, DI03, DI04
PRD CI+DB PRD DI01, DI02, DI03
SAPESX06 N/A PRD DI04
SAPESX07 DEV DEV
SAPESX08 QAS CI+DB QAS DI
QAS CI+DB QAS DI
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and VMware Distributed Resource Scheduler—A Detailed Review
33
Figure 14 shows the CPU utilization of the ESX servers when DRS was disabled, compared to the CPU utilization when DRS was enabled.
Figure 14. CPU utilization comparison in this scenario Figure 15 shows the SAP average response time when DRS was disabled, compared to the average response time when DRS was enabled.
Figure 15. SAP average response time comparison in this scenario
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and
VMware Distributed Resource Scheduler—A Detailed Review 34
The statistics of Group 1 testing show that, in a scale-in configuration of a SAP PRD instance, the average dialog response time was not significantly improved, even after DRS moved out one dialog instance. The resource constraints were on the virtual machines, not on the ESX server. Therefore, DRS could not significantly improve the performance.
Adequate CPU sizing for the SAP dialog instances can prevent this problem. DRS did balance the CPU utilization between the ESX servers in the DRS cluster. Therefore, the overall CPU utilization across the ESX servers in the datacenter was improved.
Group 2 results analysis in the CPU non-overcommitted scenario
As shown in Table 19, PRD DI03 was moved from SAPESX06 to SAPESX07 after DRS was enabled.
Table 19. Virtual machine placement in this scenario
ESX server SAP instances on virtual machines (DRS disabled)
SAP instances on virtual machines (DRS enabled)
SAPESX05 PRD CI+DB
PRD CI+DB
SAPESX06 PRD DI01, DI02, DI03, DI04 PRD DI01, DI02, DI04
SAPESX07 DEV DEV PRD DI03
SAPESX08 QAS CI+DB QAS DI
QAS CI+DB QAS DI
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and VMware Distributed Resource Scheduler—A Detailed Review
35
Figure 16 shows the CPU utilization of the ESX servers when DRS was disabled, compared to the CPU utilization when DRS was enabled.
Figure 16. CPU utilization comparison in this scenario Figure 17 shows the SAP average response time when DRS was disabled, compared to the average response time when DRS was enabled.
Figure 17. SAP average response time comparison in this scenario
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and
VMware Distributed Resource Scheduler—A Detailed Review 36
The statistics of Group 2 testing demonstrate the expected outcome for a double-ESX landscape scenario. When the PRD system was running with some workload, DRS triggered vMotion for the dialog instances without significant impact on the average response time.
Test results analysis in the CPU overcommitted scenario
As shown in Table 20, PRD DI03 was moved from SAPESX06 to SAPESX07 after DRS was enabled.
Table 20. Virtual machine placement in this scenario
ESX server
SAP instances on virtual machines (DRS disabled)
SAP instances on virtual machines (DRS enabled)
SAPESX05 PRD CI+DB
PRD CI+DB
SAPESX06 PRD DI01, DI02, DI03 PRD DI01, DI02
SAPESX07 DEV
DEV PRD DI03
SAPESX08 QAS CI+DB QAS DI
QAS CI+DB QAS DI
Figure 18 shows the CPU utilization of the ESX servers when DRS was disabled, compared to the CPU utilization when DRS was enabled.
Figure 18. CPU utilization comparison in this scenario
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and VMware Distributed Resource Scheduler—A Detailed Review
37
As shown in Table 20 and Figure 18, DRS triggered vMotion before the CPU usage of SAPESX06 reached 80 percent. Figure 19 shows the SAP average response time when DRS was disabled, compared to the average response time when DRS was enabled.
Figure 19. SAP average response time comparison in this scenario
When the CPU was overcommitted and DRS was enabled, SAP average response time was improved by 27 percent. The CPU usage on the ESX server that hosts PRD dialog instances was reduced as well. This ideal result validates that DRS enhances the overall system performance by rebalancing resources through automatic vMotion of dialog instances.
DRS phase 2 test result DRS driven migration-back was tested as well when DEV and QAS required more CPU resources. When DEV and QAS systems were fully loaded, DRS can trigger virtual machine migration for the production dialog instance back to its original ESX server.
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and
VMware Distributed Resource Scheduler—A Detailed Review 38
Conclusion
Summary This solution validates the use of VMware DRS to prevent ESX server CPU
bottlenecks and overall improves SAP performance when CPU resources are overcommitted. vMotion initiated by DRS occurs as expected and improves average SAP response time by 27 percent.
Key points The key points addressed by this solution are summarized as follows:
• VMware DRS provides an automated and nondisruptive redistribution of virtual machines to prevent CPU resource bottlenecks at the ESX server level.
• The overall SAP system performance is improved because the CPU requirements driven by SAP applications on an ESX server are satisfied when VMware DRS performs virtual machine redistribution.
• VMware DRS affinity rules provide control of virtual machine movement across the cluster to ensure the compliance with service level agreements and defined business policies.
Virtual Infrastructure for SAP Landscape Enabled by EMC CLARiiON CX4-480 and VMware Distributed Resource Scheduler—A Detailed Review
39
References
White papers For additional information, see the white papers listed below.
• SAP Solutions on VMware vSphere 4 - Best Practice Guidelines
• Virtualized SAP Performance with VMware vSphere 4
• VMware vSphere: The CPU Scheduler in VMware ESX 4.1
Product documentation
For additional information, see the product documents listed below.
• VMware ESX 4.1 and vCenter 4.1 Installation Guide
• vSphere Resource Management Guide 4.1
• Master Guide for SAP ERP 6.0
Other documentation
For additional information, see documents at the website listed below.
• SAP Note 1237682 - Configuration Parameters for SQL Server 2008
• SAP Note 1056052 - Windows: VMware ESX 3.x or vSphere configuration guidelines