optimizing application performance using vmax host i/o ... · unix, and windows (db2 for luw) in a...

White Paper

Abstract

Customers can manage application performance and make it more predictable in a multi-tenant environment using the Host I/O Limits feature of EMC VMAX. This white paper discusses applying VMAX Host I/O Limits on DB2 for Linux, Unix, and Windows (DB2 for LUW) in a multi-tenant environment. The test case scenarios covered here were developed using DB2 9.7 for LUW on multiple AIX systems, VMware with Windows Guest OS, AIX with dynamic distribution on FC and FCoE protocols.

June 2013

OPTIMIZING APPLICATION PERFORMANCE USING VMAX® HOST I/O LIMITS FOR DB2®

FOR LINUX, UNIX, AND WINDOWS

2 Optimizing application performance using Symmetrix VMAX Host I/O limits for DB2 for Linux, Unix, and Windows

Copyright © 2013 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. All other trademarks used herein are the property of their respective owners. Part Number H11981


Table of Contents

Executive summary.................................................................................................. 5

Audience................................................................................................................. 6

Introduction ............................................................................................................ 7

VMAX Host I/O Limits Architecture Overview ............................................................ 7 Cascaded Host I/O Limits ................................................................................................... 8 Dynamic distribution ........................................................................................................ 11 Dynamic distribution modes ............................................................................................. 12 Protocols supported ......................................................................................................... 12 Configuration Overview .................................................................................................... 12

Host I/O Limits with Solutions Enabler ......................................................................... 12 Host I/O Limits with Unisphere ..................................................................................... 15

Limitations, Scalability and General Restrictions .............................................................. 18

Test case scenarios of Host I/O Limits with IBM DB2 for LUW .................................. 21 Test case 1: Using Host I/O Limits to minimize the impact of Tier 2 applications on Tier1 applications sharing the same VMAX storage infrastructure ........................................ 21

Functional overview:..................................................................................................... 22 Hardware Configuration:............................................................................................... 22 Software Stack: ............................................................................................................ 23 Test Results: ................................................................................................................. 27

Test case 2 : Using cascaded Host I/O Limits to reconfigure development environment with no impact to Production ....................................................................... 28

Functional Overview: .................................................................................................... 29 Hardware Configuration:............................................................................................... 29 Software Stack: ............................................................................................................ 30 Test Results: ................................................................................................................. 33

Test case 3 : Use Dynamic Host I/O Limits feature to redistribute and failover Host I/O Limits to online FA directors ................................................................................ 34

Functional Overview: .................................................................................................... 34 Hardware Configuration: ............................................................................................... 34 Software Stack: ............................................................................................................. 35 Test Results: ................................................................................................................. 36

Test case 4: Use Dynamic Host I/O Limits feature to redistribute and failover Host I/O Limits to online FCoE directors ............................................................................ 38

Functional Overview: .................................................................................................... 38 Hardware Configuration:............................................................................................... 39 Software Stack: ............................................................................................................ 39 Test Results: ................................................................................................................. 40

Conclusion ............................................................................................................ 42


References ............................................................................................................ 43


Executive summary

In a multi-tenant environment, storage infrastructure is shared among a variety of servers and applications. Whenever an application overloads the storage in such an environment, it can affect other applications sharing that storage infrastructure, leading to resource contention and reduced performance for critical applications. As organizations grow and scale, their storage infrastructure also grows. Consequently, the need for a way, to enforce performance boundaries between applications sharing the storage, is greater than ever. This enforcement of performance boundaries is necessary to maintain service level agreements for all applications or tenants.

The EMC Symmetrix® VMAX® Host I/O Limits feature addresses this concern by providing a way to set the desired IOPS and bandwidth limits at the storage group level; these storage groups can then be associated with a set of front end VMAX ports through the provisioning of a masking view. This enables organizations to maintain their service level agreements and ensure that each application is able to maintain a predictable level of performance without adversely affecting others and still share the same VMAX storage system.

This white paper demonstrates use cases of the VMAX Host I/O Limits feature with IBM DB2 for Linux, Unix, and Windows (DB2 for LUW) databases. It covers application performance optimization using Host I/O Limits, cascaded Host I/O Limits with Dynamic distribution, support over both Fibre Channel, and Fibre Channel over Ethernet environments.


Audience

The white paper is intended for database administrators, storage administrators, architects, EMC customers, EMC field personnel and technology professionals who want to understand the management of the performance of DB2 for LUW databases using the VMAX Host I/O Limits feature of EMC VMAX arrays in a multi-tenant environment.


Introduction

VMAX Host I/O Limits, is a feature introduced in Solutions Enabler 7.5 and Enginuity 5876 Q4 2012 SR that puts a limit on the amount of Front End Bandwidth and Input/ Output operations per second (IOPS) that can be utilized by a set of Symmetrix devices over a set of front end director ports. This feature allows for Symmetrix resources to be shared between all the tenants on a VMAX storage system that are on the same port group, but ensures that each tenant has its own workload and bandwidth level, which is set based on service levels and performance requirements. Essentially, this feature enables the applications or “tenants” sharing a VMAX storage system to consume logically separate VMAX performance resources.

The test case scenarios and detailed implementation procedures provided demonstrate the use of VMAX Host I/O Limits with IBM DB2 for LUW on AIX and Windows servers using Fibre Channel and FCoE adapters, as well as illustrate how the Host I/O Limits feature can be utilized to clear performance bottlenecks in a multi-tenant storage environment.

VMAX Host I/O Limits Architecture Overview

Host I/O Limits enables users to set limits on the workload and bandwidth of their applications; this, in turn, enables many applications to coexist, maintaining their performance boundaries in a secure fashion. VMAX auto provisioning groups prevents any tenant from accessing another tenant’s data. Using auto provisioning groups, each application can create its own masking view but share the port group. This allows them to share VMAX resources on the front end adapter (FA) ports while at the same time the traffic on the FA ports is controlled by the Host I/O Limits setting on the application’s storage group. Masking views associates a group of host initiators (called an initiator group), a group of front end director ports (called a port group) and a group of devices (called a storage group). Creating a masking view is a simple operation and consolidates the mapping and masking into one operation. Any initiators, ports or devices added to an existing group are automatically added to the appropriate masking view.


Cascaded Host I/O Limits

Cascaded storage groups help to solve the problem of customers having different requirements for provisioning and Host I/O Limits; Cascaded storage groups provide easier storage management, especially in situations where a single host serves multiple applications and each application has its own Fully Automated Storage Tiering (FAST) policy and Host I/O Limits. Anytime a device is added to or removed from a child storage group, the parent storage group will be updated to reflect the change. Thus, the use of cascaded storage groups reduces the number of updates that may have to be made to parent storage group while enabling child storage groups to retain their individual FAST policies and Host I/O Limits.

The front-end performance controls or Host I/O Limits can be cascaded up to two levels. Storage groups can be assigned either at the parent level or the child level and correspondingly, Host I/O Limits are assigned at parent and child levels. Host I/O Limits are shared between the front end directors to which the storage groups are mapped.

An example of parent and child level Host I/O Limit is illustrated in Figure 1.

Figure 1 Cascaded Host I/O Limits

Host I/O Limits can be added to parent storage groups, child storage groups, or both. If a masking view has been created on a parent storage group and Host I/O Limit is set on the parent, that limit will be shared among all of the parent’s child storage groups. If a masking view is created on a parent storage group, and both parent and child storage groups have Host I/O Limits, devices in the child storage groups will be limited by both the child and the parent Host I/O Limits. Finally, if a masking view is created using a child storage group, then the devices in the


child storage group are limited only by the child Host I/O Limits. In a cascaded scenario, the Host I/O Limits can be oversubscribed, that is, the total Host I/O Limits of the children can be greater than the Host I/O Limit set at the parent level. However, the Host I/O Limits set on any of the child storage groups cannot exceed the limits set on the parent. The number of child Host I/O Limits and their ratios to each other, and to the parent, along with the number of children exceeding their Host I/O Limits are key determinants in deciding how the Host I/O Limits are distributed in oversubscribed situations. Figure 2 shows a masking view created at parent storage group level.

Port group

Initiaor group

Parent SG

Child SG

Child SG

Child SG

t Masking View

Host I/O Limits

Host I/O Limits

Host I/O Limits

Host I/O Limits

Figure 2 Masking view created on the Parent Storage Group

A parent Host I/O Limits can have multiple child Host I/O Limits attached to it; however, the converse is not true. A child Host I/O Limit can be linked to only one parent Host I/O Limit. A child Host I/O Limit inherits the parent Host I/O Limit settings if no settings are defined and will show up as being “Shared”. A child can have a different Host I/O Limit setting than its parent; in such cases, the Host I/O Limit for the child will show up as “Defined”. In the following example, Parent’s Host I/O Limit is set at 10,000 iops, Child1’s Host I/O Limit is set at 4,000 iops and Child2’s Host I/O Limit is not set. We see that Child1 has its own Host I/O Limit and shows the setting as “Defined(Shared)” and Child2 inherits the Parent’s Host I/O Limit and shows the setting as “Shared”.

# symsg -sid 351 show Parent_stor

Name: Parent_stor Symmetrix ID : 000195700351 Last updated at : Tue May 21 15:21:50 2013 Masking Views : Yes


FAST Policy : No Host I/O Limit : Defined Host I/O Limit MB/Sec : NoLimit Host I/O Limit IO/Sec : 10000 Dynamic Distribution : Always Number of Storage Groups : 3 Storage Group Names : Child1_stor (IsChild) Child2_stor (IsChild) Child3_stor (IsChild) Devices (3): { --------------------------------------------------------- Sym Device Cap Dev Pdev Name Config Sts (MB) --------------------------------------------------------- 060B N/A TDEV RW 983040 0617 N/A TDEV RW 983040 061B N/A TDEV RW 983040 } # symsg -sid 351 show Child1_stor Name : Child1_stor Symmetrix ID : 000195700351 Last updated at : Mon Apr 22 13:27:58 2013 Masking Views : Yes FAST Policy : No Host I/O Limit : Defined (Shared) Host I/O Limit MB/Sec : NoLimit (NoLimit) Host I/O Limit IO/Sec : 4000 (10000) Dynamic Distribution : Always Number of Storage Groups : 1 Storage Group Names : Parent_stor (IsParent) Devices (1): { --------------------------------------------------------- Sym Device Cap Dev Pdev Name Config Sts (MB) --------------------------------------------------------- 060B N/A TDEV RW 983040 } # symsg -sid 351 show Child2_stor


Name : Child2_stor Symmetrix ID : 000195700351 Last updated at : Mon Apr 29 15:05:54 2013 Masking Views : Yes FAST Policy : No Host I/O Limit : Shared Host I/O Limit MB/Sec : NoLimit Host I/O Limit IO/Sec : 10000 Dynamic Distribution : Always Number of Storage Groups : 1 Storage Group Names : Parent_stor (IsParent) Devices (1): { --------------------------------------------------------- Sym Device Cap Dev Pdev Name Config Sts (MB) --------------------------------------------------------- 061B N/A TDEV RW 983040 }

Dynamic distribution

The dynamic distribution of Host I/O Limits can be set across configured director ports in a port group. This allows for the Host I/O Limit allocation on each individual port to adjust to fluctuating demand. In the absence of dynamic distribution, each front end director in the port group is allocated a fixed value of Host I/O Limits and these limits remain the same, regardless of whether other front end directors in the port group are online or offline. For example, if the Host I/O Limit is 10,000 IOPS and there are two directors in the port group, each director will be assigned a static (fixed) 5,000 Host I/O Limit.


It is important to note that Host I/O Limits need to set up on a storage group before dynamic distribution can be enabled on the group.

Dynamic distribution modes

Host I/O Limits are shared between the front end directors to which storage groups are mapped and the redistribution of Host I/O Limits is provided based on customer requirements. Some customers prefer redistribution of Host I/O Limits among director ports and some prefer that if a director has failed, the unused Host I/O Limit to be failed over to the remaining online directors. Accordingly, three settings are available – “OnFailure”, “Always”, and “Never”. The setting “OnFailure” enables the Host I/O Limit to failover when one of the front end director ports in a port group fails. “OnFailure” causes the fraction of Host I/O Limits to be adjusted among the online ports if any of the ports in the port group go offline. “OnFailure” is a subset of “Always”. “Always” enables the Host I/O Limit to be distributed evenly in all kinds of situations, such as when a front end director fails, a director is added, or a director is removed. “Always” allows for the configured Host I/O Limits to be dynamically distributed across the configured ports. The redistribution is dynamic and seamless. Once the offline director comes back online, the Host I/O Limit is again redistributed across all the ports. The default setting is “Never” which means that dynamic distribution is disabled and no dynamic distribution setting is available for the storage group. In this case, the dynamic distribution flag has to be explicitly enabled.

Protocols supported

This feature is supported on Front-End ports for Fibre Channel, iSCSI and FCoE directors. It is not currently supported for FICON.

Configuration Overview

Host I/O Limits with Solutions Enabler

1. Setting up Host I/O Limits

Host I/O Limits can be set either in a single or cascaded level. The “symsg set” command is used to allow Host I/O Limits to be set on a Storage Group(SG). The Storage group needs to be assigned to a masking view for the Host I/O Limits to function.

# symsg –sid <sid> -sg <sg_name> set –iops_max <iops>


The “symsg add” command will allow a SG with Host I/O Limits to be added to a parent SG that is in a masking view.

# symsg –sid <sid> -sg <parent SG name> add sg <child SG name>

2. Activating Host I/O Limits

The “symaccess create view” command will activate the Host I/O Limits and create the cascaded relationship between the parent and child Host I/O Limits when both the parent and the child have Host I/O Limits defined.

# symaccess –sid <sid> create view –name <view name> -ig <initiator group> -pg <port group> -sg <storage group>

The “symaccess delete view” command will deactivate Host I/O Limits and remove the cascaded Host I/O relationship if both the parent and child storage group have Host I/O Limits set. 3. Setting dynamic distribution of Host I/O Limits

The “symsg –sid 351 set –dynamic” command is used to set the dynamic distribution of front end Host I/O Limits. Dynamic distribution is disabled by default and the setting is “Never”. Dynamic distribution can be activated by changing the setting to either “OnFailure” or “Always”.

# symsg –sid <sid> -sg <sg_name> set –dynamic <dynamic distribution setting>

4. symsg demand reports

The “symsg list –by_port –demand” command is used to display the Host I/O Limits demand on front end directors. In case of cascaded setup, the total demand is calculated based on the parent host I/O settings if both the parent and child storage groups have Host I/O Limits defined. The front end Host I/O Limits are divided equally among the directors in the port group used, independent of the number of ports on each director. Hence, as a best practice, it is better to use only one of the ports of a director in the same port group.

# symsg –sid <sid> list –by_port –demand


Figure 3 below shows the Host I/O Limits demand on the front end director ports.

Figure 3 Host I/O Limits on the Front end Directors

The “symsg list –by_pg –demand” command shows the demand report for all of the port groups.

# symsg –sid <sid> list –by_pg –demand 5. Removing Host I/O Limits

Setting the Host I/O Limit to “NOLIMIT” removes the Host I/O Limits set on a storage group # symsg –sid <sid> -sg <sg name> set –iops_max NOLIMIT The “symsg remove” command will allow a SG with Host I/O Limits to be removed from a parent SG that is in a masking view. # symsg –sid <sid> -sg <parent sg name> remove sg <child sg name>


Host I/O Limits with Unisphere

1. Setting up Host I/O Limits

Check the Host I/O Limits checkbox, and depending on whether setting Limits at the IOPs level or the throughput level, enter the respective values. Figure 4 shows how to set Host I/O Limits on Unisphere.

Figure 4 Set Host I/O Limits

2. Activating Host I/O Limits Host I/O Limits can be activated by creating a masking view on the storage group on which the Host I/O Limits are set.


Figure 5 shows how to create a masking view on Unisphere, which in turn activates the Host I/O Limits set on the storage group.

Figure 5 Activate Host I/O Limits

3. Set dynamic distribution Select the appropriate dynamic distribution option from the drop down menu provided for the “Set dynamic distribution” field. Figure 6 shows how to set dynamic distribution of Host I/O Limits on Unisphere.

Figure 6 Set Dynamic distribution of Host I/O Limits


4. Port group demand The Port details window lists the port group demand.

Figure 7 shows the Host I/O Limits demand on the Port Group in Unisphere window.

Figure 7 Port Group demand Report

5. Removing Host I/O Limits Uncheck the appropriate Host I/O Limits checkbox MB/sec or IO/sec, whichever is set.

Figure 8 shows how to remove Host I/O Limits in Unisphere.

Figure 8 Remove Host I/O Limits


Limitations, Scalability and General Restrictions

There is a maximum of 2048 Host I/O Limits configurable on a single VMAX. For cascaded Host I/O Limits, there can a maximum of 32 child Host I/O Limits in a parent.

When setting up Host I/O Limits, the following restrictions apply. • Creating a masking view on a child storage group is not allowed if the child’s parent storage group has an FE Host I/O Limit defined.

# symaccess-sid 351 create view -name test_view -ig lfqu1063_ig -pg lfqu1063_port -sg lfqu1231_child_vm1

Error: Cannot perform the requested operation because the group is currently within a masking view or a masking view through cascading.

• At any given time, a storage group with an FE Host I/O Limit can be associated with at most one port group in any masking view. This means if a storage group with a front end Host I/O Limit is in a masking view with a port group, the storage group and port group combination have to be used when creating other masking views on the storage group. If the user tries to create a masking view using a different port group, an error will be returned.

# symaccess-sid 351 create view -name lfqu1063_pg_view -ig lfqu1063_ig -pg pg_6e1_9e1 -sg lfqu1063_p_vm_stor

Error: The operation cannot be performed because the storage group with a Host I/O Limit can be associated with at most one port group in any masking view


• Any device can be in at most one storage group with front end Host I/O Limits. If an attempt is made to add the same device to other storage groups with front end Host I/O Limits, an error will be returned.

# symaccess -sid 351 -name lfqu1231_child_vm1 -type stor -dev 61b add

Error: The operation cannot be performed because the device already exists in a storage group with a Host I/O Limit

The parent and child storage groups must have the same Dynamic Distribution attribute value before they can be placed in a cascaded relationship. In the following example, the Parent storage group has a setting a “Always” and trying to set “OnFailure” on the child fails.

# symsg -sid 351 show lfqu1063_p_vm_stor Name: lfqu1063_p_vm_stor Symmetrix ID : 000195700351 Last updated at : Tue May 21 15:21:50 2013 Masking Views : Yes FAST Policy : No Host I/O Limit : Defined Host I/O Limit MB/Sec : NoLimit Host I/O Limit IO/Sec : 10000 Dynamic Distribution : Always Number of Storage Groups: 3 Storage Group Names : lfqu1231_child_vm1 (IsChild) lfqu2048_child_vm2 (IsChild) lfqu2049_child_vm3 (IsChild) Devices (3): { --------------------------------------------------------- Sym Device Cap Dev Pdev Name Config Sts (MB) --------------------------------------------------------- 060B N/A TDEV RW 983040


0617 N/A TDEV RW 983040 061B N/A TDEV RW 983040 }

# symsg -sid 351 -sg lfqu1231_child_vm1 set -dynamic always

Error: The operation cannot be performed because the storage group attribute conflicts with other parent or child storage group attribute


Test case scenarios of Host I/O Limits with IBM DB2 for LUW

The following use cases were implemented: Test case 1: Using Host I/O Limits to minimize the impact of Tier 2 applications on Tier1 applications sharing the same VMAX storage infrastructure

Test case 2: Using cascaded Host I/O Limits to reconfigure development environment with no impact to Production

Test case 3: Using Dynamic Host I/O Limits feature to redistribute and failover Host I/O Limits to online FA directors when one or more directors go offline

Test case 4: Using Dynamic Host I/O Limits feature to redistribute and failover Host I/O Limits to online FCoE directors when one or more directors go offline

Test case 1: Using Host I/O Limits to minimize the impact of Tier 2 applications on Tier1 applications sharing the same VMAX storage infrastructure

This section shows two applications sharing the same VMAX port group but having their own storage groups and masking views. By Limiting the IOPS on one, more front end ports resources are available to the other resulting in a better performance.


Functional overview: Figure 9 shows DB2 for LUW databases, App A and App B sharing the same port group

and Host I/O Limits are applied on App B.

Figure 9 DB2 UDB databases App A and App B sharing VMAX storage

The Environment consisted of a T1 (tier 1) DB2 9.7 for LUW database and a

T2 (tier 2) DB2 9.7 for LUW database sharing the same FA ports and storage in the same VMAX. By putting Host I/O Limits on the T2 application, the performance of T1 is expected to increase in terms of IOPS and database performance. The FA resources are shared between the two applications with reference to CPU and memory. When Host I/O Limits are put on the T2 application, it enables more FA resources to be released for use to cater to the T1 application.

Hardware Configuration:

Host AIX232 AIX123 Model IBM 8202-E4C IBM 9110-51A

Processor PowerPC_POWER7 PowerPC_POWER5 Memory 23424 MB 3920 MB

Storage Array Symmetrix VMAX40K Symmetrix VMAX40K

Disk FC 15K rpm 450 GB

drives FC 15K rpm 450 GB

drives Protocol Fibre Channel Fibre Channel


Software Stack:

Host AIX232 AIX123 Business

Importance T1 T2 Databases AIX_232 ( App A) AIX_123 ( App B)

O/S AIX 7.1 AIX 6.1 File system JFS2 JFS2

DB2 UDB version 9.7.3 9.7.3 VMAX Enginuity 5876.229.145 5876.229.145

Multipathing Powerpath 5.5 P 02 Powerpath 5.5 P 02 Storage Group lfqu1232_db2_stor lfqu1123_db2_stor

The configuration consisted of AIX P7server (T1) with DB2 for LUW and AIX P5 server (T2) running DB2 for LUW sharing the same VMAX FA ports. The DB2 databases, each 23 GB in size, were created on top of the JFS2 file system and AIX LVM, which was created from 8 underlying thin devices. Benchmark workload was run on the databases with 50 users. The benchmark test measures the ability of the system to process the most queries in the least amount of time in a multi-user environment. The benchmark test runs mainly read queries, runs multiple transactions with each one running independently on a specified number of users.

The AIX layout for the DB2 database is shown in Figure 10.

Physical Volumes

Volume Group

Logical Volumes

Filesystem

DB2 Database

AIX Layout

Figure 10 AIX DB2 UDB Layout


The Physical volumes were created from the eight Symmetrix devices in the Storage group. Each server had Powerpath installed and multiple paths to the devices were available (shown in Figure 11).

Figure 11 AIX Powerpath Pseudo devices

Logical volumes were created on top of the volume group( shown in Figure 12) and a jfs2 file system was laid on top of the logical volumes. Finally, a DB2 for LUW database was created on top of the jfs2 file system.

Figure 12 AIX Logical Volume

The following procedure was followed: • Ran a baseline benchmark workload with 50 users on the DB2 Databases on the

two AIX machines with no Host I/O Limits set on them. The IOPS was measured using Solutions Enabler symstat commands and the response time was measured using SPA.

• Set a Host I/O Limit of 5000 iops on the App B database (Tier 2) storage group. The IOPS and response were measured in a similar manner.


AIX App A Storage group (Tier 1)

# symsg -sid 351 show lfqu1232_db2_stor Name: lfqu1232_db2_stor Symmetrix ID : 000195700351 Last updated at : Tue Apr 02 10:19:35 2013 Masking Views : Yes FAST Policy : No Host I/O Limit : None Host I/O Limit MB/Sec : N/A Host I/O Limit IO/Sec : N/A Dynamic Distribution : N/A Number of Storage Groups : 0 Storage Group Names : N/A Devices (8): { --------------------------------------------------------- Sym Device Cap Dev Pdev Name Config Sts (MB) --------------------------------------------------------- 0482 N/A TDEV RW 244736 0483 N/A TDEV RW 244736 0484 N/A TDEV RW 244736 0485 N/A TDEV RW 244736 0486 N/A TDEV RW 244736 0487 N/A TDEV RW 244736 0488 N/A TDEV RW 244736 0489 N/A TDEV RW 244736 }


Set Host I/O Limits on AIX App B

#symsg -sid 351 -sg lfqu1123_db2_stor set -iops_max 5000

AIX App B Storage group after Host I/O Limits are set

Name: lfqu1123_db2_stor Symmetrix ID : 000195700351 Last updated at : Thu Apr 11 12:55:55 2013 Masking Views : Yes FAST Policy : No Host I/O Limit : Defined Host I/O Limit MB/Sec : NoLimit Host I/O Limit IO/Sec : 5000 Dynamic Distribution : Never Number of Storage Groups : 0 Storage Group Names : N/A Devices (8): { --------------------------------------------------------- Sym Device Cap Dev Pdev Name Config Sts (MB) --------------------------------------------------------- 049A N/A TDEV RW 244736 049B N/A TDEV RW 244736 049C N/A TDEV RW 244736 049D N/A TDEV RW 244736 049E N/A TDEV RW 244736 049F N/A TDEV RW 244736 04A0 N/A TDEV RW 244736 04A1 N/A TDEV RW 244736 }


Test Results:

APP A APP B

Host I/O Limits

applied IOPS

Average I/O Response time

Host I/O

Limits applied

IOPS Average I/O

Response time

No 12600 22 ms No 9000 9 ms

No 18000 12.5 ms Yes 5000 16.5 ms

• The Read and write response times of App A (Tier 1) decreases when the Host I/O Limits is applied on the Storage Group of App B (Tier 2).

• The IOPS on App A (Tier 1) show an increase from 12600 to 18000 after Host I/O Limits applied on the storage group of App B.

12600

18000

9000

5000

0

5000

10000

15000

20000

WithoutHost I/O

Limits

WithHost I/O

Limitsapplied

to App B

IOPS

App A IOPS Increases

App A

App B

22

12.5 9

16.5

0

5

10

15

20

25

WithoutHost I/O

Limits

With HostI/O Limitsapplied to

App B

Resp

onse

tim

e in

ms

App A Response time Decreases

App A

App B


Test case 2 : Using cascaded Host I/O Limits to reconfigure development environment with no impact to Production

This section shows a cascaded set up of Host I/O Limits on ESXi server with each Virtual machine having a storage group which is a member of the parent storage group. The effect of development servers affecting the Mission critical apps on the same FA ports can be managed by having the development servers in a cascaded parent SG which has a Host I/O Limit set. In this test case, an ESXi server is taken as an example of development environment. Multiple Virtual machines running DB2 for LUW on Windows guests on the same ESX server can have their individual Host I/O Limits set as per their requirements. More development servers can be introduced by varying the Host I/O Limits at any time.


Functional Overview: Figure 13 shows a cascaded Host I/O Limits setup where a third virtual machine is added

to being an existing setup.

Figure 13 Cascaded Host I/O Limits setup for Test Case 2

The test case’s focus was to understand the ability for front-end performance controls to be cascaded and allocated to children storage groups. If the Production server is on the same front end ports as a development environment, increasing the number of development servers may impact the Production performance. But, with the help of Host I/O Limits, performance controls can be set on the development environment so that it doesn’t affect Production. The Host I/O Limits of each database can be adjusted for optimum performance and more virtual machines can be introduced into the existing setup without affecting the performance of the Production server.


Model Proliant BL685c G1

Processor Dual-Core AMD Opteron™ Processor

8218 Memory 65533 MB

Storage Array Symmetrix VMAX40K Disk FC 15K rpm 450 GB drives

Protocol Fibre Channel


Software Stack:

Host ESX Server

lfqu1063( Parent) VM lfqu1231

( Child1) VM lfqu2048

(Child2) VM lfqu2049

(Child3) Storage group

lfqu1063_p_vm_stor

lfqu1231_child_vm1

lfqu2048_child_vm2

lfqu2049_child_vm3

Storage group type cascaded child child child

O/S ESXi 5.0 Windows 2008 R2

Guest Windows 2008 R2

Guest Windows 2008 R2

Guest DB2 UDB database App A App B App C DB2 UDB version 9.7.3 9.7.3 9.7.3 VMAX

Enginuity 5876.229.145 5876.229.145 5876.229.145 5876.229.145

Multipathing Powerpath VE 5.7

(build 173) Powerpath VE 5.7



(build 173) VM Disk Access RDM RDM RDM RDM

Host I/O Limits setting

(in iops) 10000 4000 3000 3000 Host I/O

Limits type cascaded child child child

The test configuration had two virtual machines each running DB2 for LUW 9.7.3 on

Windows 2008 O/S and on separate storage groups, both of which were members of a cascaded storage group of an ESXi 5.0 server hosting these virtual machines. A third virtual machine having DB2 for LUW on Windows 2008 was introduced into the setup such that the total Host I/O Limits set on the cascaded storage group did not change. A benchmark workload was run on the databases with 50 users. The baseline test was running with just two virtual machines and two DB2 databases with Host I/O Limits of 5000 IOPS each set on them. Then their Host I/O Limits were changed to 4000 and 3000 IOPS. A third virtual machine with DB2 database and a storage group with 3000 IOPS Host I/O Limit was added as a child to the parent storage group of the ESX server. The parent storage group had a Host I/O Limit of 10000 IOPS set on it.

Figure 14 shows a VMware snapshot showing the ESXi server(Parent storage group) and the virtual machines(child storage groups).


Figure 14 Vcenter snapshot showing ESX server and virtual machines

Parent Storage group

# symsg -sid 351 show lfqu1063_p_vm_stor Name: lfqu1063_p_vm_stor Symmetrix ID : 000195700351 Last updated at : Tue May 21 16:13:13 2013 Masking Views : Yes FAST Policy : No Host I/O Limit : Defined Host I/O Limit MB/Sec : NoLimit Host I/O Limit IO/Sec : 10000 Dynamic Distribution : Always Number of Storage Groups : 3 Storage Group Names : lfqu1231_child_vm1 (IsChild) lfqu2048_child_vm2 (IsChild) lfqu2049_child_vm3 (IsChild) Devices (3): { --------------------------------------------------------- Sym Device Cap Dev Pdev Name Config Sts (MB) --------------------------------------------------------- 060B N/A TDEV RW 983040 0617 N/A TDEV RW 983040 061B N/A TDEV RW 983040 }


Set Host I/O Limit on Child Storage group

# symsg -sid 351 -sg lfqu1231_child_vm1 set -iops_max 4000

Child Storage group

# symsg -sid 351 show lfqu1231_child_vm1 Name: lfqu1231_child_vm1 Symmetrix ID : 000195700351 Last updated at : Mon Apr 22 14:19:51 2013 Masking Views : Yes FAST Policy : No Host I/O Limit : Defined (Shared) Host I/O Limit MB/Sec : NoLimit (NoLimit) Host I/O Limit IO/Sec : 4000 (10000) Dynamic Distribution : Always Number of Storage Groups: 1 Storage Group Names : lfqu1063_p_vm_stor (IsParent) Devices (1): { --------------------------------------------------------- Sym Device Cap Dev Pdev Name Config Sts (MB) --------------------------------------------------------- 060B N/A TDEV RW 983040 }

The following procedure was followed:

• Ran a baseline benchmark workload with 50 users on the DB2 databases on the two Virtual machines with Host I/O Limits of 5000 IOPS on each of them and a cascaded Host I/O Limits of 10000 IOPS on the parent storage group. • Changed the Host I/O Limits on the above two virtual machines to 4000 IOPS and 3000 IOPS respectively. Added a third virtual machine with storage group having Host I/O Limits of 3000 IOPS to the parent storage group having Host I/O Limits of 10000 IOPS.


Test Results:

Cascaded - Parent Host I/O Limits at 10000 IOPS and 3 children with Host I/O Limits of 4000, 3000,and 3000 IOPS

DB2 Database

Virtual Machine

Host I/O Limits setting (in IOPS)

IOPS

Average I/O

response time(ms)

App A Lfqu1231 (Child1)

4000 3700 8

App B Lfqu2048 (Child2)

3000 2970 12.5

App C Lfqu2049 (Child3)

3000 2900 11.5

• Response time increases for the 2 Virtual Machines when the third Virtual

Machine is introduced. • The total IOPS is contained at 10000 IOPS.

The total IOPs of the environment is maintained at 10000 iops so that if this environment were to share front end VMAX ports with a Production application, the performance of the Production application will not be affected.

Cascaded - Parent Host I/O Limits at 10000 IOPs and 2 children with Host I/O Limits of 5000 iops each

DB2 Database Virtual

Machine Host I/O Limits setting (in iops)

IOPS Average I/O Response Time(ms)

App A Lfqu1231 ( Child1)

5000 4900 2.75

App B Lfqu2048 ( Child 2)

5000 4850 2.5


Test case 3 : Use Dynamic Host I/O Limits feature to redistribute and failover Host I/O Limits to online FA directors

This section shows the effect of dynamic distribution to distribute Host I/O Limits among online VMAX front end FA directors. Using this feature, Host I/O Limits are redistributed to online directors from directors that are offline or that have failed. Figure 15 shows the dynamic distribution of Host I/O Limits on FC protocol. Functional Overview:

Figure 15 Setup showing Dynamic Distribution of Host I/O Limits on Fibre channel protocol

The test case’s focus was to understand the feature of dynamic distribution of Host I/O Limits and to set the allocation of front end performance Host I/O Limits such that the Host I/O Limits are redistributed across all participating directors. Hardware Configuration:

Host AIX232

Model IBM 8202-E4C Processor PowerPC_POWER7 Memory 23424 MB

Storage Array Symmetrix VMAX40K


drives Protocol Fibre Channel


Software Stack:

Host AIX232 Databases AIX_232 ( App A)

O/S AIX 7.1 File system JFS2

DB2 UDB version 9.7.3 VMAX Enginuity 5876.229.145

Multipathing Powerpath 5.5 P 02 Storage Group lfqu1232_db2_stor

The test configuration consisted of a 23 GB IBM DB2 for LUW database running a benchmark workload with 50 users on an AIX P7 system. The Port group consisted of 4 VMAX ports all of which were associated with the storage group of the database on which the Host I/O Limits had been set. The dynamic redistribution was measured by bringing down the ports one at a time, and the response time was measured using Symmetrix Performance analyzer.

Set Dynamic distribution

#symsg -sid 351 -sg lfqu1232_db2_stor set -iops_max 14000 #symsg -sid 351 -sg lfqu1232_db2_stor set -dynamic OnFailure # symsg -sid 351 show lfqu1232_db2_stor Name: lfqu1232_db2_stor Symmetrix ID : 000195700351 Last updated at : Tue Apr 02 10:19:23 2013 Masking Views : Yes FAST Policy : No Host I/O Limit : Defined Host I/O Limit MB/Sec : NoLimit Host I/O Limit IO/Sec : 14000 Dynamic Distribution : OnFailure Number of Storage Groups : 0 Storage Group Names : N/A Devices (8): { --------------------------------------------------------- Sym Device Cap Dev Pdev Name Config Sts (MB) ---------------------------------------------------------


0482 N/A TDEV RW 244736 0483 N/A TDEV RW 244736 0484 N/A TDEV RW 244736 0485 N/A TDEV RW 244736 0486 N/A TDEV RW 244736 0487 N/A TDEV RW 244736 0488 N/A TDEV RW 244736 0489 N/A TDEV RW 244736 }

The following procedure was followed: • Started workload on the database with Host I/O Limits dynamic distribution set on the storage group. Dynamic distribution on a storage group cannot be set without setting Host I/O Limits first. • Brought down the FA ports, one at a time, and observed the IOPS getting redistributed over the ports which are online.

Test Results:

Without Dynamic distribution of Host I/O limits

Number of Front end director ports

online

Time at which taken

IOPS

Average I/O

response time(ms)

All 4 ports up 5.34 PM 14154 4 3 Ports up 5.36 PM 10484 5.5 2 Ports up 5.39 PM 7141 6

With Dynamic distribution of Host I/O Limits Number of Front

end director ports online

Time at which taken

IOPS Average I/O

response time(ms)

All 4 ports up 8.30 PM 14293 4.5 3 Ports up 8.32 PM 13991 5.85 2 Ports up 8.37 PM 14009 6.1


• The Host I/O Limits get redistributed seamlessly to the online FAs whenever

any of the FAs in the Port group go down. • Response time goes up after the redistribution because the existing FAs

have to handle more workload now.

14154

10484

7141

0

5000

10000

15000

5.34 PM 5.36 PM 5.39 PM

All 4 ports up 3 Ports up 2 Ports up

IOPS

Time

Without Dynamic distribution of Host I/O Limits

IOPS

14293 13991 14009

02000400060008000

10000120001400016000

8.30 PM 8.32 PM 8.37 PM

All 4 ports up 3 Ports up 2 Ports up

IOPS

Time

With Dynamic distribution of Host I/O Limits

IOPS


Test case 4: Use Dynamic Host I/O Limits feature to redistribute and failover Host I/O Limits to online FCoE directors This section shows the effect of dynamic redistribution when one or more front end fibre channel over Ethernet ports in a VMAX port group goes offline. The main purpose of this test is to show the support of Host I/O Limits on fibre channel over Ethernet protocol and Dynamic distribution and dynamic failover across FCoEs. Functional Overview: Figure 16 shows the dynamic distribution of Host I/O Limits on FCoE protocol.

Figure 16 Setup showing Dynamic Distribution of Host I/O Limits on Fibre channel over Ethernet protocol

The focus of the test case is to set the allocation of front end Host I/O Limits such that the Host I/O Limits are distributed across all participating fibre channel over Ethernet directors. Using this feature, Host I/O Limits are redistributed to online directors from directors that are offline or that have failed.



Host AIX122

Model IBM 9113-550 Processor PowerPC_POWER5 Memory 6720 MB

Storage Array Symmetrix VMAX40K


drives

Protocol Fibre Channel over

Ethernet

Software Stack:

The test configuration consisted of a 23 GB IBM DB2 for LUW database running a benchmark workload with 50 users on an AIX P5 system. The Port group consisted of 2 VMAX FCoE ports both of which were associated with the storage group of the database on which the Host I/O Limits had been set. The dynamic redistribution was measured by bringing down the ports one at a time, and measuring the response time.

The following procedure was followed:

• Started workload on the database with Host I/O Limits dynamic distribution set on the storage group. Dynamic distribution on a storage group cannot be set without setting Host I/O Limits first.

Host AIX122 Databases AIX_122

O/S AIX 6.1 File system JFS2

DB2 UDB version 9.7.3 VMAX Enginuity 5876.229.145

Multipathing Powerpath 5.5 P 02


• AIX_122 is a DB2 UDB database on AIX P5 which sees 2 FCoE ports, 11g:0 and 11h:0. Ran a benchmark workload with 50 users. Set iops_max at 6000 IOPS. Set dynamic to "OnFailure".

• Brought down one FCoE port and observed the IOPS get redistributed over to the port which is online.

Test Results:

Without Dynamic distribution of Host I/O limits Number of FCoE

director ports online

Time at which taken

IOPS Average I/O

response time(ms)

2 Ports up 11.45 AM 6023 3.5 1 Port up 11.48 AM 3003 6.5


Number of Front end director ports online

Time at which taken

IOPS Average I/O

response time(ms)

2 Ports up 12.15 PM 6010 3.8 1 Port up 12.19 PM 5935 6.5


• The Host I/O Limits seamlessly get redistributed to the online FCoE directors whenever any of the FCoE directors in the Port group go down.

• Response time goes up after the redistribution because the existing FCoE directors have to handle more workload now.

6023

3003

0

2000

4000

6000

8000

11.45 AM 11.48 AM

2 Ports up 1 Port up

IOPS

Time

Without Dynamic distribution of Host I/O limits

Series1

6010 5935

0

2000

4000

6000

8000

12.15 PM 12.19 PM

2 Ports up 1 Port up

IOPS

Time


IOPS


Conclusion

Host I/O Limits enable setting workload and bandwidth Limits on applications sharing resources on a VMAX. They also provide a mechanism for enforcing performance boundaries and service levels. The growth of storage needs, year after year, has made us realize the importance of efficient use of storage and more granular control of storage resources. Host I/O Limits allows us to make more economical and intelligent use of the resources of a Symmetrix VMAX, since multiple tenants or applications can share the available resources in an effective, easy and transparent manner. It also results in improved performance for the more critical applications, reduced overall application costs and an improved use of hardware resources. Management becomes easier as only the shared set of resources need to be maintained instead of multiple sets of resources. The overall energy costs are reduced because this solution allows for sharing of resources, as opposed to dedicating additional resources to individual applications.


References

1. Application workload control using Host I/O Limits for SQL Server on EMC Symmetrix VMAX.

2. Storage Provisioning with EMC Symmetrix Autoprovisioning Groups Technical note

optimizing application performance using vmax host i/o ... · unix, and windows (db2 for luw) in a...

Documents