dell reference configuration for a 12tb microsoft sql...

Database Solutions Engineering

Dell Product Group

Mayura Deshmukh

April 2013

Dell Reference Configuration for a 12TB

Microsoft SQL Server 2012 Fast Track

Data Warehouse

A Dell Technical White Paper

Dell Reference Configuration for 12TB Microsoft SQL Server 2012 Fast Track Data Warehouse

ii

This document is for informational purposes only and may contain typographical errors and

technical inaccuracies. The content is provided as is, without express or implied warranties of any

kind.

© 2013 Dell Inc. All rights reserved. Dell and its affiliates cannot be responsible for errors or omissions

in typography or photography. Dell, the Dell logo, and PowerEdge are trademarks of Dell Inc. Intel and

Xeon are registered trademarks of Intel Corporation in the U.S. and other countries. Microsoft,

Windows, and Windows Server are either trademarks or registered trademarks of Microsoft Corporation

in the United States and/or other countries. Other trademarks and trade names may be used in this

document to refer to either the entities claiming the marks and names or their products. Dell disclaims

proprietary interest in the marks and names of others.

February 2013 | Rev 1.0


iii

Contents Executive Summary .................................................................................................... 4

FTDW Reference Architectures Using PowerEdge R720xd Server .............................................. 4

12TB Dell R720XD FTDW Reference Architecture ................................................................. 5

Hardware Components .............................................................................................. 5

Internal Storage Controller (PERC H710P Mini) Settings ...................................................... 7

Application Configuration .......................................................................................... 9

Capacity Details .................................................................................................... 10

Performance Benchmarking ...................................................................................... 11

Conclusion ............................................................................................................. 13

References ............................................................................................................. 14

Tables Table 1: Dell Fast Track Reference Architectures for PowerEdge R720xd Server ............................. 4

Table 2: Tested Dell FTDW Reference Architecture Components ................................................ 5

Table 3: Mount Point Naming and Storage Enclosure Mapping .................................................... 9

Table 4: Capacity Metrics .............................................................................................. 10

Table 5: Performance Metrics ......................................................................................... 11

Figures Figure 1: Proposed Dell Fast Track Reference Architecture ....................................................... 5

Figure 2: Memory Slot Locations ........................................................................................ 7

Figure 3: Virtual Disk Settings ........................................................................................... 7

Figure 4: Internal Storage Controller Settings ....................................................................... 8

Figure 5: RAID Configuration ............................................................................................ 8

Figure 6: Storage System Components ................................................................................. 9

Figure 7: SQLIO Line Rate Test from Cache (Small 5MB File) .................................................... 12

Figure 8: SQLIO Real Rate Test from Disk (Large 25GB File) .................................................... 12


4

Executive Summary

The performance and stability of any data warehouse solution is based on the integration between

solution design and hardware platform. Choosing the correct solution architecture requires balancing

the application’s intended purpose and expected use with the hardware platform’s components. Poor

planning, bad design, and misconfigured or improperly sized hardware often lead to increased costs,

increased risks and, even worse, unsuccessful projects.

This white paper provides guidelines to achieve a compact, balanced, optimized 12TB Microsoft® SQL

Server® 2012 data warehouse configuration for Dell™ PowerEdge™ R720 and R720xd servers using

Microsoft Fast Track Data Warehouse (FTDW) principles. Benefits of implementing this reference

architecture include:

Achieve a balanced and optimized system at all levels of the stack by following hardware and

software best practices.

Avoid over-provisioning hardware resources to reduce costs.

Implement a tested and validated configuration with proven methodologies and performance

behaviors to help avoid the pitfalls of improperly designed and configured systems.

Easily migrate from a small- to medium-sized data warehouse configuration (5TB) to a large

data warehouse configuration (12TB).

Data center space comes at a premium. This configuration provides a compact, high-performance

solution for large data warehouses with 12TB of data or more.

FTDW Reference Architectures Using PowerEdge R720xd Server

The Microsoft FTDW reference architecture achieves an efficient resource balance between SQL Server

data processing capability and realized component hardware throughput to take advantage of improved

out-of-the-box performance.

As most data warehouse queries scan large volumes of data, FTDW system design and configuration are

optimized for sequential reads and are based on concurrent query workloads. Understanding

performance and maintaining a balanced configuration helps reduce costs by avoiding over provisioning

of components.

Dell provides various Fast Track reference architectures for SQL 2012 built using the Dell PowerEdge

12th Generation servers. These solutions are differentiated depending on the data warehouse capacity

and scan rate requirements. Table 1 summarizes FTDW configurations with Dell R720XD server.

Table 1: Dell Fast Track Reference Architectures for PowerEdge R720xd Server

Solution ID Server CPU Data Drives Rated Capacity

2457176 R720XD (2) Intel® Xeon® E5-2643 CPU @3.3GHz

(18) 900GB 10K SAS

12TB

The 12TB R720XD configuration described in this white paper is also available as a rapid deployment,

with hardware, software, and services included in the Dell™ Quickstart Data Warehouse Appliance 2000

(QSDW 2000). This configuration provides a low-cost and easier migration path for customers who want


5

to go from a 5TB to 12TB solution. For more information on Dell QSDW 2000, see Dell Quickstart Data

Warehouse Appliance.

12TB Dell R720XD FTDW Reference Architecture

The following sections of this paper describe the hardware, software, capacity, and performance characteristics of a 12TB Microsoft SQL Server 2012 FTDW solution with scan rates of about 2GBps using PowerEdge R720XD servers.

Hardware Components

Redundant and robust tests have been conducted on PowerEdge servers to determine best practices

and guidelines for building a balanced FTDW system. Table 2 provides the detailed hardware

configuration of the reference architecture.

Figure 1: Proposed Dell Fast Track Reference Architecture

Tested Dell Fast Track Reference Architecture Component Details

Table 2: Tested Dell FTDW Reference Architecture Components

Component Details

Server PowerEdge R720xd

CPU (2) Intel® Xeon® E5-2643 CPU @3.3GHz (HT Enabled)

Number of sockets used 2

Total Number of CPU Cores 8

Memory 128GB RAM (8 X 16GB DDR3 DIMMs @1600MHz)

http://content.dell.com/us/en/enterprise/by-service-type-application-services-business-intelligence-quickstart-data-warehouse

http://content.dell.com/us/en/enterprise/by-service-type-application-services-business-intelligence-quickstart-data-warehouse


6

Internal Hard Drives

22x 900GB 10K 2.5” SAS (18 data, 2 logs, 2 staging)

2x 900GB 10K 2.5” SAS (2 Hot Spares)

2x 900GB 10K 2.5” SAS (2 drives with OS) rear bay

Operating System Microsoft Windows® Server 2008 R2 SP1 Enterprise Edition

Database Software Microsoft SQL Server 2012 Enterprise Edition

PowerEdge R720xd Server

The PowerEdge R720xd server is a two-socket, 2U high-capacity, multi-purpose rack server offering an

excellent balance of internal storage, redundancy, and value in a compact chassis. For technical

specifications of the R720xd server, see the Power Edge R720xd Technical Guide.

Processors

The Fast Track Data Warehouse Reference Guide for SQL Server 2012 describes how to achieve a

balance between components such as storage, memory, and processors. To balance available internal

storage and memory for the PowerEdge R720xd, the architecture uses two Intel Xeon E5-2643 four-core

processors operating at 3.3GHz.

Memory

For SQL Server 2012 reference architectures, Microsoft recommends using 128GB to 256GB of memory

for dual-socket configuration. Selection of memory DIMMS will also play a critical role in the

performance of the entire stack.

This configuration was tested with various memory sizes running at different speeds—for example,

192GB running at 1333MHz, 192GB running at 1600MHz, 112GB running at 1600MHz, and so on. Using

DIMMs with memory rate of 1600MHz showed significant performance improvement (about 400MBs/s)

over DIMMS with memory rate of 1333MHz. In the test configuration, the database server is configured

with 128GB of RAM running at 1600 MHz to which create a well-balanced configuration.

To achieve 128GB of RAM on the PowerEdge R720xd server, place eight 16GB RDIMMS in slots A1-A4 and

B1-B4 (white connectors). See Figure 2: Memory Slot LocationsFigure 2 for memory slot locations.

http://www.support.dell.com/support/edocs/systems/per720/en/index.htm

http://msdn.microsoft.com/en-us/library/hh918452.aspx


7

Figure 2: Memory Slot Locations

Internal Storage Controller (PERC H710P Mini) Settings

The Dell PERC H710P Mini is an enterprise-level RAID controller that provides disk management

capabilities, high availability, and security features in addition to improved performance of up to

6GB/s throughput. Figure 3 shows the management console accessible through the BIOS utility.

Figure 3: Virtual Disk Settings

Stripe element size

By default, the PERC H710P Mini creates virtual disks with a segment size of 64KB. For most workloads,

the 64KB default size provides an adequate stripe element size.

Read policy

The default setting for the read policy on the PERC H710P Mini is “adaptive read ahead.” This

configuration was tested with “adaptive read ahead,” “No read ahead,” and “Read Ahead” settings.


8

During testing, it was observed that the default setting of “adaptive read ahead” gave the best

performance.

Figure 4: Internal Storage Controller Settings

RAID configuration

When deploying a new storage solution, selecting the appropriate RAID level is a critical decision that

impacts application performance. The FTDW configuration proposed in this paper uses RAID 1 disk

groups for database data files and database log files, nine RAID 1 data disk groups, and one RAID 1 log

disk group, each created with a single virtual disk. Additionally, two drives in RAID 0 are assigned as a

staging area. Figure 5 shows the proposed RAID configuration.

Figure 5: RAID Configuration

RAID 1Data 1

RAID 1Data 2

RAID 1Data 3

RAID 1Data 4

RAID 1Data 5

RAID 1Data 6

RAID 1Data 7

RAID 0Stage

RAID 1Data 9

RAID 1Data 8

RAID 1Logs

Hot Spares

OS

H710P Mini Monolithic

Rear Bay Drives

Drive slot configuration:

Slots 0-17: Nine RAID 1 disk groups were created, each configured with a single virtual disk

dedicated for the primary user data

Slots 18-19: One RAID 1 disk group created from two disks and a single virtual disk dedicated to

host the database log files

Slots 20-21: RAID 0 disk group created from two disks dedicated for staging

Slots 22-23: Remaining two disks assigned as global hot spares

Slots 24-25 (rear bay drives): One RAID 1 disk group for operating system

For FTDW architectures, it is recommended to use mount-point rather than drive letters for storage

access. It is also important to assign the appropriate virtual disk and mount-point names to the


9

configuration to simplify troubleshooting and performance analysis. Mount-point names should be

assigned in such a way that the logical file system reflects the underlying physical storage enclosure

mapping. Table 3 shows the virtual disk and mount-point names used for the specific reference

configuration and the appropriate storage layer mapping. All of the logical volumes are mounted to the

C:\FT folder.

Table 3: Mount Point Naming and Storage Enclosure Mapping

Disk Group

Virtual Disk

Virtual Disk Label Logical Label

Full Volume Path Capacity

1 1 Cage1-Card1-vData1 Data1 C:\FT\PRI\Cage1-Card1-vData1 837.75 GB









10 10 Cage1-Card1-vLog Log C:\FT\LOG\Cage1-Card1-vLog 837.75 GB

11 11 Cage1-Card1-Stage Stage C:\FT\Stage\Cage1-Card1-Stage 1675.5 GB

Figure 6 represents the storage system configuration for the proposed FTDW reference architecture.

Figure 6: Storage System Components

Data file 1 - 9

User database Temp DB

SQL SERVER

Logs

Data file 1-9

Non-DB Stage

INTERNAL STORAGE

RAID 1

RAID 1

Virtual disk group 1-9

Virtual disk group 10

RAID 0Virtual disk group 11

The production, staging, and system temp databases are deployed per the recommendations provided

in the Fast Track Data Warehouse Reference Guide for SQL Server 2012.

Application Configuration

The following sections explain the settings applied to operating system and database layers.

http://msdn.microsoft.com/en-us/library/hh918452.aspx


10

Windows Server 2008 R2 SP1

Enable Lock Pages In Memory to prevent the system from paging memory to disk. For more

information, see How to: Enable the Lock Pages in Memory Option.

SQL Server Configuration

The following startup options were added to the SQL Server Startup options:

-E: This parameter increases the number of contiguous extends that are allocated to a

database table in each file as it grows to improve sequential access.

-T1117: This trace flag ensures the even growth of all files in a file group when auto growth is

enabled. It should be noted that the FTDW reference guidelines recommend pre-allocating the

data file space rather than allowing auto-grow.

SQL Server Maximum Memory: FTDW for SQL Server 2012 guidelines suggest allocating no more than

92% of total server RAM to SQL Server. If additional applications will share the server, then adjust the

amount of RAM left available to the operating system accordingly. For this reference architecture, the

maximum server memory was set at 119808 MB (117GB).

Resource Governor: For SQL Server 2012, Resource Governor provides a maximum of 25% of SQL Server

memory resources to each session. The Resource Governor setting can be used to reduce the maximum

memory consumed per query. While it can be beneficial for many data warehouse workloads to limit

the amount of system resources available to an individual session, this is best measured through

analysis of concurrent query workloads. This configuration was tested with both 25% and 19% memory

grant, and the 25% setting was found to be optimal for the proposed configuration. For more

information, see Using the Resource Governor.

Max Degree of Parallelism: The SQL Server configuration option Max degree of parallelism controls

the number of processors used for the parallel execution of a query. For the configuration, settings of

12 and 0 were tested. The default setting of 0 provided maximum performance benefits. For more

information, see Maximum degree of parallelism configuration option.

Capacity Details

Table 4Table 4 shows the capacity metrics reported for the recommended reference configuration.

Table 4: Capacity Metrics

Metric Value Description

Raw Data Space (GB) 7695 Raw mirrored/striped space allocated for data

Raw User Database Space (GB)

5771.3 Raw user space (without compression) available

after allocating space for tempdb

Maximum User Database Capacity (TB)

19

Raw user space with compression (compression factor=3.5).

This is an estimate for the largest amount of user data the system will hold.

FTDW Rated Data Warehouse Capacity (TB)

12 This capacity rating is based on “up-to” capacity

but adjusted to account for FTDW rated I/O.

http://go.microsoft.com/fwlink/?LinkId=141863

http://msdn.microsoft.com/en-us/library/ee151608.aspx

http://support.microsoft.com/kb/2023536


11

Performance Benchmarking

Microsoft FTDW guidelines help to achieve optimized database architecture with balanced CPU and

storage bandwidth. Table 5 shows the performance numbers reported for the recommended reference

configuration.

Table 5: Performance Metrics

Metric Value Description

FTDW Rated I/O (MB/s) 1909 Core performance metric for validation; average

of physical and logical I/O

Benchmark Scan Rate Logical (MB/s)

2164 Reflects actual user query throughput, which

includes reads from RAM/Buffer cache

Benchmark Scan Rate Physical (MB/s)

1654 Reflects physical I/O read from disk during

benchmark

FTDW Peak I/O (MB/s) 3481 Maximum observed I/O rate

FTDW Rated CSI (MB/s) 4337.5 Represents potential throughput using

Columnstore Index

The following sections describe the detailed performance characterization activities carried out for the

validated Dell Microsoft FTDW reference architecture.

Baseline Hardware Characterization Using Synthetic I/O

The goal of hardware validation is to determine actual baseline performance characteristics of key

hardware components in the database stack to ensure that system performance is not bottlenecked in

intermediate layers.

The disk characterization tool, SQLIO, was used to validate the configuration. The results in Figure 7

show the maximum baseline that the system can achieve from a cache (called Line Rate). A small file is

placed on the storage, and large sequential reads are issued against it with SQLIO. This test verifies the

maximum bandwidth available in the system to ensure no bottlenecks are within the data path.


12

Figure 7: SQLIO Line Rate Test from Cache (Small 5MB File)

POWER EDGE R720-XD


SQL Server 2012

Intel E5-26434 core CPU

Single RAID 1 Disk GroupSynthetic I/O rate: 375 MB/s

PERC H710P Mini ControllerSynthetic I/O rate: 2674 MB/s

SQL Server 2012 EnterpriseDual Socket Intel Quad core E5-2643

Aggregate Synthetic I/O rate: 2674 MB/s Intel E5-26434 core CPU

PERC H710P Mini

Controller

INTERNAL STORAGE

RAID 1RAID 1RAID 1RAID 1RAID 1RAID 1RAID 1RAID 1RAID 1

RAID 0RAID 1

The second synthetic I/O test with SQLIO was performed with a large file to ensure reads are serviced

from the storage system hard drives instead of from cache. Figure 8 shows the maximum real rate that

the system is able to provide with sequential reads.

Figure 8: SQLIO Real Rate Test from Disk (Large 25GB File)

POWER EDGE R720-XD


SQL Server 2012

Intel E5-26434 core CPU

Single RAID 1 Disk GroupSynthetic I/O rate: 294 MB/s

PERC H710P Mini ControllerSynthetic I/O rate: 2616 MB/s

SQL Server 2012 EnterpriseDual Socket Intel Quad core E5-2643

Aggregate Synthetic I/O rate: 2613 MB/s Intel E5-26434 core CPU

PERC H710P Mini

Controller

INTERNAL STORAGE

RAID 1RAID 1RAID 1RAID 1RAID 1RAID 1RAID 1RAID 1RAID 1

RAID 0RAID 1

FTDW Database Validation

The performance of a FTDW database configuration is measured using two core metrics: Maximum CPU

Consumption Rate (MCR) and Benchmark Consumption Rate (BCR).

MCR - MCR indicates the per-core I/O throughput in MB or GB per second. This is measured by

executing a pre-defined query against the data in the buffer cache, and then measuring the


13

time taken to execute the query against the amount of data processed in MB or GB. For the

validated configuration with two Intel E5-2643 four-core processors, the system aggregate MCR

was 2488 MB/s. The realized MCR value per core was 311 MB/s.

BCR - BCR is calculated in terms of total read bandwidth from the storage hard drives—not

from the buffered cache as in the MCR calculation. This is measured by running a set of

standard queries specific to the data warehouse workload. The queries range from I/O

intensive to CPU and memory intensive, and provide a reference to compare various

configurations. For the validated FTDW configuration, the aggregate BCR was 1909 MB/s.

During the evaluation cycle, the system configuration was analyzed for multiple query variants

(simple, average, and complex) with multiple sessions and different degrees of parallelism

(MAXDOP) options to arrive at the optimal configuration. The evaluation results at each step

were validated and verified jointly by Dell and Microsoft.

FTDW Database Validation with Column Store Index (CSI)

SQL Server 2012 implements CSI technology as a nonclustered indexing option for pre-existing tables.

Significant performance gains are often achieved when CSI query plans are active, and this

performance can be viewed as incremental to the basic system design.

After the test configuration was validated, CSI was added. Then, the same set of I/O and CPU-intensive

queries were executed to compare throughput achieved using CSI. Throughput rating of 4337.5 MB/s

was achieved for CSI-enhanced benchmarks. These numbers can be used to approximate the positive

impact to query performance expected under a concurrent query workload.

Conclusion

The Dell Microsoft FTDW architecture provides a uniquely well-balanced data warehouse solution. By

following best practices at all stack layers, a balanced data warehouse environment can be achieved

with a greater performance benefits than traditional data warehouse systems.


14

References

Dell SQL Server Solutions

www.dell.com\sql

Dell Services

www.dell.com\services

Dell Support

www.dell.com\support

Microsoft Fast Track Data Warehouse and Configuration Guide Information

www.microsoft.com/fasttrack

An Introduction to Fast Track Data Warehouse Architectures

http://msdn.microsoft.com/en-us/library/dd459146.aspx

How to: Enable the Lock Pages in Memory Option


SQL Server Performance Tuning & Trace Flags


Using the Resource Governor


Maximum degree of parallelism configuration option

support.microsoft.com/kb/2023536

Power Edge R720xd Technical Guide


http://www.dell.com/sql

http://www.dell.com/services

http://www.dell.com/support

http://www.microsoft.com/fasttrack

http://msdn.microsoft.com/en-us/library/dd459146.aspx






dell reference configuration for a 12tb microsoft sql...

Documents