benchmark testing results: unparalleled scalability of itron …€¦ · 10 million meters on sql...

18
i Benchmark Testing Results: Unparalleled Scalability of Itron Enterprise Edition on SQL Server Benchmark Testing Confirms That Itron Enterprise Edition TM Meter Data Management Can Support 10 Million Meters on a SQL Server 2008 R2 Enterprise Edition Technical White Paper Published: May 2011 Applies to: Microsoft SQL Server 2008 R2 Enterprise Edition Abstract In January 2011, Itron worked with Microsoft to conduct a series of high-volume data tests on Itron Enterprise Edition™ (IEE) Meter Data Management (MDM) version 7.0 (v7.0) service pack 3 (SP3) running on Microsoft SQL Server 2008 R2 Enterprise Edition, with the goal of replicating the genuine demands of a 10 million meter deployment on commodity server hardware. The test results overwhelmingly confirm that IEE MDM v7.0 SP3 is a highly scalable system, capable of supporting 10 million meters on SQL Server 2008 R2 Enterprise Edition under real-world conditions. IEE MDM has powerful processing capabilities and the ability to scale on SQL Server—along with reduced licensing, hardware, administration, and support fees—that not only meet the current needs of large utility companies, but can also scale as utilities deploy increasingly sophisticated large-scale Advanced Metering Infrastructure (AMI) solutions. Utilities can be confident that IEE MDM v7.0 SP3, deployed on SQL Server 2008 R2, provides a reliable, cost- effective, low-maintenance meter data management solution that can support even their largest and most process-intensive deployments.

Upload: others

Post on 27-Jul-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Benchmark Testing Results: Unparalleled Scalability of Itron …€¦ · 10 million meters on SQL Server 2008 R2 Enterprise Edition. Overview of IEE MDM As a proven out-of-the-box

i

Benchmark Testing Results:

Unparalleled Scalability of Itron Enterprise

Edition on SQL Server

Benchmark Testing Confirms That Itron Enterprise EditionTM Meter Data Management Can Support 10 Million Meters on a SQL Server 2008 R2 Enterprise Edition

Technical White Paper Published: May 2011 Applies to: Microsoft SQL Server 2008 R2 Enterprise Edition Abstract

In January 2011, Itron worked with Microsoft to conduct a series of high-volume data tests on Itron Enterprise

Edition™ (IEE) Meter Data Management (MDM) version 7.0 (v7.0) service pack 3 (SP3) running on

Microsoft SQL Server 2008 R2 Enterprise Edition, with the goal of replicating the genuine demands of a 10

million meter deployment on commodity server hardware.

The test results overwhelmingly confirm that IEE MDM v7.0 SP3 is a highly scalable system, capable of

supporting 10 million meters on SQL Server 2008 R2 Enterprise Edition under real-world conditions. IEE MDM

has powerful processing capabilities and the ability to scale on SQL Server—along with reduced licensing,

hardware, administration, and support fees—that not only meet the current needs of large utility companies,

but can also scale as utilities deploy increasingly sophisticated large-scale Advanced Metering Infrastructure

(AMI) solutions.

Utilities can be confident that IEE MDM v7.0 SP3, deployed on SQL Server 2008 R2, provides a reliable, cost-

effective, low-maintenance meter data management solution that can support even their largest and most

process-intensive deployments.

Page 2: Benchmark Testing Results: Unparalleled Scalability of Itron …€¦ · 10 million meters on SQL Server 2008 R2 Enterprise Edition. Overview of IEE MDM As a proven out-of-the-box

Benchmark Testing Results: Itron Enterprise Edition Meter Data Management and Microsoft SQL Server 2008R2 Enterprise Edition

i

©2011 Microsoft Corporation. All rights reserved. This document is provided “as-is.” Information and views

expressed in this document, including URL and other Internet Web site references, may change without notice.

You bear the risk of using it.

This document does not provide you with any legal rights to any intellectual property in any Microsoft product.

You may copy and use this document for your internal, reference purposes.

Page 3: Benchmark Testing Results: Unparalleled Scalability of Itron …€¦ · 10 million meters on SQL Server 2008 R2 Enterprise Edition. Overview of IEE MDM As a proven out-of-the-box

Benchmark Testing Results: Itron Enterprise Edition Meter Data Management and Microsoft SQL Server 2008R2 Enterprise Edition

ii

Table of Contents

Introduction .................................................................................................................................................. 1

Overview of IEE MDM....................................................................................................................................... 2

The Benefits of SQL Server 2008 R2 Enterprise Edition .................................................................................... 3

Benchmark Testing Overview ........................................................................................................................ 4

Testing Goals .................................................................................................................................................... 5

The Benchmarking Test .................................................................................................................................... 5

Testing Setup ................................................................................................................................................. 7

Database and Application Servers .................................................................................................................... 8 Database Server Details ............................................................................................................................... 8 Application Server Details ............................................................................................................................ 9 Disk Configuration ........................................................................................................................................ 9

Testing Data ..................................................................................................................................................... 9

Initial Setup and Tuning .................................................................................................................................. 10

Target Times ................................................................................................................................................... 11

The 10 Million Meter Benchmark Testing Results ........................................................................................ 12

Summary ..................................................................................................................................................... 13

Additional Information ................................................................................................................................ 15

About Itron ..................................................................................................................................................... 15

About the Microsoft Worldwide Utilities Group ............................................................................................. 15

About ComTrade ............................................................................................................................................ 15

Page 4: Benchmark Testing Results: Unparalleled Scalability of Itron …€¦ · 10 million meters on SQL Server 2008 R2 Enterprise Edition. Overview of IEE MDM As a proven out-of-the-box

Benchmark Testing Results: Itron Enterprise Edition Meter Data Management and Microsoft SQL Server 2008R2 Enterprise Edition

1

Introduction Energy data drives the business of utilities, and with the emergence of smart metering and the

smart grid, deploying a robust meter data management solution has become essential to a utility’s

future success. The new smart systems gather tremendous amounts of data as millions of meters

report unique information in hourly and sub-hourly time intervals. The need to categorize

automated meter data by various functions, such as billing and distribution system management,

further adds to the challenge of dealing with so much data. Clearly, utilities need MDM systems

that can meet these challenges head on.

Itron Enterprise Edition MDM, the industry’s leading multi-vendor platform solution, is built to

maximize the value of utility metering through centralized collection, processing, storage, and

complex analysis of meter and other utility-related data. Itron has 55 IEE MDM customers, with 40

million meters worldwide. More than 30 IEE MDM customers are currently in production.

Itron has collaborated with Microsoft for more than 13 years to deliver AMI solutions that are easy

to access, deploy, and use. As a premier Global Independent Software Vendor for utilities and a

Microsoft Gold Certified Partner, Itron works with Microsoft on product roadmap alignment and

joint marketing, sales, and product development. Working with Microsoft provides Itron with the

resources, experience, and knowledge of a software industry leader to ensure that its solutions

deliver business results. The Microsoft and Itron partnership provides customers with the only

enterprise-class MDM system that supports SQL Server, a scalable and cost-effective combination

that is optimized to address the increasing requirements of tomorrow’s smart grid.

As part of their partnership, Itron and Microsoft worked together in January 2011 to execute a

high-volume benchmarking study in the Microsoft Enterprise Engineering Center (EEC) labs in

Redmond, Washington. The goal of the testing was to demonstrate the scalability, performance,

and cost efficiency of using IEE MDM v7.0 SP3 with SQL Server 2008 R2 Enterprise Edition on the

Windows Server 2008 R2 Enterprise operating system by replicating the demands of a 10 million

meter deployment.

A “day-in-the-life” scenario was designed to emulate daily data processing in a typical large utility.

To support time-based pricing, the scenario involved processing a mix of hourly, 15-minute interval,

and register data—from data import, validation, and estimation, to billing determinant calculation

and export. The benchmarking tests used standard commodity hardware (servers and storage) to

confirm a realistic cost of ownership. It was also assumed that 1% of incoming data would be

missing, as is typical in large-scale production.

The results of the benchmarking tests were impressive, clearly confirming that IEE MDM is capable

of supporting 10 million meters on a SQL Server database. The tests validated the capacity of IEE

MDM running on SQL Server to process up to 500 million meter readings in less than three hours,

and confirmed that the solution can deliver time-of-use (TOU) billing determinants for a 10 million

meter utility.

Page 5: Benchmark Testing Results: Unparalleled Scalability of Itron …€¦ · 10 million meters on SQL Server 2008 R2 Enterprise Edition. Overview of IEE MDM As a proven out-of-the-box

Benchmark Testing Results: Itron Enterprise Edition Meter Data Management and Microsoft SQL Server 2008R2 Enterprise Edition

2

“With a solution on SQL Server, our customers will save on reduced licensing, hardware, administration, and support fees, which translate into substantially lower costs over the life of the system.”

Julie Hance Vice President of Software Solutions

Itron, North America

Following are some highlights of the results:

The system successfully supported the “day-in-the-life” scenario for a 10 million meter

utility within the target time windows for all of the processes that were tested.

The test measured an average import speed of 37,500 meter reads/second, with peak

speeds of up to 47,500 reads/second after database tuning.

This means that the system validated and imported data for a 10 million meter utility in

under four hours, and was able to process 500 million meter readings in less than three

hours.

The billing determinant calculations averaged 70 bills/second.

This means that the process was completed in less than two hours for a 600,000 customer

monthly billing cycle, exceeding the daily requirements of a 10 million meter utility.

The combined critical extract, transform, and load (ETL) processes that summarize interval

and register data into daily usage values for reporting and analysis (such as demand

response and presentment) were completed in less than three hours.

These results clearly show that IEE MDM v7.0 SP3 is a highly scalable system, capable of supporting

10 million meters on SQL Server 2008 R2 Enterprise Edition.

Overview of IEE MDM As a proven out-of-the-box solution, IEE MDM is a mission-

critical application that delivers high-volume meter data to

enterprise applications. IEE MDM is a scalable, enterprise-wide,

open-architecture data-management solution that manages

commercial and industrial (C&I) and residential customers’

interval, register, and event meter data. The architecture of IEE

MDM is flexible and extensible, with a central repository that

makes it possible for utilities to interface large amounts of data

with multiple systems.

Page 6: Benchmark Testing Results: Unparalleled Scalability of Itron …€¦ · 10 million meters on SQL Server 2008 R2 Enterprise Edition. Overview of IEE MDM As a proven out-of-the-box

Benchmark Testing Results: Itron Enterprise Edition Meter Data Management and Microsoft SQL Server 2008R2 Enterprise Edition

3

“Together, Microsoft and Itron have focused on providing solutions that help utilities execute on their vision of the smart grid and IEE MDM based on SQL Server is a great example of a solution that provides the performance, administration, integration, security and availability that utilities need.”

Jon C. Arnold Managing Director

Power & Utilities, Microsoft

Itron offers commercial, off-the-shelf MDM, in addition to complementary applications to support

demand response management, multi-channel customer care, and revenue protection. Figure 1

shows IEE MDM running on a SQL Server database. Note that the IEE MDM architecture can be

easily adapted to serve different smart grid architectures.

Figure 1 IEE MDM with SQL Server

The Benefits of SQL Server 2008 R2 Enterprise Edition SQL Server 2008 R2 Enterprise Edition is comprehensive, integrated, and enterprise-ready data

management software for data management and analysis. It provides a reliable, cost-effective, low-

maintenance database framework for IEE MDM that can support

the largest and most process-intensive deployments. Hundreds of

enterprises are currently running 10 terabyte (TB) and larger

transactional databases on SQL Server.1

Running IEE MDM on SQL Server provides customers with many

benefits. These include:

Scalability and performance benefits.

SQL Server includes many new features that help IEE

MDM customers scale up and scale out as their

businesses grow.2

1 http://www.microsoft.com/presspass/itanalyst/docs/06-30-09EnterpriseDatabaseManagementSystems.pdf 2 http://www.microsoft.com/sqlserver/2008/en/us/performance-scale.aspx

Page 7: Benchmark Testing Results: Unparalleled Scalability of Itron …€¦ · 10 million meters on SQL Server 2008 R2 Enterprise Edition. Overview of IEE MDM As a proven out-of-the-box

Benchmark Testing Results: Itron Enterprise Edition Meter Data Management and Microsoft SQL Server 2008R2 Enterprise Edition

4

Lower hardware costs.

SQL Server runs on standard commodity server hardware, dramatically lowering the total

cost of ownership (TCO) for utilities.

Lower software costs.

With SQL Server, customers can typically enjoy a 3:1 reduction over the largest competitor

in licensing costs.3

Simpler systems management and lower staffing costs.4

SQL Server database administrators (DBAs) can typically manage three times as many

physical databases as a competitor’s DBAs.5

Lower maintenance costs.6

SQL Server first-year maintenance costs are up to 77% less than those of the largest

competitor.

Fewer security vulnerabilities.

SQL Server has consistently had fewer security vulnerabilities than the largest competitor.7

With SQL Server, IEE MDM customers can save with reduced licensing, hardware, administration,

and support fees, which translate into substantially lower costs over the life of the system.

Benchmark Testing Overview As part of its ongoing commitment to provide customers with a truly scalable MDM system, Itron

conducts regular scale and performance testing of IEE MDM functionality. The in-house Itron

performance and scalability testing uses two 5 million meter systems running on Oracle 10g

Release 2:

One system used for testing individual components.

A second long-running system that runs a “day-in-the-life” scenario that is designed to

emulate the business-process environment in a typical large utility. The scenario consists

of more than 20 processes that are completed every day for a 5 million meter AMI

deployment. It is also assumed that 1% of incoming data is missing; this puts additional

strain on the system, but is typical for normal utility operation.

Itron has observed that testing with partners like Microsoft brings significant benefits, including

independent verification of test results, access to expertise, and access to hardware. Therefore, in

January of 2011, Itron worked with Microsoft at their Enterprise Engineering Center lab in

3 http://www.microsoft.com/sqlserver/2008/en/us/value-calc.aspx 4 http://www.microsoft.com/sqlserver/2008/en/us/compare-oracle.aspx 5 http://www.alinean.com/PDFs/Microsoft_SQL_Server_and_Oracle-Alinean_TCA_Study_2010.pdf 6 http://www.microsoft.com/sqlserver/2008/en/us/compare-oracle-calc.aspx

7 NIST National Vulnerability Database, http://nvd.nist.gov/

Page 8: Benchmark Testing Results: Unparalleled Scalability of Itron …€¦ · 10 million meters on SQL Server 2008 R2 Enterprise Edition. Overview of IEE MDM As a proven out-of-the-box

Benchmark Testing Results: Itron Enterprise Edition Meter Data Management and Microsoft SQL Server 2008R2 Enterprise Edition

5

Redmond, Washington, to test IEE MDM at 10 million meters on SQL Server 2008 R2 Enterprise

Edition and Windows Server 2008 R2 Enterprise. Itron was assisted by ComTrade, a testing partner

with extensive experience with IEE MDM.

Testing Goals Itron and Microsoft wanted to confirm that the IEE MDM functionality could achieve acceptable

performance at the 10 million meter scale on SQL Server 2008 R2. The goal of the benchmark

testing was therefore to validate and import data for a typical 10 million meter utility within a four-

hour time window, which represents the amount of time customers typically require to meet their

strict daily billing process timelines.

Acceptable performance was defined as:

Reading import rates greater than 28,000 reads/second.

A billing determinant calculation rate greater than 45 bills/second.

Itron and Microsoft also set the goal of identifying and taking advantage of any SQL Server 2008 R2

Enterprise Edition performance improvement opportunities.

The Benchmarking Test To ensure that the testing was as realistic as possible, Itron and Microsoft used a “day-in-the-life”

scenario. This scenario included all critical automated processes that are performed by IEE MDM on

a regular basis in an AMI system:

Data validation, estimation, and import, with 1% of incoming data missing

Calculation of TOU billing determinants from interval data

These processes form the core of meter-to-cash operation in any AMI system.

In addition, Itron included two ETL processes in the benchmark testing:

One ETL process—which occurred in the time between the data import and billing

calculations—summarized interval and register data into daily usage values. This process

produces results needed by the validation, estimation, and rate calculations.

A second ETL process—which occurred after the billing calculations were completed—

calculated the metadata that was required for the next day’s data estimations, for billing

calculations, and for demand-response programs and customer presentment.

Page 9: Benchmark Testing Results: Unparalleled Scalability of Itron …€¦ · 10 million meters on SQL Server 2008 R2 Enterprise Edition. Overview of IEE MDM As a proven out-of-the-box

Benchmark Testing Results: Itron Enterprise Edition Meter Data Management and Microsoft SQL Server 2008R2 Enterprise Edition

6

Table 1 summarizes the processes used in the benchmarking test.

Table 1 Testing processes

AMI Reading Import (ARI)

Import, validate, and load 24 hours of data from a full

population of AMI meters into the database.

Estimate and load data for meters, with 1% of population

missing.

AMI Billing Export (ABE)

Calculate and export TOU billing determinants for one full

bill cycle for 6% of meter population (slightly more than

the 5% that is typically used by utilities to rotate through

the entire population in 20 working days).

Daily Reading Summary (DRS)

Calculate daily totals, peaks, and reference-period usage

for all channels (interval and register), and store the data

in warehouse tables for analytic and web use.

Like Day Algorithm (LDA) Pre-calculate “like” days for demand-response events,

estimations, and analytic uses.

To validate data, the tests used the validation rules shown in Table 2.

Table 2 Validation rules used to validate data

Gap Check

(GapCheck) Checks for missing intervals

Usage High Limit

(UsageHighLimit) Checks if interval values exceed a high limit threshold

Usage Low Limit

(UsageLowLimit) Checks if interval values exceed a low limit threshold

Usage Tolerance

(UsageTolerancePercentDifference)

Checks if the sum of intervals between consecutive

register readings did not match the difference between

the register readings, within a small percentage difference

Page 10: Benchmark Testing Results: Unparalleled Scalability of Itron …€¦ · 10 million meters on SQL Server 2008 R2 Enterprise Edition. Overview of IEE MDM As a proven out-of-the-box

Benchmark Testing Results: Itron Enterprise Edition Meter Data Management and Microsoft SQL Server 2008R2 Enterprise Edition

7

Table 3 shows which validation rules were applied to the different parts of the meter population.

Table 3 Validation rules that were applied

Residential meters GapCheck, UsageHighLimit, and UsageLowLimit

Delivered channel of C&I meters GapCheck and UsageTolerancePercentDifference

(with the percent difference set to 1%)

Received channel of C&I meters GapCheck

Interval data failing the validation checks shown in Table 2 were considered invalid and were

estimated instead using the California historic rules. Using these rules, a missing interval is

estimated by averaging intervals from three “like” days from the most recent three months of data,

with no invalid and no estimated values allowed. “Like” days for this type of estimation are defined

as:

Weekdays, if the day to be estimated is a weekday.

Saturdays or Sundays, if the day to be estimated is a Saturday or a Sunday.

Holidays, Saturdays, or Sundays, if the day to be estimated is a holiday.

Note that IEE MDM supports a large catalog of validation rules and estimation rules. The California

historic rules were used in the benchmark testing because they are relatively popular, in addition to

being among the most computationally demanding rules.

Testing of these processes was first conducted on a 5 million meter environment (the standard

Itron in-house benchmarking test) using five application servers. The test was then run for the

target 10 million meters environment using nine application servers.

The benchmarking tests measured throughput in terms of meter readings per second (reads/sec)

for the import process, and service-point bills per second (bills/sec) for the export process.

During the benchmark testing, Itron worked closely with the Microsoft SQL Server Customer

Advisory Team (SQLCAT) to identify tuning opportunities and database code changes, all of which

have been released in subsequent hotfixes for IEE MDM v7.0 SP3 or are being integrated into

existing code for future release.

Testing Setup The benchmarking test was conducted in the EEC labs in Redmond, Washington, on standard,

commodity servers and storage to confirm a realistic cost of ownership for the solution.

Page 11: Benchmark Testing Results: Unparalleled Scalability of Itron …€¦ · 10 million meters on SQL Server 2008 R2 Enterprise Edition. Overview of IEE MDM As a proven out-of-the-box

Benchmark Testing Results: Itron Enterprise Edition Meter Data Management and Microsoft SQL Server 2008R2 Enterprise Edition

8

Database and Application Servers The test environment consisted of one database server and nine application servers (note that only

five application servers were used for preliminary testing at 5 million meters; otherwise, the

hardware configuration was the same for both benchmark tests).

Figure 2 shows the benchmark testing environment.

Figure 2 Benchmark testing environment

Database Server Details

The database server was connected to an EMC storage area network (SAN) that contains both

traditional spinning disks and solid-state drive (SSD) disks to take advantage of the advanced

storage features of both Fully Automated Storage Tiering (FAST) and FAST Cache.

The following are specifications of the database server:

HP ProLiant BL680c G5

Windows Server 2008 R2 Enterprise with SP1

SQL Server 2008 R2 Enterprise Edition

Intel Xeon E7450 2.4 GHz (24 cores)

128 GB RAM

EMC CLARiiON CX4-960 SAN

o RAID 10 pool of 9.6 TB

o RAID 5 pool of 32.2 TB

o 165 physical disks (150 for use, 10 for FAST cache, 5 spares)

o 1 TB FAST cache (10-200 GB flash drives)

Page 12: Benchmark Testing Results: Unparalleled Scalability of Itron …€¦ · 10 million meters on SQL Server 2008 R2 Enterprise Edition. Overview of IEE MDM As a proven out-of-the-box

Benchmark Testing Results: Itron Enterprise Edition Meter Data Management and Microsoft SQL Server 2008R2 Enterprise Edition

9

Application Server Details

The nine application servers were HP BladeSystems. Load balancing was applied across the

application servers by assigning a portion of the overall meter population to each server, and each

application server had a gigabit circuit to the database server.

The following are specifications of the nine application servers:

IEE MDM v7.0 SP3

(Note: To run the software on the available 64-bit application servers, a custom build of IEE

MDM version 7.0 SP3 was used; this version enabled IEE MDM to run as a 32-bit application

on a 64-bit operating system. This will be publicly available in an upcoming release.)

Windows Server 2008 R2 Enterprise with SP1

Intel Xeon 5150 2.66 GHz (four cores, two physical processors)

16 GB RAM

Disk Configuration

Disk configuration is a critical success factor in the performance and scalability of any database,

particularly large databases with high transaction rates like IEE MDM.

In the benchmark testing, the 36 TB of free space on the EMC SAN was partitioned into three logical

unit numbers (LUNs):

TempDB

Reading data

Master configuration data and metadata

The EMC Flash Cache was used to provide greater input/output (I/O) for “hotspots.” In other

words, no specific tables were assigned to SSDs. Instead, control of the data cached on the SSDs

was left entirely to the EMC Flash Cache software, which allocated space according to most

frequently used sectors.

Testing Data The system was configured to resemble the meter population of a large utility, with 85% residential

meters and 15% commercial meters.

Residential, or mass-market, meters were configured with one register and one 60 minute

interval channel.

Commercial, or C&I, meters were configured with two registers and two 15 minute interval

channels.

Page 13: Benchmark Testing Results: Unparalleled Scalability of Itron …€¦ · 10 million meters on SQL Server 2008 R2 Enterprise Edition. Overview of IEE MDM As a proven out-of-the-box

Benchmark Testing Results: Itron Enterprise Edition Meter Data Management and Microsoft SQL Server 2008R2 Enterprise Edition

10

To allow for meaningful validation, estimation, and billing determinant calculations, data was

generated using a library of 100 real customer usage profiles. Each meter was assigned a reference

profile and a globally unique scaling factor, ensuring that every meter had a unique profile.

Test databases were populated with three months of historic data prior to testing. This approach

was taken to address concerns that performance of the system might degrade over time as more

historical data is stored in the database. Table 4 provides the details of the testing data used.

Table 4 Details of testing data

5 million meters 10 million meters

Total channels 11,500,000 23,000,000

Total readings per day 251,750,000 503,500,000

Total historic readings 22,657,500,000 15,105,000,000

Initial Setup and Tuning The databases and application servers were set up in accordance with the IEE MDM configuration

guide.

Initial setup and tuning was performed at 5 million meters with five application servers. Meter

configuration data was loaded using the IEE MDM program-based configuration interface.

“Smoke” tests of all four key processes (ARI, ABE, DRS ETLs, and LDA ETLs: see Table 3) were

conducted, during which Itron and Microsoft personnel monitored and tuned performance.

Once smoke testing was completed, a full “day-in-the-life” series was performed, in which ARI, ABE,

DRS ETL, and LDA ETL processes were run in series. After several successful runs at 5 million meters,

the results shown in Table 5 were recorded for an average of five runs, with 1% of incoming data

missing.

Page 14: Benchmark Testing Results: Unparalleled Scalability of Itron …€¦ · 10 million meters on SQL Server 2008 R2 Enterprise Edition. Overview of IEE MDM As a proven out-of-the-box

Benchmark Testing Results: Itron Enterprise Edition Meter Data Management and Microsoft SQL Server 2008R2 Enterprise Edition

11

Table 5 Results for 5 million meters

Process Total elapsed time Throughput rate

Importing, validating, and

estimating missing or invalid

data (ARI)

1 hour, 58 minutes 35,490 reads/second

Billing, for 300,000 meters

(ABE) 1 hour, 36 minutes 52 bills/second

Summarizing interval and

register data into daily usage

values (DRS ETL)

2 hours, 26 minutes 1,313 channels/second

Pre-calculating “like” days

(LDA ETL) 42 minutes 2,249 channels/second

Following the initial tests at 5 million meters, several stored procedures within the IEE MDM

product were tuned to improve performance.

Target Times Target times were selected based on customer's requirements to validate, estimate, and import 24

hours of data, in addition to completing the DRS ETL by 08:00. Billing one-twentieth of customers

was to be accomplished in three hours, as per typical utility specifications.

When applied to a 10 million meter system, these requirements translate to the throughput rates

shown in Table 6.

Table 6 Target times and required throughput to achieve them for 10 million meters

Process Target time Target throughput rate

Import, validation, and

estimation of missing or

invalid data

5 hours 28,000 readings/second

Billing , for 600,000 meters 3 hours 55 bills/second

By subtracting the five hour target for import, validation, and estimation from the eight hour total

time between midnight and 08:00, testers deduced a target of three hours for the subsequent DRS

ETL that also needs to complete. The LDA ETL process is not required prior to billing.

Page 15: Benchmark Testing Results: Unparalleled Scalability of Itron …€¦ · 10 million meters on SQL Server 2008 R2 Enterprise Edition. Overview of IEE MDM As a proven out-of-the-box

Benchmark Testing Results: Itron Enterprise Edition Meter Data Management and Microsoft SQL Server 2008R2 Enterprise Edition

12

The 10 Million Meter Benchmark Testing Results To perform 10 million meter tests, an additional 5 million meters were added to the initial database

set up for the 5 million meter run, along with additional historic data (see Table 2). The

benchmarking test was then run on the nine application servers.

The results were impressive: the goals for the benchmark testing were easily exceeded, confirming

that IEE MDM is a highly scalable system capable of supporting 10 million meters on a SQL Server

2008 R2 Enterprise Edition database.

The results shown in Table 7 represent the average of five tests that were performed for the

complete “day-in-the-life” scenario, with 1% of data missing.

Table 7 Average results for 10 million meters

Process Total elapsed time Throughput rate

ARI 3 hours, 43 minutes

(target: 5 hours)

37,500 reads/second

(target: 28,000 reads/second)

ABE 1 hour, 29 minutes

(target: 3 hours)

70 bills/second

(target: 55 bills/second)

DRS ETL 1 hour, 58 minutes

(target: 3 hours) 3,248 channels/second

LDA ETL 60 minutes

(target: 2 hours) 3,194 channels/second

Table 8 summarizes the peak export and import results.

Table 8 Peak results summary for 10 million meters

Process

Goal

Peak results

Percent to goal reached

Import process 28,000 reads/second 47,500 reads/second 170%

Export process 55 bills/second 112 bills/second 204%

These results validate the capacity of IEE MDM v7.0 SP3 running on SQL Server 2008 R2 Enterprise

Edition to import, validate, and estimate more than 500 million meter readings in well less than six

hours; and 500 million reads/day also equates to a 20 million meter utility with all-hourly reads.

In some cases, the system took less time to process more data in the exact same processes on

identical hardware. Some of this difference can be explained by greater efficiency in larger batches,

while other improvements can be explained by the tuning that was made possible by the assistance

Page 16: Benchmark Testing Results: Unparalleled Scalability of Itron …€¦ · 10 million meters on SQL Server 2008 R2 Enterprise Edition. Overview of IEE MDM As a proven out-of-the-box

Benchmark Testing Results: Itron Enterprise Edition Meter Data Management and Microsoft SQL Server 2008R2 Enterprise Edition

13

of the Microsoft SQL Server team. The white paper that describes best practices for running IEE

MDM on SQL Server 2008 R2 Enterprise Edition (to be published June 2011) discusses this tuning.

The benchmark testing results also confirm the horizontal scalability of IEE MDM v7.0 SP3 on SQL

Server 2008 R2. The results in the 10 million meter tests were, in all cases, better than the results

for the 5 million meter tests, probably because of the addition of the four application servers.

Because of this horizontal scalability at the application layer, and the vertical scalability at the

database layer, Itron is confident that IEE MDM is capable of supporting even more than 10 million

meters on a SQL Server 2008 R2 Enterprise Edition database.

Summary The benchmark testing results overwhelmingly confirm that IEE MDM is a highly scalable system

capable of supporting more than 10 million meters on a SQL Server 2008 R2 Enterprise Edition

database on commodity hardware.

Highlights of the benchmark testing include the following results:

The system was able to support the “day-in-the-life” processes for a typical 10 million

meter utility within target time windows for all processes tested.

The test measured an average import speed of 37,500 meter reads/second, with peak

speeds of up to 47,500 reads/second after database tuning.

The system validated and imported data for a 10 million meter utility in less than four

hours.

At peak speed, the system was able to process 500 million meter readings in less than three

hours.

The billing determinant calculations averaged 70 bills/second, resulting in the completion

of the process in less than two hours for a 600,000 customer monthly billing cycle. This

exceeds the requirements of a 10 million meter utility.

Critical ETL processes that summarize interval and register data into daily usage values for

reporting and analysis were completed in less than three hours.

The data processing rates achieved by the system—500 million reads/day—equate to 20

million meter utility with all-hourly reads.

Page 17: Benchmark Testing Results: Unparalleled Scalability of Itron …€¦ · 10 million meters on SQL Server 2008 R2 Enterprise Edition. Overview of IEE MDM As a proven out-of-the-box

Benchmark Testing Results: Itron Enterprise Edition Meter Data Management and Microsoft SQL Server 2008R2 Enterprise Edition

14

The results show that IEE MDM, running on SQL Server 2008 R2, is capable of processing the daily

load of a typical 10 million meter utility—more than 500 million meter readings per day—in less

than three hours (after optimization), and is able to deliver TOU billing determinants in two hours

for a typical daily billing run.

The results also confirm the horizontal scalability of the solution. When four application servers

were added for the 10 million meter tests, the results were better than those for the initial 5

million meter tests. With this horizontal scalability of the application layer and the vertical

scalability of the database layer—in addition to the impressive benchmark testing results—Itron is

confident that the solution can scale even beyond 10 million meters.

Utilities can be confident that IEE MDM, deployed on SQL Server 2008 R2, provides a scalable, cost-

effective MDM solution that can meet their current requirements while making it possible for them

to address the increasing requirements of tomorrow’s smart grid.

Page 18: Benchmark Testing Results: Unparalleled Scalability of Itron …€¦ · 10 million meters on SQL Server 2008 R2 Enterprise Edition. Overview of IEE MDM As a proven out-of-the-box

Benchmark Testing Results: Itron Enterprise Edition Meter Data Management and Microsoft SQL Server 2008R2 Enterprise Edition

15

Additional Information The following references provide more information about Itron, Microsoft, and ComTrade.

About Itron Itron Inc. is a Microsoft certified partner and a leading technology provider to the global energy and

water industries. It is the world’s leading provider of intelligent metering, data collection, and utility

software solutions, with nearly 8,000 utilities worldwide relying on Itron technology to optimize the

delivery and use of energy and water. Itron products include electricity, gas, water, and heat

meters; data collection and communication systems, including automated meter reading (AMR)

and AMI; MDM and related software applications; and project management, installation, and

consulting services.

For more information, visit www.itron.com.

About the Microsoft Worldwide Utilities Group The Microsoft Worldwide Utilities Group offers platform and partner solutions that empower

utilities to thrive in today’s market-driven environment through optimized business operations in

customer care, generation, trading and risk management, transmission and distribution, regulatory

compliance, and enterprise services.

For more information, contact your local Microsoft Sales Representative, or visit

www.microsoft.com/utilities.

About ComTrade ComTrade is an international provider of IT solutions and software engineering services whose

clients include high-tech vendors, telecommunication companies, financial institutions, and the

public sector. With more than 1,000 professionals, ComTrade offers industry-leading expertise in

smart meter management, data storage management, embedded systems, network systems

management, gaming, telecommunications, e-solutions, and customized application development.

For more information, visit www.comtrade.com.