real time energy consumption measurements

7/31/2019 Real Time Energy Consumption Measurements

1/72

SMSM

Real-Time Energy

Consumption Measurementsin Data Centers

ASHRAE TC9.9 and The Green Grid


2/72

SMSM

Copyright 2010, The Green Grid The Green Grid Technical Forum

ASHRAE/TGG.2009.RealTimeEnergyConsumptionMeasurementsinDataCenters.Atlanta:American

SocietyofHeating,RefrigeratingandAirConditioningEngineers,Inc.;coauthoredbyTheGreenGrid,

Beaverton,

Oregon.

First printing released at the

ASHRAE Winter Annual Meeting,

Jan 23 27, 2010

2


3/72

SMSM

Copyright 2010, The Green Grid The Green Grid Technical Forum3

Gartner, IDC, Forrester and other analysts continue to rank

Power & Cooling (energy efficiency) as the #1 concern of

data center owners/operators

TGGs PUE is becoming the proxy for data center energy efficiency

The PUE cannot be reasonably determined if energy consumption in thedata center cannot be measured

EPA reports an average PUE of 2.04 (2009 data)

The book is designed to

Provide an overview of the state of energy consumption measurementsin the data center

Educate the data center owner/operator with respect to making real-timeenergy consumption measurements in the data center

Demonstrate how to consolidate the energy consumption data into asingle energy efficiency value, e.g., PUE

Provide guidance with respect to determining energy efficiency in mixed-use facilities

PURPOSE OF THE BOOK


4/72

SMSM


PART 1 BASI CS

Chapter 1 Introduction

Chapter 2 How, What, & Where to MeasureChapter 3 Measurement Devices

Chapter 4 Measurement Collection Systems Architecture andSoftware

PART 2 COOLI NG SYSTEMS Air Measurement s

Chapter 5 Air Handler Units

Chapter 6 Computer Room Units

PART 3 COOLI NG SYSTEMS Hydronic Measurement sChapter 7 Pumps

Chapter 8 Cooling Towers

Chapter 9 Chillers

Chapter 10 Heat Exchangers

TABLE OF CONTENTS

4


5/72

SMSM


PART 4 POWER SYSTEMS MEASUREMENTS

Chapter 11 Introduction to Critical Power Distribution

Chapter 12 Upstream Critical Power DistributionChapter 13 Uninterruptible Power Supply

Chapter 14 Computer Room Transformer and Power Distribution Unit

PART 5 I T SYSTEMS MEASUREMENTS

Chapter 15 Compute and Storage Systems

Chapter 16 Networking Systems

Appendix A PumpsAppendix B Chillers

Appendix C Mixed-Use Facilities

Appendix D Uninterruptible Power Supply

Appendix E Onsite Power Generation and CCHP in Data Center Applications

TABLE OF CONTENTS

5


6/72

SMSM


KEY CONTRI BUTORS

6


7/72

SMSM


KEY CONTRI BUTORS

7


8/72

SMSM


PART 1

BASI CS

8


9/72

SMSM


MEASUREMENT DEVI CES (Chapter 3)

John Bean (APC), presenter

9


10/72

SMSM


Part 1 Chapter 3 Sensors

3.1 Overview

There are numerous attributes within the data

center that require measurements Temperature

Pressure

Flow Rates

Voltage

Current

Humidity

Sensors come in a variety of connectivity styles:direct, analog (010vdc / 420ma) and serialcommunication (ModBus, BacNet, etc)

Introduce Terms: Span, Resolution, Accuracy


11/72


12/72


13/72

SMSM



3.4 Pressure There are several types of pressure sensors available

Bourdon Tube

Strain Gauge

Compound (vacuum and positive pressure)

Absolute Pressure

Gauge Pressure

Need to ensure wetted surface material is compatible with fluidto be measured

All types may use a local transmitter to convert weak signal tomore robust signal (010vdc or 420ma) to allow longer runsof wiring to measurement device


14/72

SMSM



3.5 Flow Liquid There are several types of liquid flow sensors available

Paddle Wheel

Turbine Wheel

Shedding Vortices

Venturi

Ultrasonic

CoriolisVariable Area

Need to assure wetted surface material is compatible with fluidto be measured



15/72

SMSM



3.6 Flow Gas There are several types of gas flow sensors available

Pitot Tube

Hot Wire Anemometer

Flow Nozzle

Cup Anemometer



16/72

SMSM



3.7 Current There are several types of current sensors available

Split Core

Solid Core

Resistor (voltage drop)

Clamp-On Split Core

Extra loops of conductor thru core act as current multiplier forcurrent transformers



17/72

SMSM



3.8 Voltage There are several types of voltage sensors available

Potential Transformer (high & medium voltage)

Voltage Divider

Voltage Transducer


18/72

SMSM



3.8 Power Monitor Typically use both current and voltage sensor and have

necessary processing means to determine power in real time.

Other common metrics from power meters may include KVA (apparent power)

KVAR (reactive power)

Power Factor

Individual Leg Currents

Individual Leg Voltages

May or may not be revenue grade

Typically may have local display along with

serial communication interface to BMS or

SCADA System


19/72

SMSM


MEASUREMENT COLLECTI ON SYSTEMSArchit ecture and Soft w are (Chapter 4)

Kenneth Uhlman, Ph.D, P.E., presenter

19


20/72

SMSM


Topics

Business Questions

Measurement Levels

Scalable Hardware / Software Architecture

20


21/72

SMSM


Business Questions

Does this data metering and monitoring strategy

support the organizations sustainability objectives?

Will this project support current and future regulatoryrequirements domestically and abroad?

Is there executive sponsorship to fund and support theproject?

How will this project be funded?

What is the ROI and risks?

Who currently pays the energy bills (IT or Facilities)?

Is there a plan to charge back energy (and carbon) to theend users?

Technology is not the limiting factor, but ratherorganizational structure and behavior

21


22/72

SMSM


Measurement Levels

Minimum Best State-of Item Practical Practical the-artLinked IT / Facility Business Goals

Metrics

Measure and log metrics

Online data acquisition ?

MonitoringSeparate views (IT & Facilities) Linked IT/Facility dashboards Linked Business / IT / Facility dashboards

Proactive St rat egies

Server managing servers Auto-discovery of IT / Facility devices Proactive IT/Facility

Proactive IT/Facility/Smart Grid 22


23/72

SMSM


Typical Data Center Meters

FACILITY

HEALTH

CAPACITY EFFICIENCY

M MI T M

M

MM

M

M

M

MM

MM

M

MM

M

MM

MM

M

MM

MM

M

M

Facil i t ies Ut ili t ies

Data Center Meters

Many data centers have plentyof meters for each sub-system

Typically installed for capacityor component health, notenterprise efficiency

May be able to aggregateand recast data to measureand log efficiency

Solutions at various price pointsto meet business objectives

23


24/72

SMSM


Enterprise Energy Dashboard

While a single-pane ofglass may be an ultimate

goal, there are manytactical activities forimprovement

The cost for enterpriseviews continues todecrease with the

proliferation of networksand web-based equipmentand gateways

Initiate program withexisting metering datawhere available

24


25/72

SMSM


Sensing Hierarchy

IT and Facilities equipmenthave different constraints bare implemented similarly

The application dictates realtime monitoring acquisitionrates from seconds, minutes

days, weeks or months- IT Applications- Networks- Power or cooling systems

25


26/72

SMSM


Collecting Data Center

Telemetry in an Enterprise

Sites may be internal orexternal, domestic orinternational

Network security is crucial

Typically many vendors andvintages of Power, BuildingAutomation, HVAC, Security,IT, etc.

Local and central storage ofdata and acquisition rates

26


27/72

SMSM


Managed server monitored

by a manager server

Proactive management ofservers enables IT to matchoptimal energy rates, facilitycapacities and carbon

footprint Virtualization enables IT

loads to be shifted globally

Facility systems must be

linked to process

27


28/72

SMSM


Data Collector Software

Architecture

Is the heart of a linkedsystem

Secure and customizedscreens for:

- Executives- IT- Facilities

Proactively optimize IT,Facilities and Utilities tomeet the organizationsbusiness objectives

28


29/72

SMSM


PART 2

COOLI NG SYSTEMS AirMeasurements

29


30/72

SMSM


AI R HANDLERS (Chapt er 5)

COMPUTER ROOM UNI TS (Chapter 6)

Daryn Cline (Evapco), presenter

30


31/72

SMSM


Chapter 5 Air Handlers

5.1 Overview

5.2 Measurement Levels Chapter 6 Computer Room Units

6.1 Overview

6.2 Measurement Levels

Part 2 Cooling Systems

Air Measurements


32/72

SMSM


Chapter 5 Air Handlers 5.1 Overview

Part 2 Cooling Systems-

Air Measurements


33/72

SMSM


Chapter 5 Air Handlers 5.2 Measurement Levels


Air Measurements


34/72

SMSM


Chapter 6 Computer Room Units 6.1 Overview

CRAC Computer Room Air Conditioners

Direct Expansion (DX)

Chilled Water

Dual Source -Chilled Water Primary, DX Backup

Downflow -raised floor

Upflow -free discharge or ducted

Ceiling units

Environmental Control

Humidity Control RH, Dew point

Sensible Heat


Air Measurements


35/72

SMSM



35

MM =Measuring Points for Power Consumption


36/72

SMSM



36


37/72

SMSM


PART 3

COOLI NG SYSTEMS HydronicMeasurements

37


38/72

SMSM


PUMPS (Chapt er 7)

COOLI NG TOWERS (Chapt er 8)

CHI LLERS (Chapter 9)

HEAT EXCHANGERS (Chapter 10)

Daryn Cline (Evapco), presenter

38


39/72

SMSM



Hydronic Measurements Chapter 7 Pumps

7.1 Overview

7.2 Measurement Levels-Electrical

7.3 Measurement Levels-Fluid Chapter 8 Cooling Towers

8.1 Overview


Chapter 9 Chillers9.1 Overview


Chapter 10 Heat Exchangers

10.1 Overview10.2 Measurement Levels


40/72

SMSM


Chapter 7 Pumps 7.1 Overview


Hydronic Measurements


41/72

SMSM


Chapter 7 Pumps

7.2 Measurement Levels-Electrical



P t 3 C li S t


42/72

SMSM


Chapter 7 Pumps 7.2 Measurement Levels-Electrical



P t 3 Cooli g S t


43/72

SMSM


Chapter 7 Pumps 7.3 Measurement Levels-Fluid





44/72

SMSM


Chapter 8 Cooling Towers 8.1 Overview-Open Counter and Cross Flow Cooling Towers

8.1 Overview-Closed Counter and Combined Flow Circuit Cooling Towers

Part 3 Cooling Systems Hydronic Measurements



45/72

SMSM


Chapter 8 Cooling Towers 8.2 Measurement Levels




46/72

SMSM


Chapter 8 Cooling Towers 8.2 Measurement Levels

8.2.1 Minimum Practical Measurement-True RMS Power

8.2.2 Best Practical Measurement-Current Transducers

8.2.3 Best Practical Measurement-State-of-the-Art




47/72

SMSM


Chapter 9 Chillers 9.1 Overview




48/72

SMSM



g yHydronic Measurements


49/72

SMSM


PART 4

POWER SYSTEMSMEASUREMENTS

49


50/72

SMSM


I NTRODUCTI ON TO CRI TI CAL POWER DI STRI BUTI ON (Chapt er 11)

UPSTREAM CRI TI CAL POWER DI STRI BUTI ON (Chapt er 12)

UNI NTERRUPTI BLE POWER SUPPLY (Chapter 13)

COMPUTER ROOM TRANSFORMER & POWER DI STRI BUTI ON UNI T(Chapter 14)

Steve McCluer (Schneider/APC), presenter

50


51/72

SMSM


Chapter 11 -

Critical Power Distribution

Critical Power vs. Essential Power Focus is on the Critical Power Path

Can be several meandering paths

Supports those loads with zero tolerance for unplanneddowntime

Mostly (but not entirely) in the IT Equipment Room

Essential Power Path can tolerate a brief disruption

Highly precise meters are typically required for power qualityand forensic analysis,

overkill for routine efficiency studies.

Part 4 describes a data center rated somewhere between200 kW and 5000 kW

G t


52/72

SMSM


Generator Utility

Switchgear & Transformers

ATS

UPS

PDU

RPDU

IT Loads

Mechanical

Switchboard

Mechanical

Loads

Distribution

Switchgear

Distribution

Switchgear

Non-Critical

Loads

Non-Data

Center Loads

Critical Path Essential Path Other Loads Path

Distribution Switchgear

52

h


53/72

SMSM


Ch 12

Upstream Critical Power Distribution Service Entrance Metering

Typically highly precise revenue grade

Typically includes kW & kVAR May be difficult to separate IT loads

Automatic Transfer Switch (ATS)

Typically less precise May capture variances between sources

Primary Electrical Distribution Switchgear

May or may not be available for each circuit Ideal place to capture power into IT equipment room


54/72

SMSM


Ch t 13


55/72

SMSM


Chapter 13

Uninterruptible Power Supply (UPS) Centralized UPS

Measured outside of IT Equipment Room

Output of UPS may be ideal place tocapture IT room critical power consumption

Distributed UPS Measured close to the IT loads in the IT Room

May be multiple UPSs

May be highest power consumptionin the critical power path

Possible to capture power going tozones or rows

Metering can be at power module&/or at the system level

UPS room

Distributed UPS

UPS M t i


56/72

SMSM


UPS Metering

56

SMSMCHAPTER 14


57/72

SMSM


CHAPTER 14

COMPUTER ROOM TRANSFORMER &POWER DISTRIBUTION UNIT (PDU)

Transformers can appear in many placesin the power path

Efficiency lost at every conversion

Typically no metering

i.e., measurements taken by hand-held instruments

PDUs

With transformer, typically meter output

May be best place to aggregate actual ITequipment energy consumption

Without transformer, metering may be optional

57

SMSM


58/72

SMSM


SMSMPDU


59/72

SMSM


PDU

Figure 14.3 Power Flow Through a Power Distribution Unit

59

SMSMRack mounted PDU (RPDU)


60/72

SMSM


Rack-mounted PDU (RPDU)

Measurements taken inside the equipment rack orcabinet

Generally impractical for aggregating equipment powerconsumption

Good for equipment measurement

May be multiple units in one rackpowering the same equipment

Typically have lower accuracy thanupstream devices

Many levels of metering options

60

SMSMAppendix D UPS


61/72

SMS


Appendix D - UPS

Overview of UPS types

Redundancy and availability

Challenges for power measurement on multiple paths

Rules of thumb for minimum practical level ofUPS instrumentation

Example for quick estimation of PUE / DCIE inside thecomputer room

Sample case study: PUE and DCIE

determination for the critical power path withinthe data center (Partial PUE)

61

SMSM


62/72


PART 5

I T SYSTEMS MEASUREMENTS

SMSM


63/72


COMPUTER AND STORAGE SYSTEMS(Chapter 15)

NETWORKI NG SYSTEMS (Chapter 16)

Mike Patterson, Ph.D., P.E. (Intel), presenter

SMSMMotivation


64/72


Motivation

IT equipment poweris

the denominator in thePUE Calculation

Overview

Workload Characteristics Issues

Measurements Minimum Measurement

Best Practical State of the Art

64

SMSMPower thru the IT gear (Server Stor & Comms)


65/72


RoomRoom

Power thru the IT gear (Server, Stor, & Comms)

Measurement location

(a) can give aggregatepower

(b) best location if actualdata is avaiable

PDUPDU ServerServer

RackRackrack power striprack power strip

or PDUor PDU

PSUPSUCPU, etc..CPU, etc..aa

bb

65

SMSMProcess for Measurement


66/72


Process for Measurement

IT Equipment Inventory

Age & self reporting capability

Location; remote storage, etc

Data Center Workloads

Power draw can have unexpected patterns; power vs.

energy considerations Enterprise, HPC, Financial, Internet, e-mail, etc

Time of day, weekly variations, batch processes

Server to server variation due to workload

66

SMSMIssues and challenges


67/72


Issues and challenges

Different organizations, different systems

Different protocols

I T Facilit y IPMI -BACnet

DCMI -Modbus

SMASH -other

Where is it brought together?

Management console, spreadsheet, or back of an envelope?

Redundant Power Supplies

Must measure all feeds and attribute accordingly

Power over Ethernet (POE)

67

SMSM


68/72

Copyright 2010, The Green Grid The Green Grid Technical Forum68 Chapter 16 Network; essentially the same, see the book for details

SMSMMinimum Practical Measurements


69/72


Equipment nameplate

Better than doing nothing, but just barely

See Green Grid White Paper #23, Proper Sizing of IT Power andCooling Loads

PDU Output

Can give actual rack-level power (sometimes energy)

Not always tracked on-line Single server/network/storage measured value

Actual portable power meter

Workload, timeframe, scalability questions

69

SMSMBest Practical Measurement


70/72


Rack or row level PDU output

Rack power aggregate

No visibility to individual servers or workloads Easily tracked/trended in BMS system

Majority of PDUs have this capability

Ignores line losses This is a reasonable assumption

70

SMSMState-of-the Art Measurement


71/72


On-line server power consumption

New Energy Star for Servers requires this!

Challenge is integrating server manageability protocols with BMSsystems

IPMI, DCMI, SMASH, Vendor-specific

Extensive server data available

CPU utilization

Front panel temperature sensors

On-line network and storage power consumption

Somewhat behind Server; Energy Star on the horizon

Capabilities vary on self reporting Wireless remote sensors

Wide range of data capture

71

SMSM


72/72


Thank you for attending

The Green Grid Technical Forum 2010

For more information, visitwww.thegreengrid.org

real time energy consumption measurements

Documents