how it decisions impact facilities: the importance of mutual understanding

How IT Decisions Impact Facilities: The Importance of Mutual Understanding

Lars Strong, P.E., Upsite TechnologiesIan Seaton

Airflow Management Awareness Month

June 15, 2016

Lars is a thought leader and recognized expert on data center optimization. He currently serves as the Senior Engineer and Company Science Officer of Upsite Technologies. Lars is a certified U.S. Department of Energy Data Center Energy Practitioner (DCEP) HVAC Specialist.

05/02/2023 © Upsite Technologies, Inc. 2

Speaker Bios

Ian has over 30 years of electro-mechanical product development experience. He serves on the BICSI 002 Data Center Standard subcommittee as a working group leader and editor, and he contributes to the Green Grid Airflow Management Energy Calculator Committee. He recently retired as a corresponding member of ASHRAE TC9.9, and he is a patent holder for airflow performance algorithms.

Lars Strong Ian Seaton

Specifying Server Type: Cooling Delta T (ΔT) Specifying Server Class: Class (A1, A2, A3, or a4) Specifying Equipment That Breaths Front to Back Specifying Storage Type: Solid State or Tape Specifying Cooling Unit Set Point Specifying Cages/Layout That is Compatible with

Containment Airflow Management Best Practices


Agenda

The Importance of Mutual Understanding

Decisions and actions typically under the jurisdiction of the IT side of data center management can have a profound impact on the mechanical systems, operating costs, and capacity of the data center.

By understanding these impacts, IT and facilities management are able to develop a cooperative approach to managing the data center.

How IT Decisions Impact Facilities


SPECIFYING SERVER TYPE: COOLING DELTA T (ΔT)

© Upsite Technologies, Inc.05/02/2023 5

ΔT Through IT Equipment

Cooling unit operating expense

Cooling unit acquisition capital expense

Decisions About Server Types Affect:


Heat Transfer Equation

CFM =

CFM = cubic feet per minute of airflow through the server3.16 = factor for density of air at sea level in relation to F⁰

ΔT = temperature rise of air through the server in F⁰

3.16 x W Delta T

IT Equipment Required Flow Rate

IT Equipment Delta T ( F⁰ )

15 20 25 30 35 40

Required flow rate

(CFM/kW) 211

158

126

105

90

79



• “Pizza box” servers• at 20 ⁰F ΔT consume 158 CFM / kW

• “Blade” servers • at 35 ⁰F ΔT consume 90 CFM / kW



• 500kW of “Pizza box” servers• 1,000 servers• Would require 79,000 CFM of chilled air

• 500kW of “Blade” servers • 1,600 servers (100 chassis)• Would require 45,140 CFM of chilled air



• “Blade” servers (35 ⁰F ΔT) • Servers require 43% less airflow rate• Cooling unit fans require 81% less energy

• At 50% fan speed• 50% airflow rate• Only 12.5% of fan energy (87.5% reduction)



SPECIFYING SERVER CLASS: CLASS (A1, A2, A3, OR A4)


• Access to free cooling hours

• Chiller operating efficiency

• Capital investment for the mechanical plant

• PUE anomalies

• Warranty and associated reliability costs

• Building footprint and real estate investment

ASHRAE Server Class

Decisions About ASHRAE Server Class Affect:


ASHRAE Environmental Guidelines


ASHRAE Server Class

How to know the effect of temperature on ICT equipment reliability/life?How to know how long it is OK to operate in allowable temperature range?

The “X” Factor

1. Baseline = 24/7 operation @ 68˚F server inlet temperature

2. OEM historical data on user failure reports

3. Performance variations above, at and below baseline 68˚

4. Premise is that the data center temperature follows mother nature

ASHRAE Server Class

Time at temperature weighted failure rate calculations for IT equipment in ChicagoInlet Temperature The “X” factor% of Hours

59⁰F ≤ T ≤ 68⁰F .865

68⁰F ≤ T ≤ 77⁰F 1.13

77⁰F ≤ T ≤ 86⁰F 1.335

86⁰F ≤ T ≤ 95⁰F 1.482

67.6% 17.2% 10.6% 4.6%

Net X-Factor = 0.99


ASHRAE Server Class

Time-weighted X-factor estimates for air-side economizer for selected U.S. citiesversus baseline 68˚ for 8,760 hours Improved Failure Rates

San Francisco 90%Seattle 90%Boston 96%Denver97%Los Angeles 98%Chicago 99%

Worse Failure Rates

Miami 128%Phoenix 120%Houston 113%Dallas 110%Atlanta 104%Washington DC101%


SPECIFYING EQUIPMENT THAT BREATHES FRONT TO BACK


• Integrity of hot aisle separation

• Potential floor density thresholds

• Fan and temperature set points

• Integration of switches and servers

Specifying Equipment

Decisions on ICT Equipment Breathing Patterns Affect:


• As far as the mechanical health of the data center, all equipment should be: • rack-mountable• Breathe from front to rear

• Recognized best practice of sources as diverse as:• European Code of Conduct for Data Centers• BICSI-002• ASHRAE TC9.9



• CFD model – switch flow side to side• 3,500 sq. ft., 514 kW• 3kW rack with 73 F to 90 ⁰

F intake air temperatures⁰• 6kW switch rack ingesting

56 F to 85 F air⁰ ⁰• Allowable by manufacturer

but exceeded internal SLA• 72 F return set point, ⁰

resulting in 54 F supply⁰• 82,500 CFM cooling

supply for 67,900 CFM IT demand



• CFD model – switch flow front to back• 3kW rack with 75 F to 76 F ⁰ ⁰

intake air temperatures• 6kW switch rack ingesting 75

F to 76 F air⁰ ⁰• Supply temperature

increased from 54 F to 75 ⁰F⁰

• 38% chiller plant energy savings

• cooling supply reduced from 82,500 CFM to 72,000 CFM • 33.5% CRAH fan energy

savings



SPECIFYING STORAGE TYPE: SOLID STATE OR TAPE


• Access to free cooling

• Total operating energy budget

• Mechanical plant acquisition and construction costs

• Humidity management investment

Solid State vs Tape Storage

Decisions on Solid State vs Tape Storage Affect:


• ASHRAE TC9.9 rate of temperature change boundaries:• Tape storage - 9°F per hour• Solid state storage - 36°F per hour

• Weigh solid state storage premium against cost avoidance of acquiring, constructing and operating a mechanical plant

Solid State vs Tape Storage


SPECIFYING COOLING UNIT SET POINT


• Chiller efficiency• Cooling unit efficiency and life (hot & cold cycling) • Humidity management operating expenses• Access to free cooling• General operational discipline

Specifying Cooling Unit Set Point

Decisions Specifying Cooling Unit Set Point Affect:


• Comfort cooling ≠ data center cooling

• It is still common to see 72° F cooling unit return set today, and lower set points in the range of 68° F are still not uncommon

• Often driven by IT conservatism or hot spot mitigation



• Cooling units often cool the air 18° F, resulting in supply temperatures as low as 50° F, potentially causing dew point problems.

• At 50° F, 100% RH or condensation is reached with 55 grains of moisture per pound of dry air, a condition which would be met at any of the following data center control settings:• 60% RH @ 65° F• 50% RH @ 70° F• 45% RH @ 75° F (Quite common)• 36% RH @ 80° F



SPECIFYING CAGES/LAYOUT THAT IS COMPATIBLE WITH CONTAINMENT


Cages and Containment

• EVERYTHING!• Cages and containment don’t have to be mutually

exclusive• If the cage is not compatible with containment, there

will be a need for extra volume airflow and lower cooling unit set points, resulting in higher operating costs

Specifying Cages Compatible with Containment Affects:


• Chimney cabinets provide full containment can be deployed independent of layout

• Out of a total 79 cabinets, 12 problem cabinets fitted with chimney exhaust • 38% chiller plant energy savings• 33.5% CRAH fan energy savings

Cages and Containment


AIRFLOW MANAGEMENT BEST PRACTICES


• EVERYTHING!• Reduce required airflow rate

• Reduced fan energy cost• Increase cooling unit set points / supply air temperature

• Increase cooling capacity• increasing free cooling hours

• Lowering chiller energy costs• Defer capital expenditure

• Cooling units or even new data center

Airflow Management Best Practices

Adhering to Airflow Management Best Practices Affect:


• Responsibility of IT• Commonly cabling impedes conditioned airflow

under the raised floor• Commonly cabling impedes exhaust flow out of IT

equipment


Cable Management


• Seal cable openings• Seal under power distribution units (PDU) and other

equipment• Check perimeter of raised floor plenum• No misplaced perforated


Raised Floor Open Area Management



Raised Floor Open Area Management

Raised floor openings sealed with brush grommets


• Blanking panels• Rail seals• Under racks

• Few sites have done this fully

• Cabinet design plays a large role in AFM

• Cabinet AFM design needs to be considered by IT when purchasing cabinets


Rack Open Area Management


• Gaps between racks• Missing racks• Doors on the ends of aisles• Baffles or full roof over racks


Row Airflow Management


Angled Rack Top Baffles (Cold Aisle)

Adjustable Rack Gap Panel

Vertical Rack Top Baffles (Hot Aisle)

Modular Containment


A holistic approach An iterative process ‘Check in’ at the room level

after making any AFM improvements

Watch video at:upsite.com/resources/airflow-management/

Upsite’s 4 R’s of Airflow Management™


http://www.upsite.com/resources/airflow-management/

http://www.upsite.com/resources/airflow-management/

June 22: The 4 Delta T’s of Airflow ManagementPresented by Lars Strong, P.E.

Register at upsite.com/airflow-management-awareness-month

Upcoming Schedule


http://www.upsite.com/airflow-management-awareness-month/

Thank you!

@UpsiteTech

blog.upsite.com

On LinkedIn

Lars Strong, P.E.Senior Engineer, Upsite [email protected]

Follow Upsite for the latest news and information on data center AFM.

Ian [email protected]

https://twitter.com/upsitetech



http://blog.upsite.com/

http://blog.upsite.com/

http://www.linkedin.com/company/231420?trk=tyah

mailto:[email protected]

mailto:[email protected]

how it decisions impact facilities: the importance of mutual understanding

Technology