socomec white paper for planning a data centre electrical infrastructure

8/12/2019 SOCOMEC White Paper for Planning a DATA CENTRE Electrical Infrastructure

1/25

PLANNING AND DESIGN FOR A DATA CENTRE

ELECTRICAL POWER INFRASTRUCTURE:

DISTRIBUTION, UPS, SAFETY AND SAVINGS

WhitePaper022011

ANGELO BAGGINI,

Lecturer at the Faculty of Engineering, UNIVERSITY OF BERGAMO

Matteo GranzieroTechnical communication specialist, SOCOMEC UPS


2/25

2Planning and design for a data centre electrical powerinfrastructure www.socomec.com

CONTENTS

1. INTRODUCTION .................................................................................................................................... 32. ELECTRICAL POWER AVAILABILITY .................................................................................................. 42.1. Downtime cost analysis ....................................................................................................................... 42.2. Calculating power supply availability ................................................................................................... 52.3. Redundancy ........................................................................................................................................ 63. CLASSIFICATION OF DATA CENTRES ............................................................................................... 84. CHOOSING THE DISTRIBUTION SCHEME OF THE ELECTRICAL POWER SYSTEM ..................... 94.1. Fundamental schemes ........................................................................................................................ 9

4.1.1. Single radial scheme ..................................................................................................................... 94.1.2. Dual radial scheme ..................................................................................................................... 104.1.3. Comparison between fundamental schemes .............................................................................. 10

4.1.4. Variants of the fundamental schemes ......................................................................................... 115. SIZING A STATIC UPS SYSTEM ........................................................................................................ 155.1. Power ................................................................................................................................................ 155.2. Autonomy .......................................................................................................................................... 166. PROTECTION AGAINST INDIRECT CONTACT ................................................................................. 166.1. Power supply from the mains ............................................................................................................ 166.2. Standalone operation ........................................................................................................................ 177. EMERGENCY LIGHTING POWER SUPPLY ....................................................................................... 187.1. Back-up lighting ................................................................................................................................. 187.2. Safety lighting .................................................................................................................................... 198. SAFETY AND EMC DISTURBANCES ................................................................................................. 198.1. Connection to the earthing system: functional aspects ..................................................................... 19

8.2. Connection to the earthing system: safety aspects ........................................................................... 208.2.1. High reliability protective conductors .......................................................................................... 208.2.2. Monitoring the continuity of the protective conductor .................................................................. 218.2.3. Use of transformers .................................................................................................................... 21

8.3. Effects of the Uninterruptible Power Supply ...................................................................................... 219. ENERGY EFFICIENCY AND RELIABILITY ......................................................................................... 2210. CONCLUSIONS ................................................................................................................................. 23


3/25


1. INTRODUCTION

The data centre has become a critical component in all types of organisation. It must be suitably planned anddesigned so as to allow sufficient guarantees of quality, efficiency and service continuity, regardless of thedimensions and the sector in which it operates.

Electrical power distribution systems are fundamental infrastructure for most production processes, and ITinfrastructures contribute to determining their performance, for example in terms of safety, reliability and ease ofmaintenance. If system safety is a compulsory specification due to legal requirements, then reliability, availabilityand ease of maintenance are characteristics that nevertheless impact directly on the end user, but for which eachbusiness must establish its own policies.

Finally, when designing a data centre, it has to be considered that the IT industry, which owes its existence toelectrical power, is heading toward a crisis period arising from huge costs, and will probably be one of the firstsectors forced to adapt to new scenarios of low energy consumption. Indeed the issue of energy consumption in

data centres will probably be one of the key issues in the management of IT infrastructures over the next few years.It is estimated that data centres, web factories and similar structures will dissipate almost 5% of power generatedglobally into the environment.

Data centre planning has become a specialist subject in the design industry. The equipment housed in a datacentre has ad hoc specifications, which are not only electrical but which also regard thermal and dimensionalparameters.

A typical data centre basically consists of the following components:

Network server

Calculation and network infrastructure (copper/fibre cabling, equipment)

Network operations centre (NOC), including communication and monitoring

Electricity distribution and generation systems (UPS, generators)

Electrical power panels with overcurrent and overvoltage protection

Lighting systems

Earthing systems

Environmental control, ventilation and air conditioning systems

Fire extinguishing systems

Access control

Equipment cabinets and enclosures

Ducting, floating floors, suspended ceilings TLC circuits and equipment (public operator)

Obviously only some of these components are relevant to the design of the electrical power infrastructure.

In addition to considering typical personal safety aspects, the careful design of a data centre power distributionsystem is particularly important in relation to the high costs typically associated with downtime caused by powerdips and interruptions.

In addition to the constraint of low power consumption mentioned above, which is dealt with in a specific whitepaper in this series and which will not therefore be discussed further in this document, probably the main aspectsthat must be take into consideration in the electrical design of a data centre are power availability and maintainingpower quality. In this context the choice of distribution scheme and static UPS systems plays a central role.


4/25


The following paragraphs deal with the basic concepts of availability, and discuss the main choices that the designengineer and customer are faced with when planning and designing the electrical distribution architecture of a datacentre.

2. ELECTRICAL POWER AVAILABILITY

The main reliability parameter involved in the design of a data centre power distribution system is the availabilityof the power supply for a given user (for example a rack).

In order to be able to make the right design and investment choices when planning the electrical powerdistribution system of a data centre, it is necessary to consider the cost of interruptions in the service provided,taking into account that in the case in question even power dips result in long service interruptions.

2.1. Downtime cost analysis

Table 1shows system downtimes expressed in terms of 1-6 nines.

The comparison that must be made is between the cost of system downtime and the marginal cost of thecorresponding solution.

Table 1 - Data centre downtime as a function of power supply system availability

Availabi li ty Downt ime

90% (1-nine) 36.5 days/year

99% (2-nines) 3.65 days/year

99.9% (3-nines) 8.76 h/year

99.99% (4-nines) 52 min/year

99.999% (5-nines) 5 min/year

99.9999% (6-nines) 31 s/year

Each case must be evaluated on an individual basis, although the following general considerations can be made.

Consider as a starting point the data centre of a single-product business, in terms of lost contribution marginexpressed as a fraction of the total contribution margin. Based on these assumptions we can define:

Loss = TotCM = (NOM + FC)

where:

is the fraction of the total contribution margin, equal to the lost contribution margin. This value can be calculatedas annual system downtime caused by the unavailability of the power supply and annual production time;

TotCM is the total contribution margin

NOMis the net operating margin

FCis the fixed costs


5/25


6/25


protection device1

Dry transformer2 1.3793 10-6 725 000 40-2000

Oil immersed

transformer20.9090 10-6 1 100 000 40-2000

Electrical drive3 4.0000 10-6 250 000 10-1500

Static UPS3 41.6909 10-6 24 000 8-72

Diesel generator3 1.0000 10-6 1 000 000 12-1200

2.3. Redundancy

In order to increase the availability of the power supply it is possible to use redundant components, e.g.components that are able to supply the requested function independently or autonomously. This is an essentialrequirement when the service that the system, or part of it should provide does not allow for downtime orinterruptions, as is typically the case with data centres.

The concept of redundancy is linked to a statistical concept, namely the probability of unavailability, and impliesthe existence, in a system, of one or more components that are capable of performing the same function4.

All engineering choices should aim to achieve the ideal balance between technical and financial considerations;this implies that redundancy, in principle, must not be applied indiscriminately, on the contrary it must be carefullyevaluated.

It is not necessary to provide redundancy of failsafe components, e.g. components with markedly superioravailability compared with other components in the same system.

It is important to underline that the failsafe attribute should not be understood in an absolute sense, but rather in arelative sense to denote a system component with significantly better reliability than other components.

Only non-failsafe components must, if necessary, be made redundant or even made failsafe themselves. Thefailsafe attribute is linked, in the broadest sense of the term, to the conditions of use and therefore has a relativevalue of comparison.

In this type of analysis, evaluation of the end users is no less important; it makes no sense to install a failsafepower supply panel when loads are intrinsically susceptible to frequent disruptions. Generally speaking, in a well-coordinated and balanced design the concept of redundancy must also extend downstream of the electricalsystem.

In the concept of redundancy it is essential that the redundant components are independent of each other; thelevel of independence cannot be defined in general, but must refer to a given event.

1William H. Middendorf, What every engineer should know about Reliability and Risk Analysis, M.Modarres Center of ReliabilityEngineering University of Maryland2A. Baggini, L. Bianco, A. Bossi: Macchine statiche e rotanti: innovazioni significative per aumentarne la qualit, "La qualitdell'energia elettrica. Servizio utile, indispensabile, insostituibile". Department of Electrical Engineering of the University of Pavia AEI,

CNR, CIRED, 03-06, Jul-953R. Calloni, Affidabilit dei gruppi di continuit statici4It is important to underline that the alternative functions can be made operational with a normal manual or automatic switchgear controldevice, but can also be implemented by providing spare or backup components that can be, for example, used as a replacement for

corresponding faulty elements. From an operational point of view, it is customary to distinguish between:

active redundancy, when redundant elements should function simultaneously

standby redundancy, when only one part of the redundant elements is used while the remaining part remains unused until needed.


7/25


The independence analysis must not be limited solely to power circuits but must also be extended to theprotection and control systems.

The independence of components that constitute a redundant system can never be absolute and total, unless the

components are completely separate and autonomous, including the related control systems. In all other cases,common points exist, both for power circuits and for protection and control systems. In other words, an electricalfault, maintenance operation or circuit modification must not compromise the normal operation of another circuit.

As already highlighted, independence can be guaranteed by physical separation from other circuits (partitionsmade from fire-resistant materials), enclosures, circuits with different conduits and the absence of protectiondevices common to other circuits, at least up to the first panel and suitable selective protection devices upstream.

Physical separation can be interpreted in different ways, in particular it may be considered as referring toindependent circuits consisting of cables:

laid in separate pipes or ducts;

laid in the same duct, but separated by partitions;

separate multipolar or unipolar cables with sheath, even if laid in the same pipe or duct.

With regard to the effects of a fire, the use of fire-resistant cables for construction, even if laid in the sameraceway, in itself guarantees a level of independence usually deemed sufficient for safety circuits, guaranteeingoperation even when other circuits laid in the same raceway catch fire.

In some cases however, for example with mechanical actions in a backup system serving an important datacentre, the situation described may not ensure sufficient independence and geographically separate conduitsmay be required.


8/25


3.CLASSIFICATION OF DATA CENTRESThe article Tier Classifications Define Site Infrastructure Performance by the Uptime Institute introduces a 'tier'method of classification for data centres. The classification envisages four tiers as follows:

a) Tier I Data Centre: basic data centre

A data centre classified as Tier I has its own distribution and cooling system, it may have a raised floor and mustbe connected to a UPS system and/or a diesel engine for safety and backup power. Annual service downtime forcarrying out repair work and preventive maintenance is acceptable. Urgent situations may require more frequentsystem shutdowns and any errors or faults in subsystems would cause disruption to the data centre.

b) Tier II Data Centre: redundant components

Systems present in a Tier II data centre have redundant components and are slightly less susceptible to plannedor unplanned disruptions. They may have a raised floor but their UPS, diesel generator and power supplyscheme must be N+1. Maintenance of critical power supply conduits or for other parts of the infrastructure requirethe system to be shut down.

c) Tier III Data Centre: simultaneous maintenance and operation

In data centres classified as Tier III, any planned activity is permitted in the infrastructure without disrupting thehardware operation in any way. Planned activities include preventive and scheduled maintenance, repair,replacement, addition or removal of system components, system or component testing, etc. For largeinfrastructures the cooling system must have two separate, independent circuits, each of which can meet thecapacity and distribution requirements of the entire system. Unplanned activities such as errors in operation orfailures of system components, especially during maintenance work or after a first fault, may cause further data

centre malfunctions or breakdown. Many Tier III data centres are designed to be easily converted to Tier IV sites,obviously only when the client's business case justifies the cost of the additional design.

d) Tier IV Data Centre: fault-tolerant

Tier IV data centres permit planned activity to be performed without disruption of any kind. Fault-tolerantfunctionality also provides the ability to sustain the system with no load impact even in the presence of anunplanned failure or event. This requires parallel power distribution paths (in System + System configuration) withtwo UPS in which each system has N+1 redundancy. Infrastructures with Tier IV data centres are the mostcompatible with redundancy concepts for achieving reliability, availability and easy maintenance.


9/25


4. CHOOSING THE DISTRIBUTION SCHEME OF THE ELECTRICAL

POWER SYSTEMChoosing the distribution scheme is one of the fundamental elements in electrical system design, regardless ofsystem complexity, on which however solution analysis and development will depend. In order to make thischoice it is necessary to know and fully define load requirements and power source specifications. It is notpossible to define specific criteria for establishing the ideal structure of an electrical power distribution network,since they are very widely based on power ratings involved and local conditions, for example.

Nevertheless it is possible to draw up a number of general guidelines for selecting the most suitable systemlayout.

As a general rule, the conditions that influence the definition of the electrical power distribution scheme for agiven data centre can be divided into two categories: those dependent on the characteristics of the system itself(overall power requirement, continuity needs, maintenance needs, etc.), which can be defined as internal

conditions, and those dependent on the characteristics of the power supply linked to the utility company (short-circuit power, voltage level, reliability, etc.), which can be defined as external conditions.

4.1. Fundamental schemes

The possible configurations of an electrical power distribution system serving a data centre can be traced back totwo fundamental schemes:

single radial;

dual radial;

in addition to the mesh scheme, typically adopted by utility companies, and the loop scheme.

4.1.1. Single radial schemeIn a single radial scheme the power supply is branched from a system of primary feeders, from which power isthen distributed radially to the individual users or to secondary feeder systems.

Figure 1shows an example of a single radial scheme.

Figure 1 - Example of a single radial scheme


10/25


4.1.2. Dual radial scheme

The dual radial scheme essentially consists of the combination of two single radial systems, which extend fromupstream to downstream in association with each other.

The duplication of system components can be extended as far as the individual user or, more frequently, as faras one or more distribution nodes. Redundancy must be provided not only with regard to power components butalso with regard to the command and control system, if present.

Figure 2shows an example of a dual radial scheme.

Figure 2 - Example of a dual radial scheme

4.1.3. Comparison between fundamental schemes

Figure 3 highlights, in summary form, the main reliability parameter values of the basic schemes with reference tothe power supply of the second load from the left. The results calculated on the basis of statistical data are givenin Table 3.

Figure 3 - Example of distribution

Table 3 - Quantitative comparative summary of the availabil ity of two fundamental power distribution schemes


11/25


Component [h-1/year] MTTR [h/year] MTBF [h/year] Availabili ty [%]

MV public network 1.142 10-4 0.2 8.760 10+3 99.997717

MV-LV transf. 9.094 10-7 52 1.100 10+3 99.995271

Circuit 1.575 10-6 8 6.348 10+3 99.998740

Single radial 1.166 10-4 - 8.573 10+3 99.990000

Dual radial 1.588 10-6 - 6.296 10+3 99.999000

In summary it can be stated that:

- in both cases the general performance of the power supply system is limited mainly by the public distributionnetwork which, as is common practice, renders the use of a backup power source indispensable;

- in the example case of a double radial scheme, in practice the second MV network (independent of the first forthe purpose of this hypothesis) constitutes a backup source that only marginally improves availability (99.999%compared with 99.99%) although not sufficiently for most centres, which require much higher availability;

- the dual radial scheme nevertheless guarantees vastly superior availability compared with the single radialscheme;

- in general it can be stated that redundant schemes guarantee higher availability but also higher costs, in termsof both original equipment and maintenance.

4.1.4. Variants of the fundamental schemes

Faced with the complex balance between the costs and advantages of the fundamental schemes, it is commonpractice to adopt numerous variants to the basic schemes, some of which are well designed while others are justexercises of imagination.

The following figures illustrate and compare some of the most common variants of fundamental schemes in termsof:

availability;

repairability;

expandability;

fault tolerance;

cost.


12/25


Figure 4- Dual Bus 2(N+1) Architecture

Figure 5 - Double Bypass Architecture (note the bypasses must be independent).


13/25


Figure 6 - 2N Architecture with STS

Figure 7 - 3N Architecture with STS


14/25


Figure 8 - 2N+1 Architecture

Figure 9 - 2N+1 Architecture


15/25


5. SIZING A STATIC UPS SYSTEM

Choosing the power of a static UPS system serving a data centre is a decision that involves various elements

and which cannot be made before, or independently from, the choice of distribution scheme.The main elements to consider can be summarised as follows:

two from the following parameters of the loads to be supplied: Active Power (PRL), Apparent Power (SRL) orPower Factor (P.F.);

type of load power supply (voltage, frequency, number of phases);

load coincidence factor;

required back-up time;

type of mains power supply (voltage, frequency, number of phases).

5.1. Power

The fundamental characteristic of IT equipment loads, and more generally of all loads equipped with switchingpower supplies, is the current waveform and phase. Since these power supplies only absorb current close tomaximum voltage, the typical waveform, far from being sinusoidal (Figure 10), has a rather reduced base line anda vertex in correspondence with the voltage peak. Given the same effective value, this waveform has thepeculiarity of having a much higher crest factor than that of a sinusoidal wave. The static UPS must be able tosupply this peak current value, which is normally indicated in the product's technical specifications as the 'crestfactor'.

In accordance with standard EN 62040-3, the system must not be derated for standardised non-linear loads witha crest factor lower than three (3:1)5.

With regard to the current phase it should be noted that the power factor of the loads under consideration isleading, and therefore specific precautions must be taken when sizing the UPS. Current computer loads have

input power factors up to 0.9 leading. Note however that there are UPS systems on the market that are able tosupply power without derating even this type of load.

Figure 10 - Typical current waveforms of computer loads compared with a sinusoidal wave (dotted line).

If the static system is also required to deliver a large inrush current, as in the case of lighting fixtures withfluorescent bulbs, this must duly be taken into account.

The parameters are:

5Sometimes the nominal power of a static UPS system is confused with terms such as 'switching power', 'computer power'and 'actual power'. These expressions were probably coined in an attempt to define a parameter capable of modelling system

power even in conditions where the current and voltage waveforms are distorted, nevertheless it should be borne in mind thatthese parameters do not have an official regulatory definition and therefore cannot have any correlation with the apparentpower and nominal active power of the static system.

Consequently, they cannot be used to size a static UPS system.


16/25


UPS(maximum current value from the UPS);

tUPS(time for which UPS is sustainable);

load(overload current required by the load);tload(theoretical overload time required by the load).

the apparent power required for sizing the UPS will be:

UPS

loadRLUPS

I

ISS with tloadtUPS

Typical values of UPSand tUPScan range from typical current values of 150% of nominal current for one minute upto 200% per 100ms without mains power or for UPS systems not equipped with bypass.

The typical cold inrush current of an appliance of the type under consideration is equal to 6 - 8 times the nominalcurrent, and in practice is only limited by the impedance of the UPS and the conductors comprising the part of thedistribution network affected by the event.

5.2. Autonomy

Autonomy is essentially linked to the time that the data centre is able to guarantee continued operation and ifnecessary perform a controlled shutdown (typically 30-60 min).If service must be guaranteed for a long time (roughly 60-90 min), as is often the case, a rotary generator shouldbe provided to supply power to the static system, implementing procedures to ensure fuel replenishment ifnecessary.

6. PROTECTION AGAINST INDIRECT CONTACT

Protection against indirect contact in the presence of static UPS systems, as applied typically to data centres, isrelatively complex since the downstream circuits can be powered either by battery in standalone operation,independently from the network, or from the network through the inverter or directly.

At the design stage it is important to give due thought to this choice so as not to risk compromising personalsafety on the one hand, and nullifying the availability of the power supply on the other.

6.1. Power supply from the mains

The options typically available are the adoption of a single residual-current device (RCD) upstream of the staticsystem or a RCD for each user or group of users downstream.

Even if tripping of the RCD occurs whether the power is fed from the mains or from the inverter, due to differentdynamics.

Consider the TT system as an example. In the first case (Figure 11), since the neutral conductor of the mainspower supply is not interrupted by the static system, an earth fault would cause the upstream RCD to trip, in theabsence of the downstream device or if not an S-type device.An earth fault in the group upstream of the transformer, if present, also causes the RCD to trip, Figure 12.


17/25


Figure 11 - Power supply direct from mains: a fault downstream of the static system results in the tripping of residual-current

device Q1 or Q2, if present and Q1 S-type device.

Figure 12 - Fault upstream of the transformer: residual-current device Q1 trips.

In other words the adoption of a single RCD upstream of the static system protects against indirect contact in allof the considered cases and also guarantees, in the event that a device trips, continuous power to all loadsincluding the load affected by the fault, but for a time restricted by the autonomy of the system.

The design choice to include a RCD for each load or group of loads powered by the static system in the event oftripping removes the faulty load from the service, but guarantees a continuous supply of power to all the otherloads for an indefinite time.

The behaviour of the protection devices in the case of power supply from a TN system can easily be inferred fromthe cases examined for the TT system.

6.2. Standalone operation

In standalone operation the electrical system downstream of the transformer is isolated from earth:


18/25


- on the first earth fault the condition RT 50/Idis of course satisfied since current Id is leading and, given thelimited extension of the circuit, it is in the order of several mA6;

- a second earth fault must be eliminated, in principle, within the times stipulated in standard IEC 60364 (in Italy

CEI 64-8), but typically overcurrent protection devices do not trip since the short-circuit current is in the order of1.25 - 2 IR, but the protection device in the static system trips within a few seconds or tenths of a second

7, leadingto the disconnection of the inverter.

The need to indicate the first earth fault in IT systems could also be considered as extended to circuits poweredby a static UPS system in standalone operation, nevertheless further considerations must be made:

typically the circuits powered by the static UPS are not very extensive:

the length of time that the UPS can function in standalone operation is generally limited to between 10 and 30minutes,

therefore the probability of a fault occurring in these conditions is small and the indication of the first earth faultcan consequently be omitted.

However it can be considered that in the case of UPS with bypass line, the indication of the first earth fault caneffectively be implemented by equipping the RCD device with a tripped relay contact.

7. EMERGENCY LIGHTING POWER SUPPLY

Lighting is defined as emergency lighting when it is intended for operation when standard lighting fails.Emergency lighting must be powered by an independent energy source (typically a central power supply system)and is classified by the UNI EN 1838 standard into:

1. Back-up lighting

2. Safety lightingWhile safety lighting is governed by special regulatory, technical and legislative requirements, both in terms ofconstruction and compliance, for backup lighting it is necessary to refer to the general installation rule, and thedecision to adopt it or not is typically a financial one.

In both cases the problem can be viewed in terms of guaranteeing given reliability and availability.

7.1. Back-up lighting

In the case of back-up lighting (or more generally back-up services), identifying the level of reliability to adopt andthe subsequent implementation of lighting in terms of plant engineering solutions is inevitably left entirely to thediscretion of the system designer and customer. If a static UPS system is used to power back-up lighting, it isnecessary to determine the autonomy needed to continue doing the same activity and the same tasks that were

being performed while the standard lighting was functioning, without significant changes. The level of illuminationprovided by backup lighting must generally be at least equal to that provided by standard lighting, as if this werenot the case it would be impossible to continue the previous activity. One typical case where it is possible to haveback-up lighting which is dimmer than standard lighting, is when back-up lighting is only used to finish off work inprogress, and not to continue indefinitely (for example non-automated operations before data centre equipment is

6Service continuity not requiring interruption of the service on the first earth fault is preferable.

7However in such a short space of time anyone in contact with both faulty appliances is not in danger. In fact simultaneous

contact with the two faulty appliances exposes the body to a negligible voltage, due to the modest short-circuit current value,which is equal to:

IRV where:

R is the resistance of the protective conductors

I is the short-circuit current


19/25


shut down). It is also necessary to consider the peculiarities of the bulbs normally used in lighting fixtures, inorder to correctly size a UPS used as a back-up power source. For example rephased linear fluorescent bulbs,used widely for reasons of efficiency in lighting systems, are characterised by high inrush current.

The typical cold inrush current of a lighting fixture of the type in question is equal to 6 8 times the nominalcurrent, and in practice is only limited by the UPS and the conductors comprising the part of the distributionnetwork affected by the event.

7.2. Safety lighting

The purpose of safety lighting is to provide an adequate level of safety to people who find themselves withoutstandard lighting, and to therefore prevent accidents or dangerous situations from occurring. It is not lighting thatcan be used to carry out ordinary tasks, but functions only to ensure the safe mobility of people.

For the power supply of safety lighting it is necessary to use a central power supply system (CPSS), e.g. a UPS

that also meets the additional requirements of standard EN 50171, and the related circuits must be independentfrom any other circuits (standard IEC 60364-5 chapter 56).

In a data centre, safety lighting can be provided in the form of self-powered equipment, however sometimes it ismore convenient to use the backup CPPS of the CED to supply power to safety lighting as well.In this case autonomy time must be adapted and it is also essential that:

a central power supply system is used rather than a straightforward UPS;

back-up usage does not compromise safety;

selectivity is guaranteed;

mutual interference of loads is verified in terms of electrical power availability.

8. SAFETY AND EMC DISTURBANCES

In data centres, new requirements and further constraints often arise during implementation of the earthingsystem, already a key element of electrical installations in relation to safety requirements.

Indeed electronic appliances require not only an earth connection8 for safety reasons (protective earthing), butalso an earth connection for functional reasons (operational earthing).

These distinct requirements pose a problem of electromagnetic compatibility. Indeed the simultaneous need foran earth connection for safety and functional reasons, subjects electronic appliances to disturbances they wouldotherwise not be subjected to on the one hand and poses safety problems due to permanent leakage currentsthey produce on the other.

In these cases the earthing system must be implemented in such a way as to reconcile both safety requirementsand functional requirements, in other words without jeopardising the level of safety of the electrical system and atthe same time without compromising appliance functionality.

8.1. Connection to the earthing system: functional aspects

The solution to EMC problems therefore requires a systematic approach, that cannot be limited solely to theimplementation of a correct functional earth connection. Nevertheless, accurate implementation of the earthingsystem may contribute decisively to the reduction of disturbances.

8Please refer to the definition of earth given in Art. 24.1 of Standard CEI 64-8, Part 2: The conductive mass of the earth,

whose electric potential at any point is conventionally taken as equal to zero.


20/25


The CEI 64-8 standard stipulates that for a combined protective and functional earth system, provisions regardingprotection measures must take precedence.

The functional earth must preferably be implemented using different conductors to those used for protection.

These insulated conductors will then connect, directly or through intermediate collectors, to the main earthcollector, to which the protective conductors will also be connected9.

Conductors used exclusively for connection to the functional earth are not subject to provisions relating toprotective conductors. Their characteristics must therefore be chosen on the basis of electromagneticcompatibility requirements of the installation.

Within the same circuit the functional earth connection must be implemented in such a way that multiple earthconnections (intentional or effective) are avoided.

8.2. Connection to the earthing system: safety aspects

The presence of filters in electronic appliances to suppress radiofrequency interference may result in high

10

earthleakage currents. In these cases a continuity fault in a protective conductor represents a dangerous condition.

Indeed until the body of the appliance is connected to earth, dangerous conditions do not exist; if the protectiveconductor is interrupted and someone touches the body of the appliance, both would be exposed to the leakagecurrent with serious risk.

In reality this occurs for any appliance that leaks a current to earth. The increased level of danger in this case islinked to the value of the current; while the leakage current of normal user appliances is limited, that of certainelectronic appliances may be between 3.5 and 10 mA and, in certain cases, even higher11.

Hence the need to stipulate additional provisions for the earth connection of this type of appliance. To this effectthe CEI 64-8 standard emphasises the continuity of the protective conductor as well as its physical characteristics(cross-section, connections etc.) with the aim of ensuring maximum reliability of the conductor, proposing threealternative solutions:

high reliability connection;

monitoring of the protective conductor;

use of transformers (excluding autotransformers).

The CEI 64-8 standard also stipulates additional provisions for protection against indirect contact for TT and ITsystems.

8.2.1. High reliability protective conductors12

The stated purpose of this provision is to implement reliable protective earth connections. This requirement isdeemed to be satisfied by the use of protective conductors with a cross-section of not less than 10 mm 2if madefrom a single cable or two conductors in parallel each having a cross-section of not less than 4 mm2 and

independent terminals.If the protective conductor is made using a core of a multipolar cable then the sum of the cross-sections of all theconductors that constitute the multipolar cable must not be less than 10 mm 2, including the protective conductorwhich must furthermore have a cross-section of not less than 2.5 mm2.

9Standard CEI 64-8, Part 5, Art. 546.1.10According to standard CEI 64-8, high leakage currents are currents that exceed the limit specified and measured in

accordance with Art. 5.2 of European Standard EN 60950 (CEI 74-2), or in Annex G thereof (3.5 mA).11

The current threshold conventionally established for the tetanisation threshold is equal to 10 mA (for times greater than 2s).12Standard CEI 64-8, Part 7, Art. 707.471.3.3.1.


21/25


8.2.2. Monitoring the continuity of the protective conductor

In order to check the continuity of the protective conductor13, section 707 of standard CEI 64-8 stipulates therequirement to provide a device, or multiple devices, that interrupt the power supply to the appliance in the eventof discontinuity in the protective conductor, in accordance with provisions relating to protection against indirectcontact.

8.2.3. Use of transformers

Another measure to limit earth leakage currents consists of using transformers14.

If the appliance is powered through a transformer and the secondary circuit is connected as a TN system, thenthe leakage currents flow through the transformer secondary without involving the protective conductor.

Figure 13 - Example of connection of an appliance with high leakage current to the user system through a transformer

The transformer does not need to have special characteristics (e.g. insulation transformer) even if the use ofautotransformers is not allowed.

8.3. Effects of the Uninterrupt ible Power Supply

Special attention must be paid to the static UPS system, its electromagnetic compatibility requirements andcurrents to earth.

With regard to EMC requirements, standard EN 62040-2 defines categories for the classification of UPS in termsof emission and immunity based on their application. The system designer is responsible for the selection andideal positioning of the static system.

Table 4 - Summary of EMC classification of UPS according to standard EN 62040-2

First environment Second environmentComplex

environment

Definition Residential, commercial, light Residential, commercial, light Defined by the

13Standard CEI 64-8, Part 7, Art. 707.471.3.3.2.

14Standard CEI 64-8, Part 7, Art. 707.471.3.3.3.


22/25


industrial connected directly tothe LV network withouttransformer.

industrial connected directly tothe LV network by means of atransformer.

customer

Neutralsystem

TT TN, IT Defined by thecustomer

UPScategory

C1 C2 C3 C4

Definitionof UPS

category

UPS for use withoutrestrictions in theFIRSTENVIRONMENT

UPS for use withoutrestrictions in theSECONDENVIRONMENT

UPS for use incommercial orindustrialinstallations at aminimum distanceof 30 m from theFIRSTENVIRONMENT

UPS for use inCOMPLEXENVIRONMENT andsubject to agreementsbetween customerand supplier

With regard to UPS earth leakage current, it should be remembered that in the case of stationary equipmentstandard EN 60950-1 stipulates that the protective conductor current must not exceed 5% of input current.

9. ENERGY EFFICIENCY AND RELIABILITY

In the case of a data centre, energy efficiency and low power consumption are certainly important, as much forenvironmental reasons as for economic ones; nevertheless this must not lead to the reliability of the power supplybeing sacrificed. In other words design choices must achieve a balance between these two aspects.

The two situations that can most commonly lead to incorrect design conclusions are:

- operation at reduced load due to high redundancy

- adoption of an off-line system to improve efficiency

The first case refers to the consequences that the adoption of a redundant scheme can have on powerconsumption due to the efficiency trend of static UPS systems.

Indeed the efficiency of a static UPS system has a typical trend, such as that represented in Figure14e.g. for lowloads, efficiency is drastically reduced.

Taking the example of a dual radial system (redundancy at 100%), each static system is at 50% load. With thedata in Figure 14the overall efficiency of the UPS would be reduced from 96%, at full load, to 95% or as low as93%, depending on the technology of the static UPS system.

Figure 14 -Example of the efficiency of two static UPS systems (solid line - System A, dotted line System B).


23/25


One solution to this problem is the adoption of a Static Transfer System (STS) that allows greater flexibility interms of the system architecture.

The second caserefers to the consequences that the adoption of a VFD or VI system can have on the reliabilityof the power supply for efficiency purposes.

Although VFD or VI systems promise higher efficiency, simplicity and compactness, in the case of the supply ofcritical loads such as IT equipment in a data centre, it cannot be overlooked that:

- transfer times are not null, for example the load is subject to voltage drops from the mains or duringpower dips and interruptions;

- the operation of many IT appliances can also be affected by very slight disturbances in the supplyvoltage;

- national distribution networks, with regard to quality, are not exempt from overvoltage, frequencyvariations, flicker and harmonics.

The main solution to this problem is to adopt double conversion VFI UPS systems for the supply of IT appliances.

10. CONCLUSIONS

The design requirements of a data centre must include not only site selection, positioning, space, availablepower, air conditioning, load, access points, environmental quality, risk evaluation and the possibility ofexpansion, but also the requirements of the electrical power supply system.

The main aspects that must be taken into account in the electrical design of a data centre are the availability andmaintenance of the power supply, aspects in which the selection of the distribution scheme and UPS systemsplay a central role.

In order to be able to make the right choices in designing and defining the investment dedicated to the electricalpower distribution system, it is necessary to consider the cost of service downtime.

The availability of statistical reliability data together with recognised methods makes it possible to carry outaccurate quantitative evaluations on which to base the strategic choices that the planning and design of a datacentre involve.


24/25


BIBLIOGRAPHICAL REFERENCES

A. Baggin i, M. Granziero: UPS systems A practical guide to selection, installation and maintenance, EditorialeDelfino, Milan 2009

IEEEDesign of Reliable Industrial and Commercial Power Systems, IEEE Gold book, 493, 2007

A. Baggini, F. Bua: Collegamento alla terra funzionale delle apparecchiature elettroniche, institutionalconferences CEI 2004, Impianti di terra e impianti elettrici nei luoghi a maggior rischio in caso di incendio, Milan 3March 2004.

A. Baggini, F. Bua: Impianti di emergenza. Scelta delle apparecchiature, U&C, May 2008.

Telecommunications Industry Association: Telecommunications Infrastructure Standard for Data Centers,Arlington (USA) 2004.

M. Granziero:Approccio Green alla disponibilit dellenergia elettrica, Elecrotechnical news, October 2009.

______________________________________________________________________________________


25/25

White Paper 02/2011

PLANNING AND DESIGN FOR A DATA CENTRE ELECTRICAL POWER INFRASTRUCTURE:DISTRIBUTION, UPS, SAFETY AND SAVINGS

Authors:

ANGELO BAGGINI, Lecturer at the Faculty of Engineering, University of BergamoMATTEO GRANZIERO, Technical Communication Specialist, SOCOMEC UPS

Media & Marketing Department

SOCOMEC UPS

Via Sila, 1/3

36033 Isola Vicentina (VI) Italy

Media Marketing coordinator: [email protected]

Head Offices

SOCOMEC UPS

11, route de Strasbourg

B.P. 10050

F-67235 Huttenheim Cedex France

socomec white paper for planning a data centre electrical infrastructure

Documents