thermal analysis of a 15u telecoms cabinet chris hill ... · thermal analysis of a 15u telecoms...

of 49 Thermal Analysis of a 15U Telecoms Cabinet

Thermal Analysis of a 15U Telecoms Cabinet

Chris Hill ([email protected])

This document formed part of my course work for the University of Bolton’s MSc in Electronic Product Development, 2006 – 2009.

The product names FloTHERM, FloPCB and FloMOTION are trademarks of Mentor, A Siemens Business.


Table of contents. Table of contents. ...................................................................................................................... 2 1. Defining the enclosure. .......................................................................................................... 3 2. Manually calculating the router board operating temperatures. ............................................ 3

Calculating the airflow resistance of the system. .................................................................. 4 Defining the fan operating point. ........................................................................................... 6 Calculating temperature rise within the system. ................................................................... 7

3. Using the FloTHERM thermal simulation software................................................................ 7 Determining convergence. .................................................................................................... 8 Gridding the simulations. ....................................................................................................... 8 Solver Option. ........................................................................................................................ 9 General modelling parameters. ............................................................................................. 9 Radiation. ............................................................................................................................ 10

4. Defining the router board. .................................................................................................... 10 Importing the board from FloPCB. ...................................................................................... 10 Rationalising the router board model. ................................................................................. 11 Optimising the router board local gridding. ......................................................................... 12

5. Modelling the enclosure. ...................................................................................................... 13 Empty enclosure specification. ............................................................................................ 13 Simulating airflow in the empty enclosure. .......................................................................... 14

6. Simulating the enclosure plus router boards. ...................................................................... 15 Initial configuration. ............................................................................................................. 15 Initial configuration results. .................................................................................................. 16

7. Modifying the enclosure design. .......................................................................................... 20 a) Varying the size of the grills. ........................................................................................... 20 b) Varying the location of the grills. ..................................................................................... 24 c) Varying the grill construction. .......................................................................................... 26 Altering the fan specification. .............................................................................................. 27 Summary of proposed modifications to the enclosure design. ........................................... 29

8. Adding two 55A power supplies. ......................................................................................... 29 Choice of power supply. ...................................................................................................... 29 Specification and assumptions. ........................................................................................... 29 Incorporating the LPS350 power supply models into the enclosure. .................................. 31 Results and comments. ....................................................................................................... 32

9. Adding a single 110A rack-mounted power supply. ............................................................ 34 Choice of power supply. ...................................................................................................... 34 Specification and assumptions. ........................................................................................... 35 Incorporating the HPS35 power supply model into the structure. ....................................... 35 Results and comments. ....................................................................................................... 35

10. Power supply replacement – summary of results.............................................................. 37 11. Possible real-life additions to the enclosure. ..................................................................... 38

Adding backplanes and sub-racks. ..................................................................................... 38 Adding maintenance and access hatches. ......................................................................... 40

12. The influence of external factors on cooling within the enclosure. .................................... 41 i. Blockage of the fan outlet. ................................................................................................ 41 ii. Removal of a complete side panel. ................................................................................. 42 iii Build-up of dust in the ventilation grills. ........................................................................... 43

13. Revisiting the manual calculation. ..................................................................................... 44 14. Summary and conclusions. ............................................................................................... 45

Manual calculation method. ................................................................................................ 45 Simulation techniques. ........................................................................................................ 46 Geometry definition. ............................................................................................................ 46 Modifying the assembly. ...................................................................................................... 46 Possible real-life additions to the enclosure. ....................................................................... 47 The influence of external factors on cooling within the enclosure. ..................................... 47

15. Errata. ................................................................................................................................ 48


1. Defining the enclosure. Before carrying out any analysis work it is first necessary to define the location of the router boards and vents within the enclosure. Overall dimensions of the enclosure have been given in the assignment specification, and these are for a cabinet which is 15U high, 465mm wide and 450mm deep. Two rows of seven router boards, each row 6U high, are to be situated within the cabinet, although the exact location of these items, plus the two vents, are left to the student to decide. The decision was made to locate the two rows of boards centrally, one on top of the other, with a 2U gap below and 1U gap above the router boards. See Fig. 1. The system also comprises a fan mounted on the top of the enclosure, although this has been omitted from Fig.1 for clarity. Note that the drawings in Fig. 1 are not to scale.

Fig. 1. The enclosure. Isometric view (left) and side view (right).

Vents

Router boards

Spacing between PCBs was set at 50.8mm (2”), with a gap of 73mm between the right- and left-hand cards and the enclosure walls. The PCBs were positioned so that they were in contact with the enclosure front panel, as it was felt likely that there would be switches, indicators, etc., associated with the PCBs which would need to be accessible to an operator. The vent heights were set to the full 2U height of 88.9mm and a width of 400mm.

2. Manually calculating the router board operating temperatures. The procedure for manually calculating the router board temperatures was as follows;

1. Based on the enclosure and router board geometry, calculate the airflow resistance of the system.

2. Use the airflow resistance figure to calculate the system pressure loss curve. Overlay

this curve on the fan’s static pressure curve (given in the assignment specification) to determine the system’s operating point and mass flow rate.


3. From the mass flow rate calculate air velocity in the vicinity of the router boards and use this figure to estimate a convection loss figure for the router boards. From the convection loss figure and board power dissipation, an average board temperature rise may be calculated.

Calculating the airflow resistance of the system. The method used to calculate the airflow resistance of the system is taken directly from Gordon N. Ellison’s paper “Fan cooled enclosure analysis using a first order method” [1]. The paper provides simple formulae for calculating the airflow resistance figures of various geometrical structures, as well as demonstrating how those resistances may be combined in series and parallel to create a model for an entire enclosure plus contents. The first step in calculating an overall resistance figure for the enclosure is to break the enclosure down into discrete resistive units. This is shown in Fig. 2.

Fig. 2. The enclosure resistances defined. Front view (left) and side view (right).

RGRILL

RBGAP

RMGAP

RTGAP

RBB RBC

RREAR

Starting at the bottom of the enclosure, RGRILL is the resistance due to the ventilation grills, with two of these items present in the analysis. RBGAP is the resistance of the volume directly below the first row of router boards. Note that this volume is slightly taller than 2U as the router boards are less than a full 6U in height. The airflow path now splits into several elements. At the rear of the enclosure is a single volume extending vertically from the bottom of the lower row of boards to the top of the upper row of boards (RREAR). This volume exists because the router boards do not extend to the full depth of the enclosure. At the front of the enclosure are the eight parallel resistances associated with the lower row of boards. RBC is the resistance between the outer boards and the enclosure walls whilst RBB is the resistance between adjacent boards. Between the lower and upper row of boards is a single volume represented by the resistance RMGAP, which again exists because the boards are less than 6U in height. Above this volume is the upper row of boards, which is identical to the lower row. Finally, a further gap extends across the full depth of the enclosure and is represented by the resistance RTGAP. The resulting resistor network is shown in Fig. 3.


Fig. 3. The enclosure resistance network.

RGRILL

RBGAP

RMGAP

RTGAP

RBB x 6

RBC

RREAR

RBC

RBC RBC

RBB x 6

From Ellison, the resistance of a perforated plate or grill is given by;

2

8280

fA

.R

where Af is the free cross-sectional area for flow. The resistance of a channel is given by;

2

214

A

L.R

where L is the channel length and A is the cross-sectional area. Using the above equations, the following initial values in (N/m

2)/(kg/s

2) were calculated;

RGRILL 2555.6 RBGAP 10.14 RBC 2349.3 RBB 4861.4 RMGAP 3.18 RTGAP 5.84 RREAR 336.8


It is now possible to reduce the resistance network to a single equivalent resistance using the rules for series and parallel addition described in Ellison’s paper. Series resistances are simply added together, however the method for combining parallel resistances is a little more unusual and is given by;

...RRR

21

111

Once the elements representing the rows of PCBs are combined it can be seen that the overall equivalent resistance for a row of boards is only 62.0[(N/m

2)/(kg/s

2)], and the

resistance of the two grills in parallel is 625.0[(N/m2)/(kg/s

2)]. In other words, the ventilation

grills are the single biggest obstacle to the flow of air in this system. Defining the fan operating point. A single resistance figure (RSYS) of 689.9[(N/m

2)/(kg/s

2)] was finally determined for the system

overall. As we know that RSYS, the mass flow rate “m” and the system pressure loss “SYS” are linked by the relationship;

SYS = RSYS.m2

we may now produce the curve for the system pressure loss and hence determine the fan operating point for this system. See Fig. 4.

Fig. 4. The fan curve and operating point.

Pressure v mass flow rate

0

20

40

60

80

100

120

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

Mass flow rate (kg/s)

Pre

ss

ure

(P

a)

Fan static pressure

System pressure loss

Operating point

The graph of Fig. 4 indicates that the mass flow rate for the total system is predicted to be ~0.27kg/s.


Calculating temperature rise within the system. Now that we have a mass flow rate figure for the overall system, it is possible to calculate the mass flow rate and velocity through the board rows only. Referring to the resistance network of Fig.3, we can see that the card rows plus middle gap are in parallel with the empty volume at the rear of the enclosure represented by RREAR. It is reasonable to assume that the mass rate through RBGAP and RTGAP are the same and are equal to the calculated mass flow rate for the whole system (0.27kg/s). We therefore need to calculate the proportion of the mass flow through the board rows only. Fortunately, Ellison provides an equation for determining the flow rate through one arm of a system;

RACK

SYSSYSRACK

R

R.mm

where mRACK is the mass flow through the board rows and RRACK is the overall resistance of the board rows. Given that mSYS = 0.27kg/s, RSYS = 48.9[(N/m

2)/(kg/s

2)] and

RRACK = 127.2[(N/m2)/(kg/s

2)], we can calculate a value of mRACK of 0.167kg/s. From the

geometry of the board row volume, we can also calculate that this mass flow figure corresponds to a mean airflow of 6.3m/s (which is relatively high, but not beyond the bounds of common sense). It will be interesting to compare this figure with that calculated in the later FloTHERM simulations. Ellison offers a method of calculating the temperature rise in a volume based on the mass flow rate through the volume (m) and total energy dissipated within the volume (Q);

m

Q.x.T

410659

Substituting the appropriate figures into this equation, where Q is the total power dissipation

for 14 boards and m = 0.167kg/s, yields a temperature rise of only 2.9C. Clearly this figure is unrealistically low, doubtless because the boards have been treated as a single homogenous unit rather than a complex network of heat sources and thermal resistances. An alternative approach is to consider the air velocity across a single board (6.3m/s) together with a knowledge of heat transfer coefficient h to determine the temperature rise of a single board. As ever, determining a realistic value of h for a given scenario is a far from trivial exercise – particularly so in this case as the airflow is likely to be highly turbulent, there will be thermal coupling effects between adjacent boards and the “ambient” temperature for the upper row of boards will be higher than that for the lower row due to the heating effects of the lower row. Nevertheless, a web search revealed a rough “order of magnitude” figure of >45W/m

2.k in a presentation titled “Thermal Management Considerations for PCBs” by Dr.

Graham Berry of Dundee University[2]. Use of this figure together with the surface area and

power dissipation figures for a single board indicates an average temperature rise of 12.1C, which is higher than that calculated previously, but will still be subject to the same simplifications mentioned above.

3. Using the FloTHERM© thermal simulation software. (Some of the following is paraphrased from the FloTHERM help files, particularly the section titled “Background Theory”.) As the bulk of this assignment is concerned with the use of the FloTHERM thermal simulation software, it is reasonable to first consider some aspects of the software’s functionality as it relates to the application under consideration, before carrying out any simulations. As is common with many complex software packages, there are often several ways to achieve a particular objective, and some of those ways may be more efficient than others. This part of the assignment will consider the main aspects of the modelling methodology chosen, and will explain why those particular methods have been adopted. This section will not, however, seek


to explain every aspect of the software, as it is assumed that the reader is already familiar with the general usage of FloTHERM. Determining convergence. As described in the “Background Theory” section of FloTHERM’s on-line help files, the solution of a FloTHERM thermal simulation is essentially a matter of solving a large number of simultaneous equations. Each grid cell in the simulation model has five associated variables (pressure, temperature and velocity in three dimensions), hence for even a modest number of grid cells, the number of variables to be solved for can be exceedingly large. Such a large number of simultaneous equations cannot be solved by algebraic means, and so a numerical method is instead adopted. The numerical method involves making an initial guess at the value of the variables, determining the degree of error (the “residuals”) associated with that guess and then refining the next guess in an attempt to reduce the magnitude of the residuals. This iterative process is repeated until certain criteria are met and the simulation is declared solved or “converged”. A great deal of academic research has gone into refining the algorithms used to refine the guesses, and a wealth of links to such material may be found at the cfd-online website[3]. It is highly unlikely that any simulation could be run until all residuals are exactly zero – nor would it be necessary to do so. Instead, simulations are usually run until a specific set of criteria are met. Usually those criteria are concerned with residual values falling below a specific limit – typically below 1. However, it is possible that residual values may remain relatively high yet the temperatures of any monitor points in the simulation have become stable over successive iterations. In such a case, a simulation may still be considered converged as the residual errors are not affecting the calculated temperatures. This is an option available for determining convergence in FloTHERM from version 5 onwards. From the FloTHERM Project Manager window if we select Solve -> Overall Control we can see in the Overall Solution Control window that an option exists for “Monitor Point Convergence for Temperature”. Activating this option reveals three new fields – “Required Accuracy”, “Number of Iterations” and “Residual Threshold” with default values of 0.5, 30 and 10 respectively. These values would cause a simulation to be considered converged if the temperatures of all

monitor points within the simulation varied by less than 0.5C over 30 iterations, with the values of all residuals being less than 10 at the 30

th iteration.

Since complex simulations may never result in all residuals having values less than 1, yet may well have monitor point temperatures which are stable, it has been decided to use the Monitor Point Convergence method for determining convergence in this assignment. Observing how monitor point temperatures vary with successive iterations also provides a useful “sanity check” when running a simulation. Generally, calculated monitor point temperatures will be roughly in the “ball-park” of their final values within twenty or thirty iterations, even though it may take many more iterations for the convergence criteria to be met. If a mistake has been made in setting up the simulation (i.e. incorrect power assigned to a dissipating object, material property incorrectly assigned, etc.) which cannot be highlighted by FloTHERM’s internal sanity checker, then this may well result in monitor point temperatures being grossly higher or lower than expected. Observing the monitor point temperatures at an early stage, even before convergence, may highlight the existence of such an error without having to wait for full convergence. Gridding the simulations. The nature of the gridding in a simulation can have a profound effect both on the accuracy and run-time of that simulation. A grid that is too coarse (too few grid cells) may converge quickly but will produce results that are inaccurate, whilst a grid that is too fine (excessive number of grid cells) will take a long time to converge or may not even converge at all. Another important point to note is that a FloTHERM simulation is not restricted to cubic grid cells (in fact, typical grid cells are rarely cubic), and a proliferation of grid cells with large “width to height” ratios may also result in non-convergence and should be avoided.


Fortunately, we are not limited to having one fixed gridding scheme for the entire volume of a FloTHERM simulation, and it is possible to have fine gridding for areas of detail (such as PCBs and components) and coarser gridding for volumes of space where there is little geometric detail. There are several ways of applying such “mixed gridding” to a FloTHERM simulation. The method chosen for this assignment is as follows;

1. The overall volume of the rack (which initially occupies the entire solution domain) is gridded using a coarser “System Grid”. This grid will be applied throughout the solution domain unless locally overwritten by a volume of finer gridding located further down the “Root Assembly” tree in the Project Manager.

2. Each router board will have an associated Volume Region whose dimensions are

slightly larger than those of the router board. The sole function of the Volume Region is to apply its own attached grid constraints to the volume in and around the router board. The Volume Region is located within the router board sub-assembly, so that when the router board is moved or duplicated the local gridding is also moved or duplicated. This was felt to be the easiest approach to localised gridding, as specifying a single Control Volume is much easier than determining the grid dimensions, grid inflation, etc., for the grid constraints attached to numerous items of geometry. Isolating the grid for each router board in this manner also has the advantage of preventing any “keypoint” gridding from protruding too far from the router card geometry, as will be discussed later.

The question of exactly how much gridding to apply in a given simulation is one which has no simple, single answer. An excellent paper on this subject – “Validation of Localized Grid in Version 4.2” by Bornoff [4] – covers the subject in great detail. A simple summary of the findings in this paper appears to be; “keep adding more grid until the solution stops changing!”, which is the method which will be applied in this assignment. Bornoff also makes the point that, in simulations with mixed gridding, large jumps in gridding should be avoided. Gridding in CFD models generally has also been the subject of a great deal of research, and links to material on this subject are available at the previously referenced cfd-online website. Solver Option. This setting is accessed from Project Manager -> Solve -> Overall Control… Two types of solver are available – “Segregated Conjugate Residual” and “Multi Grid”. The former operates according to the method described in the “Background Theory” section of the FloTHERM Help, where each grid cell is treated as a distinct separate entity. The latter is a relatively new innovation available in version 5 of FloTHERM, where groups of grid cells are initially grouped together in coarser “lumps” which are solved to create intermediate solutions. In practice, the Multi Grid solver has proved extraordinarily effective in aiding the solution of certain types of simulation. Specifically, those cases employing detailed package models composed of various different types of conductor in close thermal contact. The Multi Grid solver has reduced convergence times by up to a factor of ten and, in some cases, enabled convergence where previously none had been possible. General modelling parameters. A FloTHERM simulation consists of two volumes of space – the “solution domain” and “ambient”. The solution domain is the volume in which calculations are carried out, whilst ambient is treated as an infinite volume outside the solution domain. Matter and energy a free to pass into or out of the ambient environment, however the thermal characteristics of the ambient environment are fixed according to settings made by the user. Before a simulation is run it is necessary to define certain parameters affecting both the solution domain and ambient. These settings will be briefly described in the order in which they are accessed in Project Manager.


From Project Manager -> Model -> Fluid This setting determines the fluid (gas or liquid) within both ambient

and the solution domain and is set to “At at 50 deg C”. The temperature of the air will remain constant in the ambient environment, but is variable within the solution domain according to the calculations carried out therein.

Global This dialogue allows settings to be made for the ambient pressure,

temperature and radiant temperature. For this simulation these would be 1 atmosphere, 50 deg C and 50 deg C respectively.

Modelling Set to; Flow and Heat Transfer, 3-Dimensional, Radiation On and

Steady State. It is also useful to select “Store Surface Temperatures” as this will allow surface and plane plots of temperature to be viewed in FloMOTION.

Gravity Normal, negative Y-direction. Transient not applicable. Initial variables default settings. Auxiliary variables default settings. Radiation. The role of radiation in a cooling scheme depends very much on whether the scenario is being cooled by natural or forced convection. For cases cooled by natural convection, cooling by radiation can be significant – accounting for up to perhaps 15% of the heat lost from the system. For systems employing forced convection, such as the one under consideration here, the role of radiative heat exchange is not so important, and becomes less significant with increasing air speed. FloTHERM’s treatment of radiation exchange is describe in the relevant help files and will not be repeated in detail here. However, to summarise briefly, any surface may be assigned a radiation attribute. This attribute will determine whether the surface is able to transmit and receive thermal radiation and to what degree the surface is subdivided for the purposes of view factor calculations. The latter point is an important one as it can have a dramatic effect on the time taken to calculate radiation view factors. Dividing a surface up into smaller sub-divisions may result in a more accurate calculation of radiation exchange, and FloTHERM allows the user to set subdivisions from none (termed “SINGLE” i.e. the surface is treated as a single radiating entity) down to almost any sub-division such as 1mm or less. In cases where highly accurate modelling of radiation is required (such as finely detailed geometry experiencing natural convection only, or radiation exchange in a vacuum where there is no convection at all), then fine subdivisions may be necessary. However, given that we are dealing with a scenario in which there is significant forced convection cooling, and one where there is a substantial amount of geometry to consider, it seems reasonable to use the SINGLE setting for radiation exchange between all surfaces.

4. Defining the router board. Importing the board from FloPCB. The router board used in this assignment has already been optimised and simulated in FloPCB in Assignment 1. The easiest way to transfer the board geometry from FloPCB to FloTHERM is to save the project as a .pdml file in FloPCB and then import this file into FloTHERM. The .pdml file format contains all the information regarding geometry, materials,


environment definitions, etc., but does not contain any results data. The decision was made to use the FloPCB project without detailed models i.e. with components modelled by 2-R structures only. This decision was made in view of the fact that;

1. There was very little difference in results between the detailed and 2-R models when run in FloPCB.

2. The application in this assignment will require fourteen identical instances of the

board, and so some steps must be taken in order to minimise complexity, gridding and hence simulation time.

The board was imported into FloTHERM and run as-is, the only modification being to move the PSU plus heatsink to the lower right-hand corner of the PCB, as discussed in Assignment 1 <Board.pack>. Temperature results were found to closely match those produced for this assembly in FloPCB. It is now possible to observe the heat plume rising from the PSU heatsink, as discussed in Assignment 1, using the FloMOTION facility in FloTHERM. See Fig. 5.

Fig. 5. FLOMOTION plot showing the heat plume rising from the PSU heatsink.

Rationalising the router board model. Before duplicating the router board model or attempting to position the router boards within the rack model, it was felt sensible to determine whether the router board geometry could be simplified in any way. At the very least, a full complement of 14 router boards would comprise 378 2-R component models, 392 temperature monitor points, 14 PCBs and 28 heatsinks as well as the geometry associated with the rack unit and fan. This is a substantial amount of geometry, and so any superfluous elements should be eliminated from the model. Close inspection of the router board model revealed the following;


1. Numerous empty sub-assemblies were present in the model. These were assumed to

be artefacts of the FloPCB modelling methodology and were deleted as they serve no purpose in a FloTHERM simulation. The grouping of component sub-assemblies within the model hierarchy was also rationalised.

2. Each component model had its own case temperature monitor point, in addition to the

junction temperature monitor point inherent in a FloTHERM 2-R model. It seemed reasonable to delete these monitor points as case temperatures are not specifically of interest in this exercise, and to keep them would have increased the number of monitor points in the final assembly to almost 700.

3. Each component 2-R model had been given only a generic name such as “PQFP144”

rather than a component identity such as “U1”. For this reason, when viewing junction temperatures in the Tables view or Profiles window, it would not be possible to identify individual components within a group of identical component types. All components were therefore renamed with their specific “U” numbers.

4. Some slight misalignment between component edges was observed. This would

result in the creation of undesirable long, thin grid cells as described above, as well as contributing unnecessarily to the total number of grid cells in the model. Some of the components were therefore moved slightly in order to rationalise their alignment.

In addition, individual component grid constraints were deleted, with the intention of having only a single grid region per router board, as described below. The solution domain was resized somewhat in order to allow more space around the router board model. The solution domain boundaries were all set to “open” with ambient conditions set to match those inside the solution domain. From Solve -> Overall Control, the Multi Grid Solver Option was selected. The “Monitor Point

Convergence for Temperature” option was also selected, with Required Accuracy of 1C, Number of Iterations = 30 and Residual Threshold of 20. From Model -> Modelling, the Store Surface Temperatures option was selected so that the PCB and component temperatures could be inspected in FloMOTION <Board v02.pack>. Optimising the router board local gridding. The local gridding for the router board model was provided by a single volume region, sized to be slightly larger than the maximum external dimensions of the router board. As well as allowing local gridding to be specified for the router board, this approach also prevents the board keypoint gridding from spreading into the simulation as a whole and therefore helps to rationalise the gridding for the overall simulation. (Keypoints are grid lines which are applied by default to the corners of geometry, even when no overall gridding has been specified by the user. Keypoint gridding will extend from its geometry to the edges of the solution domain unless it is constrained by a volume of localised gridding. Allowing keypoint gridding to extend unchecked is generally a bad idea as it can result in a proliferation of long, thin grid cells and regions where there is a sudden large change in gridding ratio. Surrounding geometry with a volume region with attached grid constraints will limit the extent of the keypoint gridding, provided no part of the geometry is actually coincident with the boundaries of the volume region.) In order to investigate the effect of localised gridding on simulated temperatures, the router board model was run with a number of different values of maximum grid cell size applied to the localised gridding volume region. The results are summarised in Table 1. In all cases, minimum grid cell size was set to 1mm.


System grid cell max size (mm)

Region cell max size (mm)

Region cell count

Average temp (C)

Max temp (C)

Min temp (C)

10 5.0 80154 100.8 105.5 90.2

10 7.5 40749 100.9 105.7 90.2

10 10 22848 101.1 106.0 90.4

12.5 12.5 15360 101.5 106.4 90.4

20 20 11520 101.5 106.5 90.4

Table 1. The effect of various gridding arrangements on router board temperatures.

Note that in Table 1 the System grid cell size was adjusted to be the same as the Region cell size for the larger Region cell variations. This is because the localised gridding of a volume region may not be larger than the overall System gridding (in such a case, the cell size of the System grid will simply over-write the localised gridding). It is clear from this exercise that the cell size of the router board localised gridding makes very little difference to the simulated temperatures, and hence a maximum grid cell size of 20mm may be used without significant loss of accuracy. For a total of 14 router boards we would therefore have 14 x 11520 = 161280 grid cells.

5. Modelling the enclosure. Empty enclosure specification. The internal dimensions and wall thickness of the enclosure are given in the assignment specification. From these, external dimensions of 469mm (w) x 671mm (h) x 454mm (d) may be calculated. The enclosure material is also specified as mild steel. There are two possible approaches to sizing the solution domain for this simulation;

1. Specify a solution domain size which is somewhat larger than the enclosure. In this way, the effective heat transfer coefficients from the sides of the enclosure will be calculated by FloTHERM.

2. If we already know the heat transfer coefficients from the enclosure sides, then we

need only specify a solution domain as large as the outer dimensions of the enclosure. Since any reduction in solution domain size should result in a faster-converging simulation, and the heat transfer coefficients have been given in the assignment specification, it is reasonable to size the solution domain to the same dimensions as those of the enclosure i.e.469mm x 671mm x 454mm. In this way, air may pass to/from ambient via the enclosure vents and fan, whilst heat energy may be transferred to ambient by conduction through the walls of the enclosure, according to the specified heat transfer coefficients.

The empty enclosure with vents and fan was constructed in FloTHERM, and is shown in Fig. 6 below.


Fig. 6. The enclosure (from FLOTHERM Drawing Board).

Vents

Fan location and airflow

direction

Simulating airflow in the empty enclosure. Before adding any router board geometry to the simulation, it was felt useful to simulate the airflow in the empty enclosure. This was done as a “sanity check” to verify that airflow within the enclosure was as expected i.e. air drawn in from the two vents and expelled via the fan. The fan characteristics were adjusted as per the fan curve in the assignment specification and the simulation set to solve for “Flow only” as no heat sources were present at this time. Gridding for the empty enclosure was set to 20mm maximum size in all three directions, resulting in a total of 38130 grid cells. The residuals plot from this simulation is shown in Fig. 7 <Assignment2 v02.pack>.


Fig. 7. Residuals plot for the empty enclosure.

It can be seen from Fig. 2 that even after 1000 iterations the residuals for the empty enclosure simulation have remained greater than one. If we were to consider convergence purely in terms of residual values then we would rightly assume that this simulation has not converged. In the absence of any dissipating elements (and their associated temperature monitor points) it is not possible to consider these results in terms of Monitor point Convergence. However, it is entirely possible that when the router boards are added we may well achieve Monitor Point Convergence, even though the residuals continue to remain high. For this reason, it was not felt necessary to investigate the “non-convergence” of this simulation any further, particularly as the FloMOTION plot of airflow revealed patterns of airflow which were consistent with expectation.

6. Simulating the enclosure plus router boards. Initial configuration. The enclosure plus router boards were initially configured as per the assignment specification. The FloTHERM Drawing Board view of this configuration is shown in Fig. 8. Board locations and spacing are as described in section 1, except for a small gap between the front edges of the cards and the enclosure front panel <Assignment2 v04.pack>.


Fig. 8. The enclosure plus router boards – initial setup. Initial configuration results. The residual and monitor point plots for this simulation are shown in Fig. 9.


Fig. 9. Residual and monitor point plots for the initial setup.

Referring to the monitor points (right-hand) pane of Fig. 9, the figures highlighted by the red ellipse are those corresponding to the 14 on-board PSUs. These are not considered “real” data for the following reason stated in Assignment 1;

“The PSU was modelled using a slightly different approach to the other components. ...this item is specifically designed to dissipate the majority of its heat energy into the surrounding air via its base plate and attached heatsink, with very little heat energy passing into the PCB. If we consider the PSU as a single heat generating node, then the thermal resistance from node to PCB would be very high, and the resistance from node to component top side would be very low – almost a 1-R model, in fact. FloPCB does not allow the use of 1-R models, therefore a lumped 2-R component of dimensions 60mm x 60mm x 12.7mm was created (matching the dimensions of the PSU). The Rth j-b value for this component was set to 1000k/W and the Rth j-c value was set to 0.01k/W …A power dissipation figure of 8.8W was set for this component, representing the worst-case load condition. Although a temperature for the PSU “junction” will be reported by FloPCB, it is not felt that this figure will in fact be useful, as the real PSU would consist of multiple heat sources rather than just a single point source.”

The PSU “junction” temperatures are equally meaningless in FloTHERM simulations. Unfortunately there is no way to deactivate the reporting of temperatures for a 2-R model, and hence the PSU temperatures will always appear in the Profiles window. The temperatures for each router board’s Q1 – 27 components are shown in the dense grouping of plots lower


down in the monitor points pane, with minimum and maximum temperatures of approximately

72C and 100C respectively. As there are nearly 400 monitor point temperature plots to report, the analysis of individual monitor point temperatures from the Profiles window is impossible. Furthermore, if the data is exported as a .csv file from FloTHERM’s “Tables” view, the data still cannot be analysed as a spreadsheet in Excel as the total number of columns exceeds Excel’s maximum column count. We therefore have no choice but to use the Profiles view of monitor point temperatures to infer approximate maximum and minimum component temperatures. Similarly, when investigating the effect of geometry changes on component temperatures, we must also use the Profiles windows (and possibly FloMOTION temperature planes) to determine trends in component temperatures. This is hardly an ideal situation, but I can see no alternative method. A plane plot of air velocities for the initial configuration is shown in Fig. 10.

Fig. 10. Airflow speed plot for the initial setup.

The file “A2 v04.avi” also shows an animation of the airflow for this simulation. Note that in this and subsequent FloMOTION views, some of the enclosure geometry has been hidden in order that detail inside the enclosure may be seen


It will be recalled that in section 2 we calculated an airflow speed across the router boards of 6.3m/s. Whilst this calculation was based on an extremely simplified model, it is gratifying to note that the maximum airflow speed in this simulation, as reported by FloMOTION, is 5.8m/s, and that Fig. 10 demonstrates this degree airflow is present across several of the router boards in the middle of the rows. Hence the methodology presented by Ellison is somewhat validated by this result. It can be seen from Fig. 10 that the pattern of airflow is generally as would be expected, with air entering from the two vents on the lower left- and right-hand sides, travelling upwards towards the fan and exiting the closure through the fan on the top of the enclosure. Note, however, the unexpected change in direction around the outer boards in the lower row, where the airflow is calculated to be flowing downwards rather than upwards. Assuming we accept the results produced by FOTHERM, it would be reasonable to describe this phenomenon as non-intuitive, and this serves to demonstrate the difficulty of calculating complex airflow patterns manually. A similar plane plot, this time for temperature, is shown in Fig. 11.

Fig. 11. Temperature plot for the initial setup.


The temperature scale of Fig. 11 has been deliberately chosen to provide a high degree of contrast between high and low temperature areas of the simulation. The plane was also set to bisect the PSUs so that the heat plumes emanating from these elements could be clearly seen. In general, we can see lower temperatures where the air velocity is highest (around the boards in the middle of the rows) and also where items are not subject to heating effects from elements directly below them (i.e. in the lower row of cards). The position of maximum cooling efficiency is therefore in the middle of the lower row, whilst the least effective cooling occurs in the outer positions of the upper row. Even in the worst case, however, the highest junction temperature (as reported in Profiles, Fig. 9, and ignoring the PSUs) is only slightly above

100C and therefore still within specification. With regard to air temperatures, the situation is unfortunately less clear. By varying the scaling of the FloMOTION temperature plot it was possible to determine that the highest air temperatures occurred around the PSU heatsinks.

The lowest temperature, in the middle of the lower row, was found to be approximately 80C (and therefore just within specification) whilst the highest temperature, on an outer board in

the upper row, was found to be approximately 188C, which is clearly well out of specification. Although the PSU model is very simplistic, as described in the Assignment 1, the reasoning applied to the creation of the model, plus the choice of heatsink as outlined in the manufacturer’s documentation, still appear to be sound – at least as far as the “best-case” location is concerned. However, none of that reasoning took into account a scenario where there would be significant heat input from adjacent sources. On balance, I am therefore inclined to conclude that there will be a significant temperature problem associated with the air immediately adjacent to some of the PSU heatsinks.

7. Modifying the enclosure design. From a thermal point of view, any improvement to the enclosure design must increase the airflow around the dissipating elements within the enclosure. Only by doing so will it be possible to reduce operating temperatures, as the flow of air provides the primary thermal pathway for the heat energy to reach ambient. Whilst it is the fan which provides the “motive force” for the flow of air in the system, that flow is impeded to a degree by the mechanical parts of the system, as has already been discussed in section 2. To increase the flow of air we could use a more powerful fan (if such is available within the existing size constraints), however a more elegant solution would be to reduce the system’s resistance to airflow. Some of the factors contributing to the resistance cannot be changed – for instance, the number of boards is fixed as is, to a large extent, their position. Similarly the enclosure dimensions may not be changed. However, one element that can be changed, and is likely to have a significant effect on the system’s resistance to airflow, is the size and location of the grills. It will be recalled from section 2 that the grills were by far the largest contributor to the overall system resistance, so optimisation of these components would seem to be a logical next step. Three possible variations in grill properties will be explored;

a) Grill size. b) Grill location. c) Grill construction

a) Varying the size of the grills. In the simulations carried out so far, the grills have measured 400mm wide and 88.9mm tall, with all other attributes as per the assignment specification. In order to investigate the effect of grill size on cooling efficiency, two further simulations were created. In these simulations, the grill heights were increased to 340mm <Assignment2 v04a.pack> and 608mm <Assignment2 v04b.pack>, with all other aspects of the simulation left as-is. From these simulations it was hoped to compare both airflow patterns and temperatures for the three different grill sizes, to see if any overall trend could be deduced. Plots of airflow speed for the three simulations are shown in Fig. 12. Note that the these plots are different from the velocity plot shown previously as they only show scalar (magnitude)


values, not direction. It was felt that this was a clearer way to present the information when directional information was not required.

Fig. 12. Airflow speed plots for the 89.9mm grills (top left), 340mm grills (top right) and 608mm grills (bottom left).


Referring to Fig. 12, the top left plot is for the original simulation and it can be seen that the region of highest air speed (in red) correlates very well with the previously observed region of most effective cooling in the middle of the bottom row. As the grill size is increased, the region of highest air speed tends to move upwards and, in the bottom left plot, the trend is almost reversed with comparatively little movement of air across the boards in the lower row. As the cooling of the boards is largely dependent on air speed, we would expect to see temperatures increase as air speed decreases, and this is exactly what is shown in the temperature plots of Fig. 13.


Fig. 13. Temperature plots for the 89.9mm grills (top left), 340mm grills (top right) and 608mm grills (bottom left).

The upper left plot of Fig. 13 shows the temperature distribution of the original configuration, with the coolest locations in the centre of the lower row as previously discussed. As the grill size increases, and the region of greatest air speed moves upwards, so the temperature of


the lower row generally increases whilst the temperature of the upper row decreases somewhat. The variations in maximum component Tj and air temperature with grill height are shown in Table 2.

Grill height (mm) Maximum Tj (C) Maximum air temperature (C)

89.9 ~100 ~188

340 ~95 ~150

608 ~98 ~166

Table 2. The effect of grill height on temperatures.

The results in Table 1 suggest that increasing grill size only improves temperatures up to a point – make the grills too large and temperatures may start to increase in some regions of the enclosure. This result may initially appear to be counter-intuitive – surely larger grills equate to increased airflow and hence lower operating temperatures? In fact this is not the case. In general, the air will always tend to flow mainly along the path of least resistance. As the grill height increases, so air will tend to flow preferentially through the top of the grill (which is nearest the fan and hence has the path of least resistance) at the expense of airflow lower down. Hence we see a very large grill providing a benefit for the upper row of boards, but not the lower row. A medium grill height – 340mm – would appear to be the best compromise, allowing generally increased airflow without unduly depriving the lower region of the enclosure of airflow. Conclusion – larger grills improve cooling, but only up to a point. b) Varying the location of the grills. So far we have only considered grills located on the sides of the enclosure. Although the possibilities for other locations are limited, a potential alternative lies in having grills on the front and rear of the enclosure rather than on the sides. Such an arrangement ought to allow for more uniform airflow through the boards than was the case for the grills located on the enclosure sides. This is because, for side grills, the airflow is partially obstructed by the boards at the ends of the rows. The new arrangement will also create which is naturally travelling parallel to the surfaces of the boards – the direction which is required for optimum cooling efficiency. A new simulation was run, with the side grills removed and replaced by solid enclosure walls. A 2U high grill was inserted in the enclosure front panel, directly under the lower board row, and a 340mm high grill was also inserted in the enclosure rear panel <Assignment2 v07.pack>. The simulation was run, and a plot of air velocity is shown in Fig. 14.


Fig. 14. Airflow plot for the enclosure with front and rear grills.

Fig. 14 clearly shows that the pattern of airflow in the modified arrangement is much more straightforward than in previous examples, and is running parallel over the surface of the boards. This is also reflected in much shorter convergence times for this simulation. A plot of temperatures (Fig. 15) also reveals that cooling throughout the enclosure is much more

uniform, with a maximum air temperature of ~138C - an improvement of 12C over the previous best-case for side grills. This optimisation exercise was therefore deemed to have been successful, though it is noted that the air temperatures associated with the PSUs are still very much over specification. Conclusion – front- and rear-mounted grills allow for better airflow and more uniform temperature distribution. N.B. The possibility of including both side and front-and-rear grills has not been considered. This configuration was not felt to be viable due to the likely weakening effect on the overall structure.


Fig. 15. Temperature plot for the enclosure with front and rear grills.

c) Varying the grill construction. So far we have only considered the grill type detailed in the assignment specification i.e. perforated with holes 2mm in diameter with a 2.5mm pitch. In order to assess the influence of the grills on overall system resistance to airflow, a further simulation was carried out with the grill areas removed but the holes in the enclosure walls left in place <Assignment2 v09.pack>. In other words, with zero resistance. The results are shown in Table 3.

Configuration Max Tj (C) Max air temp (C) Max air speed (m/s)

With grills ~94 134 4.6

No grills ~94 138 4.7

Table 3. The effect of grills on operating temperatures and air speed.

The data in Table 3 demonstrates that removing the grills completely has had no effect on Tj and only a minimal effect on air temperature. As there are very good reasons for having the


grills in place – not the least of which being safety – and it is not possible to have a grill with less resistance than empty space, there is no point in removing or attempting to optimise them further. Altering the fan specification. A survey of fan manufacturers’ data sheets reveals that, in general, the open volume flow rate of a fan is roughly proportional to the its dimensions. So, for a given size of fan, there will always be a maximum upper limit to the flow rate, irrespective of the subtle differences between different manufacturers’ products. As we are limited to a fan size little greater than that already specified, there would seem to be little scope for specifying a different single fan with a higher flow rate. In addition, in his book “More Hot Air” [5], Kordyban states that several smaller fans operating in parallel do not always produce an overall net flow rate which is the simple sum of their individual capabilities. Interference effects between adjacent fans may result in an overall flow rate which is significantly lower than might be expected. Therefore the possibility of replacing a single large fan with several smaller ones in parallel would appear to be limited also. One possibility which was considered worthy of further investigation (primarily due to a paucity of data on the subject), was that of employing a second fan in a “push-pull” or “series” configuration. In other words, the original fan would remain situated on top of the enclosure, pulling air up through the space below it as before, whilst a second fan would be added lower down in the enclosure, pushing air into the space. The enclosure variation shown in Fig. 14 was therefore modified so that the grill shown on the right was replaced with a duplicate of the fan, with source direction into the enclosure, and the smaller hole plus grill on the left was completely removed. This was done so that none of the air from the new fan was allowed to escape directly back out into ambient. The simulation was run, producing the plot of airflow shown in Fig. 16 <Assignment2 v10.pack>.


Fig. 16. Airflow plot for the enclosure with additional fan added.

The airflow plot of Fig. 16 looks remarkably similar to that of Fig. 14 in the region of the router boards – especially so considering that the scale is the same for both plots. Disappointingly, a comparison of results from this simulation against results from the previous case shows no improvement in junction temperatures and a significant increase in air maximum temperature. See Table 4.

Configuration Max Tj (C) Max air temp (C) Max air speed (m/s)

One fan ~94 134 4.6

Two fans ~94 143 5.1

Table 4. The effect of adding a second fan on operating temperatures and air speed.

Whilst this result may again appear to be counter-intuitive, it is possible to feel more comfortable with the result if we consider that a fan is simply a machine for moving a certain volume of air through space per unit time. Apart from the two fans, the enclosure is a sealed unit. Therefore the rate at which air can move through the enclosure is ultimately limited by the capability of the slowest fan. Even if we replaced one of the fans with a unit with much


higher capability, the overall speed would still be limited by the less capable unit. Whilst it is true that a mismatch in fan flow rates could result in a change of pressure within the enclosure itself (due to air being put in faster than it can be removed, or vice versa), in these circumstances any such likely change in air pressure would have a negligible effect on the cooling capability of the air stream. In addition, a fan’s mass flow rate will tend to decrease as back pressure increases (see the shape of a typical fan curve), so there is a negative feedback mechanism at work which will tend to also limit the flow rate in such a situation. Summary of proposed modifications to the enclosure design. The following are proposed as modifications which will be beneficial to the design of the enclosure;

1. Grill size should be increased, though not to the point where some regions of the enclosure are deprived of airflow.

2. The grills should be repositioned at the front and back of the enclosure rather than at

the sides, so that airflow over the router boards is direct and even. The following modifications were found to be either ineffective or harmful to the overall effectiveness of the enclosure;

1. Increasing the grill size above a certain dimension was found to improve the cooling of some regions at the expense of others, therefore moving a problem rather than solving it. Care should therefore be taken to avoid making the grills too large.

2. Removing the perforated plates was found to make little difference to the thermal

situation. As grills are required for other reasons such as safety, they should therefore be retained.

3. Adding another fan in a “push pull” arrangement was not at all beneficial and in no

way warranted the cost of such a modification.

8. Adding two 55A power supplies. Choice of power supply. The assignment specification states that the individual PCB power supplies should be replaced with two 55A power supplies – one for each “sub rack” or row of router boards. This gives us a clue that the new power supplies should be capable of fitting into a space no more than 6U high. Beyond that there is no more information given regarding the specification of the new supplies other than the current rating. A search of various power supply manufacturers’ websites revealed a number of possible candidates, with the LPS350 from Astec finally being chosen [6]. This is a universal-input module with a single output voltage of 5V at a maximum of 70A. Although the current rating is well above that required by the specification, it was not possible to find one closer to the 55A figure, and it was also felt sensible to allow a degree of spare capacity so that the unit would not be continually running at its maximum capacity. In a real-life situation this would also allow some scope for upgrading the router boards in the future. Specification and assumptions. A review of the LPS350 data sheet reveals that it has external dimensions of 127mm (h) x 63.5mm (w) x 273.8mm (d) and is hence able to be fitted within a 6U high rack space. The power supply is open at one end and has a small fan fitted at the opposite end. It is assumed that the additional side-mounted fan is not fitted. No other details concerning construction are available, so the following assumptions have been made;


1. The case is made from 2mm mild steel, identical to the enclosure material.

2. The fan draws air into the case, which is expelled through the open space at the rear of the unit.

3. The stated efficiency for this power supply is 75%. Therefore at an output current of

current of 55A the power dissipation of the unit is 92W.

4. The fan has a flow rate of 30cfm (from the data sheet). In the absence of more detailed information, the power supply was modelled as an enclosure with one open face and a hole for the fan at the opposite and. The fan was modelled as a simple “fixed volume” type with dimensions 29mm x 29mm. The unit power dissipation was modelled as a source of 92W, sized to completely fill the internal dimensions of the case. The net effect of this structure is therefore to act as a hot air source. See Fig. 17 <LPS350.pack>.

Fig. 17. FLOMOTION plot of the LPS350 model.

In Fig. 17 the LPS350 model is shown on its side with one additional side made invisible, so that the internal airflow in the model can be seen. Note the vector plot of air speed, showing air drawn in by the fan on the front of the case and expelled at the open rear of the model. The LPS350 case was modelled as conducting so that heat may also pass through the sides by conduction.


Incorporating the LPS350 power supply models into the enclosure. The router boards in both rows were moved slightly to the left in order to make room for the power supplies on their right, and the power supplies were aligned centrally with their respective router boards. As there is significant hot air vented from the rear of both 55A power supplies, it was felt sensible to position the units so that they vent directly to ambient via holes in the rear of the enclosure, rather than into the enclosure itself. To accommodate this arrangement, the lower power supply was mounted flush with the rear grill whilst a new hole was made in the enclosure rear panel for the upper power supply. It seemed likely that the hot exhaust air from the power supplies could in reality be drawn straight back into the enclosure via the rear grill, with an adverse effect on the overall cooling of the enclosure. With the solution domain sized in its present form it would not be possible to observe this effect as the power supplies would vent directly to ambient and any “return path” for their hot air would not be simulated. The solution domain size at the rear of the cabinet was therefore increased somewhat in order that this phenomenon could be modelled. See Fig. 18 <Assignment2 v06.pack>.


Fig. 18. LPS350 power supplies fitted to the enclosure. Note that the router boards have

been hidden, for clarity.

LPS350 power

supplies

Solution domain

extended at rear

Results and comments. The simulation was run and the results compared against those obtained for the earlier configuration with power supplies mounted on the router boards (Table 5).

Configuration Max Tj (C) Max air temp (C)

On-board power supplies ~94 134

2x LPS350 power supplies ~100 100


Although the maximum Tj figure has risen somewhat (possibly because the boards are now mounted slightly closer together), the components are still within specification and, the

maximum air temperature has decreased by 34C. This is undoubtedly because the heat energy from the power supplies, which had previously been heating up the air inside the enclosure, is now channelled out into ambient. However, as previously mentioned, it is worth


investigating whether any of this heat is finding its way back into the enclosure via the rear grill. The plot of Fig. 19 reveals that this may indeed be the case.

Fig. 19. Airflow and temperature plot showing heat flowing back into the enclosure via the rear grill.

In order to eliminate the possibility of hot air flowing back into the enclosure, a pair of “cowl” structures were added to the model. These structures were intended to channel the hot air away from the grill and towards the right hand of the enclosure as viewed in Fig. 19. See Fig. 20 <Assignment2 v05.pack>.


Fig. 20. Power supply exhaust cowls added to the rear of the enclosure.

A FloMOTION analysis of airflow revealed that the cowls were indeed channelling the air away from the grill as intended. However, it was disappointing to note that this measure made no difference to the component or air temperatures. Perhaps if the degree of airflow had been greater, or the router boards mounted nearer the grill, then some effect may have been seen.

9. Adding a single 110A rack-mounted power supply. Choice of power supply. The assignment specification states that the individual PCB power supplies may also be replaced with a single 110A rack-mounted power supply which provides power for all the router boards. A search of various power supply manufacturers’ websites revealed a number of possible candidates in a 1U format, and the HPS35, also from Astec, was finally chosen [7]. The HPS35 has standard 1U, 19” rack dimensions, and a depth of 330.2mm. The unit is in fact comprised of an overall rack cage (HPR1) populated with four HPS35 sub-modules not dissimilar to the LPS350 described above. The complete unit has a single output voltage of 48V at a maximum current of 117A.


Specification and assumptions. The power supply was modelled using exactly the same approach as for the LPS350, apart from the following;

1. The stated efficiency for this supply is typically 80%. Therefore at an output current of 110A the power dissipation of the unit is 330W, and this was set as the model’s “source” figure.

2. The unit has eight fans, each assumed to be 30cfm rated, as before. An initial attempt

to model the eight parallel fans resulted in severe convergence problems – probably due to the complexity of the resulting turbulent airflow. As this is only a simplified model anyway, it was felt justifiable to replace the eight fans with a single “fixed flow” of area equivalent to the fans and flow rate equal to 240cfm.

As before, the net effect of this structure is to act as a hot air source. Incorporating the HPS35 power supply model into the structure. The HPS35 power supply could potentially cause problems for the enclosure’s overall cooling scheme as it s shape more than half covers the whole internal footprint of the enclosure. In other words, if we were to place the power supply directly below the fan at the top of the enclosure, then most of the fan opening would be blocked. Similarly, placing the power supply between the board sub-racks would deprive the lower rack of airflow, also with potentially disastrous consequences from a cooling point of view. Logically, it would seem that the best location for the power supply is as near to the bottom of the enclosure as possible. This also makes sense from the point of view of mechanical stability as the HPS35 weighs approximately 10kg. The power supply was therefore located at the bottom of the cabinet in the first 1U space with a baffle to exclude the power supply exhaust air from directly entering the enclosure. The new arrangement was initially modelled without an exhaust cowl for the HPS35’s exhaust air <Assignment2 v11b.pack>. Results and comments. It was found that convergence could not be achieved for this simulation even after almost 1300 iterations and eight hours run-time. See Fig. 21. However, it was felt that the monitor point values, though unstable, were indicating an approximate rise in maximum T j of around

10C. A FloMOTION plot of temperature and speed indicated that hot air was indeed re-entering the enclosure through the rear grill – this time with significant magnitude to influence the temperatures of the components therein. See Fig. 22.


Fig. 21. Residual and monitor point plots for the enclosure with HPS35 power supply fitted.

It is undoubtedly the case that the exhaust air from the HPS35 has a much more direct path back into the enclosure than was the case for the LPS350 power supplies. This is because the HPS35 exhaust runs directly under the rear grill for the entire length of the grill. It was therefore felt worthwhile to run this simulation again with a cowl fitted to exclude the exhaust air. The concept employed was similar to that for the LPS350s, except that the cowl was reshaped to cover the full width of the enclosure <Assignment2 v11a.pack>. The results are shown in Table 6 together with those for the 2x LPS350 solution.


2x LPS350 power supplies ~100 100

1x HPS35 power supply with cowl ~100 100


The fact that both sets of results are the same should not be a surprise as both power supply solutions aim to do the same thing – remove the power supply dissipation from the individual router boards and channel it safely to the outside environment.


Fig. 22. Exhaust air from the HPS35 power supply re-entering the enclosure.

10. Power supply replacement – summary of results. The results of these exercises are summarised below;

1. The on-board power supplies are a significant source of heat in and around the router boards.

2. Removal of the on-board power supplies results in a reduction of the localised air

temperature of approximately 34C and a reduction in component temperatures of

approximately 6C (best-case improvement).

3. The improvements observed apply whether the 2x LPS350 or 1x HPS35 approach is adopted.

4. Care should be taken, particularly with the HPS35, that hot air is not allowed to re-

enter the enclosure via one of the enclosure grills.

5. Despite all the improvements made so far, we have still not achieved the target that

air temperature within the enclosure should not exceed 80C. However, it has been possible to reduce component temperatures below the specified maximum limit of

110C.


11. Possible real-life additions to the enclosure. Adding backplanes and sub-racks. So far we have only considered scenarios where the router boards are almost completely suspended in space. In reality, of course, this would not be the case – there would definitely be a need to make electrical connections to the boards as well as a requirement to provide some form of mechanical support and rigidity to the assembly. A common way of making electrical connections to a row of boards such as this is by use of a “backplane”. Typically, a backplane would be a PCB mounted a right-angles to the router boards into which the boards connect. The backplane would mainly be passive, i.e. would not be a significant source of power dissipation itself, but would be used for carrying power lines, signal busses, card interconnections, etc. The normal position of a backplane would mean that it will inevitably interfere with the airflow around the router boards and hence have a negative effect on their cooling. In terms of providing mechanical support to the boards, a common approach is to employ a “sub-frame”. A sub-frame is typically a mechanical slot arrangement which allows the router boards to be slotted into precise locations and hence make connection with the backplane. Sub-frames come in several types, from completely enclosed to “skeleton” types. Clearly, any structure which completely surrounds the router boards will have a disastrous effect on their cooling, and it is therefore wise to employ a skeleton sub-rack which has minimal surface area and so minimal influence on the cooling of the router boards. There are many different suppliers of 19” racks, enclosures and accessories. The Schroff company [8] in particular offers a wide range of products supported by excellent technical information, and has been used as reference in this assignment. The influence of both backplanes and sub-racks were investigated in a further simulation – based on the HPS35 variant described above. The modified configuration is shown in Fig. 23. <Assignment2 v13.pack>The backplanes were modelled as simple 4-layer “smartpart” PCBs, with 2oz 80% copper on the outer layers and 1oz 40% copper on the inner layers. The sub-racks were modelled as flat, steel plates incorporating a series of holes between the router board locations.


Table 7 compares the component and air temperatures for the enclosure with and without these modifications.


Without modification ~100 100

With modification ~104 104

Table 7. The effect of backplane and sub-rack additions on component and air temperatures.

The presence of the new internal structures has made a small difference to the temperatures within the enclosure, though not to a disastrous extent. An airflow plot from this simulation demonstrates that the pattern of airflow has certainly been altered – particularly by the presence of the backplanes (Fig. 24). The backplanes also potentially provide an additional conduction path for heat generated by the router boards. However, a surface plot of temperatures for these items revealed that almost no heat was flowing in this path – undoubtedly because of the small area of contact and relatively low conductivity of the backplane PCBs.


In general, it is safe to say that adding any internal components to the enclosure will influence the pattern of airflow – and hence the cooling of the router boards – in some way. Apart from the sub-racks and backplanes already considered, it is likely that cable looms, additional PCBs, conduits, modules, etc., could also be added and that these items may have an adverse effect on the enclosure’s cooling scheme. Care should therefore be taken when adding such items and their influence should preferably be modelled to determine the likely effect on cooling within the enclosure. Adding maintenance and access hatches. The subject of ventilation grills has already been covered at length in section 7 and will not be repeated here. However it is worth repeating the point that larger holes in the enclosure do not always equate to better overall cooling and may be of benefit to some boards at the expense of others. This is an important point to bear in mind when considering the possibility of adding maintenance and access hatches to the enclosure, which would have a similar effect to increasing the size of the ventilation grills. Although the addition of such items may be desirable from the point of view of maintenance and servicing, the investigation in section 7 has demonstrated that they could have a severe impact on the cooling of some parts of the enclosure. If access hatches must be added to the enclosure then they should be of a type which makes an airtight seal when closed. Furthermore, they should only be opened for short periods of time when the rack is powered up. It would be sensible to attach warning notices to the


enclosure stating that the hatches should be kept closed, and this would help eliminate the possibility of someone “helpfully” leaving them open in order to help cool the unit down!

12. The influence of external factors on cooling within the enclosure. This section will examine the effect of the following three circumstances on cooling within the enclosure;

i. Blockage of the fan outlet by, for instance, placing the user manual on top of the enclosure.

ii. Removal of a complete side panel.

iii. Build-up of dust in the ventilation grills.

i. Blockage of the fan outlet. The effect of blocking the fan outlet is primarily to remove any forced convection cooling from within the enclosure. Hence the router board PCBs will be cooled by natural convection only. Another implication of blocking off the fan is that the only source of cooler, ambient air is via the ventilation grills. Setting up the fan blockage in FloTHERM proved to more difficult than anticipated. Initially, an object was placed so as to partially obscure the fan and opening in the enclosure. However, it quickly became apparent that the airflow was hardly affected by this modification. The only explanation I can think of is that the fan “primitive” will still produce its specified flow rate, irrespective of the size of aperture through which it is projecting air. This behaviour is somewhat at odds with the operation of a real fan! To overcome this quirk in behaviour, and hence simulate the condition of no forced airflow, the fan and opening were simply removed from the simulation altogether <Assignment2 v14a.pack>. When running the simulation, it became obvious that convergence was unlikely to be achieved. Even after ~800 iterations the residuals remained high and the monitor points showed no sign of stabilising. See Fig. 25. This can often be the case for simulations where there is no simple path to ambient.


Fig. 25. Residual and monitor point plots with the fan removed.

The best that can be inferred from Fig. 25 is that the monitor point temperatures have risen

significantly – possibly up to a maximum of around 180C. Without doubt a highly unacceptable result, and therefore steps should be taken to ensure that the system fan is never blocked – perhaps by the addition of a wire cage to the top of the enclosure. ii. Removal of a complete side panel. The removal of a complete side panel (as might occur during maintenance) is similar to fitting a very large grill on one side of the enclosure, as has already been discussed in detail. Simulation reveals that making such as modification to the enclosure will completely change the pattern of airflow within. As ever, the air tends to follow the path of least resistance and so the majority of flow is through the open enclosure side rather than through the ventilation grills. See Fig. 26 <Assignment2 v15.pack>.


Fig. 26. Airflow plot with one side of the enclosure removed.

This factor, together with the channelling effect of the router boards and sub-rack, has tended to make the centre of the rows (especially the upper row) the region of fastest airflow and hence most efficient cooling. Perhaps surprisingly, though, this has not resulted in a dramatic

increase in component temperature, with the highest recorded being approximately 102C – which is still within specification. One might argue that there is therefore a good case for not having sides to the enclosure at all. However, there are other practical reasons why a completely enclosed system is desirable, including the need to exclude, dust, foreign objects, insects, prying fingers, etc. iii Build-up of dust in the ventilation grills. The ventilation grills were originally modelled with a hole diameter of 2mm and pitch of 2.5mm, giving a free area ratio of approximately 0.5. In order to simulate the build-up of dust in the grills, the hole diameter was reduced to 1mm, giving a free area ratio of approximately 0.13, and the simulation re-run <Assignment2 v16.pack>


2mm hole diameter ~104 104

1mm hole diameter ~118 117.5

Table 8. The effect of grill hole size on component and air temperatures.


The effect of reducing the hole sizes is of course to limit airflow and hence cause higher temperatures within the enclosure. Taking this idea to its worst-case limit, we would have a situation where the grills were completely blocked and hence forced-air cooling was prevented altogether. This is exactly the same as the case discussed above where the fan was completely blocked, and would also result in excessive temperature rises within the enclosure. In order to prevent such a situation in real life, it would be necessary to ensure that the grills are never blocked with dust or other debris. N.B. Referring to the work done in section 8, on safe removal of the power supply exhaust air, it is worth making the point again in the context of hot air emanating from an external source. If the enclosure is sited next to a source of hot air (such as a radiator or a similar rack unit) then care should be taken that hot air is not drawn into the enclosure via its ventilation grills, otherwise unexpected temperature rises within the enclosure.

13. Revisiting the manual calculation. It will be recalled from section 2 that the manually calculated temperature rise in the enclosure

was considered to be somewhat low – only 12.1C over an ambient of 50C. The reason for such a low figure was felt to be the simplifications employed in the calculation – the boards being modelled as simple flat planes with their total power dissipations distributed evenly across their surfaces. A final simulation was run <Assignment2 v17.pack>, mimicking the simplified configuration of section 2, to see if there was any agreement between the manually calculated and simulated answers. The simulation had the following characteristics;

1. Original grill configuration located at the sides of the enclosure.

2. Backplanes and sub-racks not included in the simulation.

3. All discrete components removed from the router boards and replaced with a single power dissipation figure per board of 35.45W.

4. A single temperature monitor point added to the geometric centre of each board,

used to report an average temperature for the boards.

The simulation was run and it was noted that the board temperatures varied between 59.0C

and 67.8C, i.e. a temperature rise of between 9.0C and 17.8C. Given the high degree of simplification employed in the manual calculation, this was felt to be a reasonably fair correlation! A plot of board surface temperatures is shown in Fig. 27.


Fig. 27. Router board temperature plots for the simplified model.

It can be seen that, even for this greatly simplified scenario, there is still a large temperature difference between some of the boards. My conclusion is therefore that the manual method may give some indication of temperatures and air speeds within a system, however for a more detailed analysis it is necessary to use more sophisticated analysis methods.

14. Summary and conclusions. Manual calculation method. The method described by Ellison for calculating airflow within a system has shown a remarkably good correlation with the simulation results. Ellison’s method predicted an airflow of 6.3m/s across the router boards whilst simulation predicted a peak figure of 5.8m/s – a difference of less than 10%. Unfortunately, even if we have a reliable figure for airflow, the difficulty comes in translating that figure into heat loss for a complex arrangement of PCBs and components. This difficulty is not unique to Ellison’s method, and would exist in any situation where similar calculations were attempted. At best, only a simple calculation can be made, where the heat from a board’s components is considered to be distributed evenly across the surface of the board. Simulation of such a simplified scenario gave a reasonable order-of-magnitude correlation with manually calculated results, though the manual calculation relied very much on


determining a reliable value for h – the convection coefficient. As ever, finding a good value of h for a given scenario is not an easy task. Simulation techniques. Due to the complex nature of the simulation, and the likelihood that residual values would remain high or fall to less than one very slowly, it was decided to use the Monitor Point Convergence method to determine convergence. Given the large number of monitor points present in most of the simulations, this method was not without its own difficulties – not the least of which being the near impossibility of tracking the temperature of a single monitor point within the large volume of data generated by the software. This difficulty was overcome to a degree by using surface plots of temperature in FloMOTION to identify “hot-spots” within the simulations. The Values -> Results Ranges… feature in FloMOTION was also useful for determining maximum temperatures within a simulation. When formulating a gridding scheme for the simulations, great care was taken to a) keep the grid cell count as low as possible and b) avoid grid cells with large width to length ratios. Both these objectives were largely met by the use of localized gridding regions around the router board sub-assemblies. Radiation exchange was incorporated in the simulations, though this was employed at a rather undetailed level. As almost all the simulations featured forced-air cooling, it was felt that radiation exchange was not a dominant factor in cooling the boards and enclosure. Geometry definition. The router board geometry was originally created in the FloPCB package. On importing the geometry into FloTHERM it was found to be over-complicated, with various redundant items and grid constraints applied to individual components. In order to simplify the router board model, the redundant geometry elements were deleted, the remaining items rationalised and the individual grid constraints replaced with a single gridded volume region for the entire board assembly. An investigation was also carried out to determine the optimum grid size for the board gridding region. The enclosure was initially modelled without router boards, in order to check that the airflow within the enclosure was in line with expectation. Once the correct airflow pattern had been confirmed, the router boards were added and subsequent simulations run with the complete assembly. Modifying the assembly. Various proposed modifications to the assembly were investigated. These included

i. Varying the grill size. ii. Varying the grill location. iii. Varying the grill construction. iv. Altering the fan specification. v. Replacing the on-board power supplies with two 55A units. vi. Replacing the on-board power supplies with a single 110A unit.

i. Varying the grill size. It was found that increasing the size of the grills does improve

cooling, but only up to a certain point. If the grills are made too large then the air will tend to flow preferentially through certain parts of the enclosure at the expense of others. The result of this would be that certain regions would be better cooled whilst others would tend to run hot. The conclusion was therefore that increasing grill size could help overall cooling – but care should be taken no to make the grills too large.

ii. Varying the grill location. A distinct improvement in overall temperature distribution

was found when the grills were moved to a front-and-back configuration rather than


the existing side configuration. The new grill location allowed more uniform airflow over the router boards and helped to eliminate hot-spots.

iii. Varying the grill construction. No improvement was found when making the grill holes

larger than originally specified (in fact, replacing the grills with open spaces). As there are good reasons for wanting grills in the first place, it was not felt necessary to modify the grill construction.

iv. Altering the fan specification. A review of fan manufacturers’ data revealed that it

would not be possible to significantly improve upon the existing fan flow rate within the specified space restrictions. This was true whether a single fan or multiple parallel fans were considered. A second fan, identical to the existing one, was added in place of the rear ventilation grill in a “push-pull” arrangement. However, this modification was not found to improve the thermal performance of the assembly.

v. Replacing the on-board power supplies with two 55A units. Removing the individual

power supplies from the router boards and replacing them with two separate power supplies was found to give a significant improvement in air temperature as well as a more modest improvement in component temperature. This was possible because the two 55A power supplies were positioned so as to vent all their hot exhaust air to ambient rather than into the enclosure. In effect, the total amount of heat energy released into the internal volume of the enclosure was significantly reduced by this arrangement.

vi. Replacing the on-board power supplies with a single 110A unit. The benefits from this

modification were the same as for the two 55A units above. However, care had to be taken that the hot exhaust air from the power supply was not allowed to re-enter the enclosure via the rear grill. A cowl was fitted to the exterior of the enclosure in order to prevent this from happening.

Incorporating all of the above modifications resulted in a maximum component temperature

(Tj) of 100C and a maximum air temperature of 100C, Whilst the component temperature is

below the specified limit of 110C, the air temperature is above the maximum limit of 80C. One possibility for lowering this figure is to spread the components out over the router boards utilising the space left by the on-board power supplies. Other more drastic solutions would include reducing component power dissipations, reducing the number of components per board or possibly reducing the number of boards per rack. Due to time constraints, none of these possibilities has been investigated. Possible real-life additions to the enclosure. It is highly likely that in real-life the assembly would have more geometry than was present in the simulations so far. At the very least this would include backplanes and sub-rack assemblies. A simulation was created to determine the influence of these structures on temperatures within the enclosure. The simulation indicated a moderate temperature rise in

the region of 4C. The possibility of adding maintenance and access hatches was also considered. It was reasonable to assume that these could have a similar effect to the over-size grills which had already been discussed at some length, and their use should therefore be treated with some caution. The influence of external factors on cooling within the enclosure. Three external factors were considered;

i. Blockage of the fan outlet. ii. Removal of a complete side panel. iii. Build-up of dust in the ventilation grills.


Further simulations were run in order to assess the impact of these factors.

i. Blockage of the fan outlet. Although this simulation failed to converge, the results obtained strongly suggested that blockage of the fan outlet would result in unacceptably high temperatures within the enclosure – possibly peaking at around

180C. This result is in keeping with common sense expectation and hence steps should be taken to ensure that such blockages cannot happen.

ii. Removal of a complete side panel. Whilst this factor did not result in a disastrously

high temperature rise within the enclosure, there are practical reasons for operating the system with all side panels in place. Those reasons are mainly associated with operator safety and equipment integrity, and therefore operation with panels missing should not be considered.

iii. Build-up of dust in the ventilation grills. Build-up of dust would effectively reduce the

size of the holes in the grills and hence increase the grill resistance. The effect of this would be to reduce air speed within the enclosure and so reduce the effectiveness of the cooling system. In simulation it was observed that reducing the grill hole size from 2mm to 1mm diameter resulted in a temperature rise within the enclosure of

approximately 14C. Hence, the ventilation grills should be kept free of dust. It was also noted that the enclosure as a whole should be sited such that hot air from external sources is not allowed to enter the enclosure via its ventilation grills.

15. Errata. The following is a list of errors, mistakes, etc., which were discovered after reviewing this assignment. It was not possible to correct these errors because either a) time limitations did not allow such corrections to be made or ) no way could be found to do so. Despite my efforts to rationalise the simulation gridding by the use of localised gridding regions, some areas of sub-optimal gridding were apparent and could not be rectified. Considering Assignment2 v04, boards “Motherboard:0” and “Motherboard:7” can be seen to have “leaking” keypoint grid lines on their left-hand sides. This should not have occurred, but it has not been possible to determine the reason why it has occurred. In the simulations involving the single HPS35 power supply and baffle, a small hole was detected between the baffle and inside surface of the enclosure (Assignment2 v11a, Assignment2 v11b). It can be seen that the hole is allowing a small amount of exhaust air to flow directly into the enclosure rather than out via the grill (Fig. 24). In some simulations involving the larger rear grill, this item has become slightly offset to one side and reduced slightly in width.


References [1] Ellison, Gordon N. (1995). Fan cooled enclosure analysis using a first order method.

Available at: http://www.coolingzone.com/Content/Library/Papers/Oct 1995/Article 02/Oct 1995_02.html

[2] Berry, Dr. Graham. Thermal Management Considerations for PCBs. (no date).

Available at: http://www.personal.dundee.ac.uk/~gjzcallo/gjberry4.ppt [3] cfd-online website: http://www.cfd-online.com/ [4] Bornoff, R. (2003). Validation of Localized Grid in Version 4.2. Available at:

http://www.flomerics.com/FloTHERM/support/software/documentation/v42_localized_grid_validation.pdf

[5] Kordyban, T (2005). More Hot Air. ASME Press. pp55 - 58. [6] Astec (no date). LPS350 Series data sheet. Available at:

http://www.astec.com/products/datasheets/lp/lps350.pdf [7] Astec (no date). HPS35 Series data sheet. Available at:

http://www.astec.com/products/datasheets/bulk/HPS35.pdf [8] Schroff website: http://www.schroff.de/

http://www.coolingzone.com/Content/Library/Papers/Oct%201995/Article%2002/Oct%201995_02.html

http://www.coolingzone.com/Content/Library/Papers/Oct%201995/Article%2002/Oct%201995_02.html

http://www.personal.dundee.ac.uk/~gjzcallo/gjberry4.ppt

http://www.cfd-online.com/

http://www.flomerics.com/flotherm/support/software/documentation/v42_localized_grid_validation.pdf



http://www.astec.com/products/datasheets/lp/lps350.pdf

http://www.astec.com/products/datasheets/bulk/HPS35.pdf

http://www.astec.com/products/datasheets/bulk/HPS35.pdf

http://www.schroff.de/

thermal analysis of a 15u telecoms cabinet chris hill ... · thermal analysis of a 15u telecoms...

Documents