wbs 2.6.3 readout infrastructure

WBS 2.6.3Readout Infrastructure

Craig DukesUniversity of Virginia

Bill GilbertUniversity of Minnesota

CD2/3a DOE Review of NOAOctober 23-25, 2007

October 23-25, 2007 CD2/3a DOE Review Breakout Dukes/Gilbert/WBS 2.6.3 2

Organization

• 2.6.3.1 Low voltage power supplies/racks• 2.6.3.2 High voltage power supplies• 2.6.3.3 Power cables/cable trays• 2.6.3.4 Power distribution boxes (PDBs)• 2.6.3.5 Cooling• 2.6.3.6 Shipping

1. Power Distribution System (PDS) (Dukes)

2. APD Thermoelectric Cooler Cooling System (Gilbert)

Craig Dukes, Nicole Fields, Emmanuel Munyangabe, Yoshihiro Masui, Andrew Norman

University of Virginia

Bill Gilbert, Roger RusakUniversity of Minnesota

Leon MualemCaltech

WB

S I

tem

s


Power Distribution System

• Feeds electrical power to:– The Avalanche Photodiodes (APDs)– The Thermoelectric Coolers (TECs)– The Front End Boards (FEBs)– The Data Concentrator Modules (DCMs)

• Requirements– Low noise (3.3V + 450V)– Remote control and monitoring– Floating power supplies– Remote sensing for LV supplies– Reliable operation over 10+ years


Power Distribution System Layout

Detector CAEN Wieners PDBs FEBsFD 1 66 198 12,036ND 1 4 10 497

IPND 1 2 6 310


PDS Power Requirements

Power Delivered per Power Distribution Box

Item Voltage Current Max Channels

Total Current

TotalPower

FEB 3.3 V

1 A

64

64 A

260 W

TEC 24 V 0.15 A 64

10 A

232 W

DCM 24 V

1.25 A

1

1.25 A

30 W

Timing 24 V 1.04

A

1/24

1 A

24 W

APD 450 V

40 A

64

0.003 A

1.35 W

FD Power

Delivered Total

3.3 V 51 kW 72 kW

24 V 52 kW 72 kW


Far Detector Di-block Layout


Near Detector Layout


IPND Layout


Electronics Counts


High Voltage Supply• CAEN SY1527LC mainframe• CAEN A1511B card

• 12 ch, 0-500 V, 15 mA• Individually floating channels• Voltage resolution: 100 mV (~0.5%)• Ripple: < 30 mV pp

• Each channel serves 2 PDBs• Only 1 mainframe needed for entire FD• Ethernet (TCP/IP) interface• Widely used at Fermilab

0

20

40

60

80

100

120

140

160

180

200

0 20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 320 340 360 380 400 420 440

V

Ga

in M

A1 A2 A3 A4 A5 A6 A7 A8B1 B2 B3 B4 B5 B6 B7 B8C1 C2 C3 C4 C5 C6 C7 C8D1 D2 D3 D4 D5 D6 D7 D8


Low Voltage Supply• Wiener PL508 System• Floating• Remote sense• 3 3.3V pods, 3 24V pods → each MF

feeds 3 PDBs → 66 MFs needed• Ripple: < 3 mV pp• Ethernet (TCP/IP) interface• Widely used at Fermilab


• Fans out 3.3V, 24V, and 450V to up to 64 FEB boards, and 24V to 1 DCM

• 3U crate

• 16 x 4 + 2 = 66 FEB channels

• LED indicators for crate 3.3V, 24V, and 450V power

• LED indicators for card 3.3V and 24V power

• Front panel on/off switches

• Fused with TVS for 3.3V lines

• Note: FEBs have voltage regulators

• Reference to ground

• Designed and built at UVa

• FD: 198

• ND: 10

Power Distribution Box


Prototype PDB Fabricated

• Works according to specifications

• Minor redesign in progress (mainly to simplify construction)


Mounting the PDS on the Far Detector


Cables

Type

Size

(AWG) Cond.

Length (m) Different

Lengths

#

CablesMin. Max. Ave. Total

3.3V 2 2 2.6 16.0 9.3 1,842 12 198

24 V 10 2 2.4 15.9 9.2 1,822 12 198

Sense 22 2 2.4 15.9 9.3 3,665 12 396

450 V 22 2 7.2 50.2 27.6 2,975 49 198

PDB-FEB 18 6 0.7 3.8 2.3 27,500 64 12,036

PDB-DCM 18 2 1.0 1.0 1.0 198 1 198

Ground 6, 1/0, 3/0 1 2.0 15.0 4 232

• All cables “tray cable”• Oil resistant jackets

Pow

er S

uppl

y to

PD

B


Cable Safety

PDB-FEB4A max

Wiener-FEB24V: 23A max

Wiener-FEB3.3V: 110A max

Power supply to PDB cables can handle full current capacity of power supplies.


Power Budget and Voltage Drops

Detector Total

Total 144 kW

Output 103 kW


Cable Trays


Detector Cooling Overview

• 17 air breathing 2 ton chillers located on upper walkway• Each chiller serves 2 detector blocks• Started one at a time to reduce peak electrical and heat load

Steady state heat load to air per chiller:Heat from APD/TE modules 3.35w x 744 = 2492wHeat from chiller operation (33% capacity) + 2310wNet* heat load to air per chiller = 4802w

Net* total heat rejected to air by 17 units = 82 kw

*Ignores pipe & tube loading, gained from air, rejected to air, approx 2.3 kw per chiller

17 chillers

Detector Top View


Water Distribution Scheme

prefabricated distribution manifolds

prefabricated hose manifolds


Hose Manifold Section

3/4”Sch 120 plastic pipe

1/8” I.D. hoses with valved disconnect

• With single-sided detector readout, all manifolds are identical, 32 hose style.

• Two piece construction eases shipping and handling

overall length 350 cm

Cap

compressioncouplings

distributionmanifold

1.25”

175 cm


Prototype Hose Manifold

Shown with pipe insulation, will be tested with & without insulation


Distribution Manifold Section

1.25” plastic pipe

262 cm(8.6 ft)

+/- 3.37cm

131cm116cm 15cm

3/4” pipe taps

union couplings

ball valvesExpansion joint,6.75cm range,

handles exp/contrand dimensional

tolerances

Return Pipe Section


Water Schematic Far Detector


Water Schematic IPND

Each manifold serves31 APD modules

ChillerSensorPanel

310 APD Modules Total

2 APD rows detector top

3 APD rows detector side

9.8 GPM, 1.04 kW


Water Schematic Near Detector


Water System PressuresWorking pressure and worst case pressure is well under design pressure of 100 PSI


Chiller Control & InstrumentationDCS monitors cooling system, can adjust water temp, and start/stop chillers


Flow Network Modeling

http://www.inres.com/Products/MacroFlow_Electronic/macroflow_electronic.html

Cooling system design is being modeled using:


Hose/Heatsink Net Model


Hose Assy Diff Pressures

Drilled orifices in hose barbs are the dominant restriction governing flowChanges in hose length or heatsink resistance will not be significant


Hose Manifold Network Model


Hose Manifold Temp Uniformity

Total Spread = 0.15 DegC


Hose Manifold Flow Uniformity

Excellent uniformity is due to large relative diameter (0.690 in) of manifold pipe


Distribution Network Model


Distribution Flow Balance

Use of 0.040 orifices in hose barbs creates inherent balance

Detector Top Detector Side

Flow balance will not be disturbed by minor modifications to supply pipingFlow can be adjusted using one valve near chiller without affecting balance


Distribution Temp Uniformity

Detector Top Detector Side

Total Spread = 0.06 DegC


Testing the Cooling Model

Two key model elements will be tested:• Hose/heatsink assembly flow and pressure characteristics will be

measured with various orifice sizes• Prototype manifold pair with hose assemblies will be tested for ambient

heat gain and temperature uniformity

Details:• Chiller will provide 15 DegC water at 64 mL/s• 4 hose/heatsink assemblies along manifold will be instrumented with

thermistors• Input & output water temperatures measured with thermistors• Overall flow rate measured with rotameter, controlled with valve• Input & output pressures measured with transducers• Temp & pressure recorded using data logger and computer


Costs (ND+FD)

Item M&S Labor Contingency Total

Low voltage supplies $542K $6K 25% $682K

High voltage supplies $104K $1K 15% $120K

Power cables $205K $47K 32% $334K

PDBs $370K $68K 21% $526K

Cooling $1,335K $17K 31% $1,778K

PDS shipping $21K $3K 15% $28K

Total: $2,578K $142K 28% $3,471K


Procurement

• Power Distribution System

– Power supplies to be bought and shipped to Virginia to be tested.

– Cables will be cut, terminated and tested at Virginia.

– Power Distribution Boxes will be fabricated and tested at Virginia.

– Cable trays and relay racks will be delivered directly to the Far Detector site and FNAL.

• Cooling System

– Manifolds will be fabricated and shipped by vendor to the Far Detector site and FNAL on a di-block by di-block basis.

– Chillers and cooling interface panels will be ordered and shipped by vendor to Far Detector and FNAL.

Installation in WBS 2.9.4


Schedule

• Lots of float: no critical-path items!• Power Distribution System

• Fabrication of PDBs:• Need 215, including spares. Make and test 1/day with 1

tech• 200 days/1 tech

• Fabrication of cables:• Dominated by 12,000 PDB-FEB cables: • 1 student: cut, terminate, pack 5 cables/h: 80 days/4

undergrads• Will do it in summer of 2010 with student labor

• UVa HEP building has enough storage for complete system → will deliver to detector in 3 shipments

• Cooling System• Vendors can deliver as needed to keep up with outfitting

schedule

wbs 2.6.3 readout infrastructure

Documents

power distribution boxes

power cablescable trays2

high voltage power supplies2

a30 wtiming24 v1

a6410 a232 wdcm24 v1

v1 a6464 a260 wtec24

nicole fields

lv suppliesreliable