TPC Electronics overview

Takao Sakaguchi May 3, 2019

TPC PDR and FDRASR 1 May 3, 2019

2

Design Parameters (I)

TPC PDR and FDRASR

• 160K readout channels from both ends – 48 measurements in radial direction – Continuous readout mode

• 15KHz is the trigger rate – limit of DAQ rate prior to livetime fall-off – ~100KHz interaction may happen for |z|<1m

• dNch/dy = 180 (Au+Au) ~400 tracks in |η|<1.1 – For rate estimate, # of tracks is doubled to effectively

account for bkgd. and fakes~800 – With radial-dependent η-coverage change into account,

the effective # of tracks is ~1050

Design Parameters (II) • Effective # of electrons for Ne-CF4 (90-10): Ntot = 60 ± 7.75 (MIP ∆E for 1.25cm)

– With GEM gain of 2000, it will be 19.2 ± 2.48 [fC]

• Assuming 4 × Ntot as the maximum input charge: 76.8 [fC] – Either gain of 20mV/fC or 30mV/fC will work (2.2V is max output)

• Noise requirement: <0.258 [fC] or <1610 e in σ – Assuming charge is spread into 3×3 in φ×z equally

• 4σ lower from the signal peak for each pad is (19.2 – 2.48×4)/9= 1.03 [fC] – Setting 1.03 [fC] as 4σ of noise, the noise should be <0.258 [fC] or <1610 e.

• Charge collection width in time for Ne-CF4 is 17.5nsec – 80nsec shaping time covers 4σ of the collection time

TPC PDR and FDRASR 3

GAS Ntot Neff DT 𝑫𝑫𝑻𝑻 𝑳𝑳/ 𝑵𝑵𝑵𝑵𝑵𝑵𝑵𝑵 vDrift Tdrift σt(chg)

𝜇𝜇m/ 𝑐𝑐𝑐𝑐 µm 𝜇𝜇𝑐𝑐/𝑛𝑛𝑛𝑛𝑛𝑛𝑐𝑐 µsec nsec

Ne-CF4 60 32.1 60 106 80 12.5 17.5

May 3, 2019

4

TPC readout system overview

TPC PDR and FDRASR

• FEE receives analog signal from padplane of TPC, and sends the digitized data to the backend electronics (DAM) via optical cable, which will be buffered and compressed at EBDC, and then sent to RCF.

256 ch/FEE 24 DAMs + EBDCs

624 FEEs

Total # of FEEs is 624

# of FEEs per sector: 6 for 20<r<40cm 8 for 40<r<60cm 12 for 60<r<78cm

1 sector = 2π/12 per side

1-2 Tbps (readout)

May 3, 2019

FEE Technical Overview • Continuous readout mode

• Use of 8 SAMPA v5 chips per FEE (256ch)

– SAMPA = 32 * (CSA + Shaper + FADC + DSP) – Newly developed chip with 80nsec shaping – Prototype chips are under test. – Design of full chip is on-going.

• FPGA handles ADC data from SAMPA and

sends to optical module, and processes clock and slow control data from DAM.

• ADC clock will be 18.8 MHz, as we base on

the RHIC beam-crossing clock.

• Two optical transceivers will be equipped per FEEs for 20<r<40cm and 40<r<60cm.

– 960 optical links in total to backend (DAM)

TPC PDR and FDRASR 5

FPGA: Xilinx Artix-7

May 3, 2019

Data rate accounting

6 TPC PDR and FDRASR

• One cluster consists of 3-pads(φ) x 5-times(z) – 10 (bits/hit) * 3 * 5 + 60 (bits, header) = 210 bits

• 48 clusters per track: 210*48 = 10K bits/track

• dN/dη=180 in MB AuAu, 2.29 in MB pp – η coverage of TPC is 2.2, but radius-dependent

η-coverage change is to be accounted for. – To effectively account for background tracks,

numbers are to be increased by 2

• Raw data size for single event – 10M bits for MB AuAu, and 130K bits for MB pp.

• For data size after reduction, see Jin’s talk

System AuAu (Y-1)

AuAu (Y-5)

pp

Collision rate [kHz]

100 170 12900

Raw data size [Gbps]

1100 1800 1700

After LVL-1 trigger [Gbps]

290 400 260

After lossless compression (Gbps)

170 240? 160

Per-event size [Mbytes/evt]

1.4 2.0? 1.3

May 3, 2019

Peripherals (cooling of FEE) • Each FEE dissipates 20W (max)

– 12kW from whole TPC

• Use of Heatpipe employed for cooling CPUs in PCs

– Liquid in the pipe. – Now under test

TPC PDR and FDRASR 7

Cooling scheme testbench by Cacace, Kuczewski, and Pisani

May 3, 2019

Peripherals (LV power to FEE) • Power to be distributed: 20W/FEE, 12.5kW for whole TPC

– 8V (for 3.3V): 4A each (2lines), 5V: (1 line for 4V): 1A – Power distribution should be carefully designed in order to minimize the loss at cables

TPC PDR and FDRASR 8 May 3, 2019

Concept: 4KW MegaPac per 2 sectors: • 4 DC-DCs for 8V • 1 DC-DC for 5V One power distribution box per sector • 18-20AWG cables from the

box to each FEE • (20m cable each)

Status/Highlights (more from John) • Signal from X-ray gun (X-ray fluorescence on 55Mn) to a GEM chamber at has

been readout by the prototype FEE (256 channels)

• Clear signal has been seen in trigger mode (now onto continuous mode). – Hit position plot will be shown by Jin Huang and/or Martin Purschke

TPC PDR and FDRASR 9

Plot made by Jin

May 3, 2019

SAMPA progress (FE) • SAMPA v5 components were produced in a multi-project wafer (MPW) run • Initial test shows a good linearity for 80nsec shaping and 30mV/fC gain.

– Power consumption: 6mW/ch – Noise: ~500e @ Cin=0, ~600e @ Cin=20pF

TPC PDR and FDRASR 10

80nsec shaping

1, CSA+Shaping only 2, ADC only 3, Inclusive chain (FE+ADC)

May 3, 2019

SAMPA progress (ADC, FE+ADC) • ADC and FE+ADC components • ENOB of ADC is found to be better than that of SAMPA v4

– Improvement at 18MHz is seen and is close to expected • Pulse shape is successfully measured by FE+ADC

TPC PDR and FDRASR 11

1, CSA+Shaping only 2, ADC only 3, Inclusive chain (FE+ADC)

May 3, 2019

80nsec, 30mV/fC

ADC

FE+ADC

Rad-hard test • Radiation hardness requirement for FEE

– 25krad TID for 5 years. – We require 50krad hardness.

• Test has been performed using 60Co γ

source (10krad/hr) at BNL – Most of the parts passed the test

TPC PDR and FDRASR 12

Results

May 3, 2019

Magnetic-hard test • Optical transceiver had been of suspect

• Magnetic field hardness test was performed at the dipole magnet at BNL

instrumentation. It went up to 0.5 T (0.7 T for short time).

TPC PDR and FDRASR 13

• We looked for voltage fluctuation as well as error rates on optical communication

• FEE worked just fine at 0.5T

• We will repeat the test at MIT where a 1.5T magnet is available

May 3, 2019

Massive QA for electronics

TPC PDR and FDRASR 14

• ~700 FEE cards and 30 DAM/EDBC should be tested

• Data collection from 26 FEEs will be tested with one DAM and EBDC – After initial burn-in test which can be run in parallel

• Charge linearity, cross-talk, and noise will be checked at: – 80/160nsec shaping w/wo zero-suppression, etc.

• Testing time: ~1 day per sector – ~30 days for whole including spares 6 weeks

May 3, 2019

Pulse distribution board Peripheral Control and DAQ

Workstation

1 DAM + PC (EBDC)

26 FEEs Ethernet

Pulser Control/trigger signal

FEE FEE FEE

Summary and Prospects

15 TPC PDR and FDRASR

• Readout electronics scheme is finalized and detailed design of individual components are being finalized

• SAMPA chip development is one of the key components in TPC FEE

– Test results of component circuits from U. Sao Paulo met requirement. – No issue found so far.

• Most of the FEE parts were found to meet radhard requirement

– Some were not, but we already found alternative parts.

• Magnetic field test for FEE has been performed up to 0.7T – We will repeat the test at MIT where higher magnet field (1.5T) is available

• Readout electronics as whole will be test on beam at FNAL in June.

May 3, 2019

Back Up

16 TPC PDR and FDRASR May 3, 2019

Schedule Drivers

TPC PDR and FDRASR 17

• SAMPA chip development – Start: 29-May-2018. End: 13-May-2020. – Mass testing of the chip will be conducted by Lund U. group (contributed labor)

• Test speed: 800 chips/day 10k chips = 4 weeks including initial setup • Can run in parallel to the SAMPA chip evaluation on pre-production prototype

• Final FEE completion is after completion of SAMPA chip production

– Start: 10-Dec-2019. End: 11-May-2021. – Final FEE board except for SAMPA chips will be designed/fabricated before the

completion of SAMPA chip production and testing – Mass testing of the boards is also included in the schedule

May 3, 2019

Cost Drivers

TPC PDR and FDRASR 18

• SAMPA chip development (U. Sao Paulo, IMEC): $808K – Designing of components, submitting a MPW run, producing MWP chips, and testing

(almost in complete) – Designing full chip, submitting to ER, producing full chips, and testing on prototype

board (on-going)

• Final FEE board production: $527K – Boards fabrication and assembly – FPGA, optical transceiver, and other parts ordering

• LV power distribution system: $216K

– Power modules, distribution boxes, and cables

• Cooling system: $68K – Plates, heat pipe, and peripheral structures.

May 3, 2019

Status and Highlights (more FEE) • New board with two

optical transceivers has been produced

– A small revised version of prototype v1

– Now in assembly

TPC PDR and FDRASR 19

• Hit distribution from 55Fe over two FEEs – Proof-of-principle

that DAM properly handles data from multiple-FEEs

– Data taken by John Kuczewski

Up: FEE1 (SAMPA v4) Down: FEE0 (SAMPA v2)

Two optical transceivers

May 3, 2019

48F MTP Cable MTP coupler MTP-LC breakout LC-duplex cable LC-duplex coupler SFP+

Layout in sPHENIX (1/24 sectors shown)

Rack

EBDC

Patch

sPHENIX

TPC

... 40x LC Duplex

Patch panel Exp. disconnect 5x 1U space requested

MTP (F)

3m

MTP (F)

MTP (M)

58m MTP (F) MTP (M)

MTP (F)

MTP (M)

DAQ Room sPHENIX IR

~1m

15m

One-way optical loss: 1.5dB

DAM FEE x26

2x 48-Fiber MTP

20

Reduction by triggering

21 TPC PDR and FDRASR

• FEE data rate vs radial layers of TPC – as a function of beam clock (BC) – Outer radius gives lower rate

• Case for pp MB @ 12900KHz – Raw data size is 1700Gbps

• Data within a time period with respect to a trigger is extracted at DAM/EBDC (triggering)

• After triggering, the data rate is reduced to 260Gpbs

• Same triggering scheme is applied for AuAu MB as well.

― LVL1-triggered events

After triggering

Raw data

Beam clock (BC)

Beam clock (BC) Ra

dial

laye

r num

ber

Radi

al la

yer n

umbe

r May 3, 2019