uscms hcal

USCMS Mayl 20031

TriDAS UpdateDrew Baden

University of Marylandhttp://www.physics.umd.edu/hep/HTR/hcal_may_2003.pdf

USCMS HCALUSCMS HCALUSCMS HCALUSCMS HCAL

USCMS Mayl 20032

Shield Wall

SBS

HPD

FE MODULE

12 HTRs perReadout Crate,2 DCC

FRONT-ENDRBXReadout Box (On detector)

READ-OUT CrateTrigger Primitives

Fibers at 1.6 Gb/s3 QIE-channels per fiber

QIE

QIE

QIE

QIE

QIE

QIE

CC

A

GOL

DCC

TTC

GOL

CC

A

HTR

HTR

CAL

REGIONAL

TRIGGER

32 bits@ 40 MHz

16 bits@ 80 MHz

CC

A

S-Link: 64 bits @ 25 MHz

Rack CPU

FE/DAQ ElectronicsFE/DAQ ElectronicsFE/DAQ ElectronicsFE/DAQ Electronics

CLK

HTR

USCMS Mayl 20033

DCCDCCDCCDCC

Production boards made

Logic boards madeo Firmware

shakedown

LRBs made DCC motherboards

successfully tested testbeam 2002

Spare StandardPMC Site

(33MHz 64 bit)

3x LinkReceiver

TTCRx

FastTiming/Control

235 pin 2mmConnector

DAQS-LINK64

DCC LogicMezzanine Card

USCMS Mayl 20034

HTR Principal FunctionsHTR Principal FunctionsHTR Principal FunctionsHTR Principal Functions

1. Receive front-end data for physics running Synchronize optical links Data validation and linearization Form TPG’s and transmit to Level 1 at 40 MHz Pipeline data, wait for Level 1 accept

o Upon receiving L1A: Zero suppress, format, transmit to the concentrator (no filtering) Handle DAQ synchronization issues (if any)

2. Calibration processing and buffering of: Radioactive source calibration data Laser/LED calibration data

3. Support VME data spy monitoring Will adhere to CMS VME64 standards

USCMS Mayl 20035

HTR StatusHTR StatusHTR StatusHTR Status

• Rev 1 run Summer 2002 testbeam Board worked well – all functional requirements met Big concern on mechanical issues for production

o Had a difficult experience with previous board manufacturing

• Rev 2 produced March 2003 Board production changes:

o New assembler, in-house X-ray, DFM review, QCo Gold plated (Rev 1 was white-tin) for better QC

Changes to HTR:o Change from Virtex 1000E FBGA (1.00mm) to Virtex2 3000 BGA (1.27mm)o Added stiffenerso Moved all SLB/TPG output to front-panel daughterboardso Modified Rx refclk scheme (the usual TTC/refclk clocking concerns)

Full 48 channel capability (Rev 1 was “half HTR”) As of this date, no issues – this board is functionally a success

USCMS Mayl 20036

HTR Rev 3HTR Rev 3HTR Rev 3HTR Rev 3

• No more design changes – this is the final HTR

• 30 boards delivered April 21 As of Friday (May 2) 12 have gone through final checkout

o All systems except connectivity to SLB

o Fiber links checked out at 1.7Gbaud bit rate (1.6Gbaud is CMS requirement)Frame clock up to 2.0Gbaud bit rate and it stays synchronizedNo BER yet…will do a lab measurement soon12 boards x 16 links ~200 links(~5% of total) with no problems

Minor adjustments will be needed for front panels, stiffeners, etc.

Will battle test these boards this yearo Testbeam to begin this month

o Vertical Slice tests after summer

USCMS Mayl 20037

HTR Rev 3 (cont)HTR Rev 3 (cont)HTR Rev 3 (cont)HTR Rev 3 (cont)

16 Dual-LC O-to-E

VME

Deserializers

Xilinx

Sti

ffen

er

s

6 SLBs

TTC mezzanine

USCMS Mayl 20038

HCAL ClockingHCAL ClockingHCAL ClockingHCAL Clocking

•DCC – no difficult synchronization issues here•For HTR, need 2 different kinds of clocks:

1. Synchronized LHC clock for Xilinx system clock and SLBs

o Maintain phase synchronization with entire CMS pipeline

o Allow SLBs to do their jobo Frequency jitter requirement not critical

2. Precise 2xLHC clock for Deserializer refclk ONLY

o 30-40ps pkpk jitter speco Used ONLY for deserializerso Phase synchronicity with LHC clock not

important

•Princeton fanout board will receive TTC, clean up clocks with QPLL, fanout signals

USCMS Mayl 20039

Princeton Fanout BoardPrinceton Fanout BoardPrinceton Fanout BoardPrinceton Fanout Board

USCMS Mayl 200310

Clock DistributionClock DistributionClock DistributionClock Distribution

HTR

TTC fiber

TTC

CLK80

BC0

Cat6Eor Cat7Cable(very lowX-talk)

CLK40

distributionto 6 SLBs

and to 2 Xilinx

Brdcst<7:0>,BrcstStr, L1A

O/E

BC0

TTCTTC

TTC

FPGA

..

..

TestPoints forRxCLKand RxBC0

..

..

..

..

80.0789 MHz

..

..

..

..

TTCrx

to Ref_CLK of SERDES(TLK2501)

CLK40

CLK80

Princeton Clock/TTC

FanoutBoard

TTCrx

QPLL

USCMS Mayl 200311

TTC receiver - TTCumdTTC receiver - TTCumdTTC receiver - TTCumdTTC receiver - TTCumd

• General purpose TTC receiver board (TTCumd)

TTCrx ASIC and associated PMC connectors

• Will be used to receive TTC signal by HTR, DCC, and clock fanout boards

• No signal receivers! Copper/fiber receivers must be

on the motherboard Signal driven through TTC

connectors

• Tested successfully by Maryland, Princeton, BU groups

USCMS Mayl 200312

HTR Integration Goals 2003HTR Integration Goals 2003HTR Integration Goals 2003HTR Integration Goals 2003

• Continued development of HTR firmware Commission TPG path

o Firmware, LUTs, synchronization, SLB output…

Monitoring, error reporting, etc. (information sent to DCC)

• Testbeam May 2003 Support calibration effort and continue commissioning the system Run synchronously in May

• Vertical slice tests, Fall 03 Fully pipelined, monitoring, TPG, DAQ, synchronization, clocking….

• Develop software to support DAQ activities Testbeam software improvements

o Princeton group built testbeam DAQ Software for commissioning

o Allow us to verify fiber mappingo Download LUTs, firmware version, etc.

USCMS Mayl 200313

HCAL TPGHCAL TPGHCAL TPGHCAL TPG

• Under development… Preliminary FPGA code for TPGs done

o LUT for linearization (downloadable), 0.5GeV steps, 255Gev max ET

o E to ET and sums over as many as 7 channelsNot implemented in code yet…TBD

o Muon window in Eo BCID filter algorithm TBD from testbeamso Compression LUTs for output to SLBs

Utilization is ~50% of Virtex2 3000o We are confident this chip will be sufficient

Simulation effort under way…

• Latency issue See below – we are working on this…

USCMS Mayl 200314

HTR ProductionHTR ProductionHTR ProductionHTR Production• Full contingent of HTRs: 260 boards

Includes 10% spares, 20% spares for parts

• Full production will begin after: Testbeam demonstrates I/O works under battle conditions Successful testing of the 6 SLB daughter card functions Understanding of how to meet latency issues

o We are still some clock ticks short, but firmware is still very immature for the TPG part of the HTR (see slides below)

• If all goes well…sometime this summer or fall There is no reason to hurry other than to finish with the R&D part

of the project We are confident that the current board design will be final

USCMS Mayl 200315

Overall Commissioning ScheduleOverall Commissioning ScheduleOverall Commissioning ScheduleOverall Commissioning Schedule

• Summer 2003 testbeam Repeat previous test w/production prototype boards

• Fall 2003 Slice tests HCAL will join as schedule allows

• 2003/2004 HCAL burn-in Continue with firmware development/integration as needed

• 2004/2005 Vertical Slice and magnet test We will be ready All HCAL TriDas production cards involved

• October 05 beneficial occupancy of USC Installation of all racks, crates, and cards We do not anticipate any hardware integration

o Should be all firmware / timing / troubleshooting

USCMS Mayl 200316

ESR Review Item 1ESR Review Item 1ESR Review Item 1ESR Review Item 1

“Use of an obsolete TI component for the data link receiver”

FPGAXilinxXC2V

LCLC

TITI

TI

TI

TI

TITI TI

LCLC

• Misconception on the part of the committeeo TI TLK2501 is NOT obsolete.

• This is a Gigabit ethernet transceiver.

• There is no reason to believe TI will stop making these parts.• If they do, someone will make

something else compatible.

o Stratos receivers are also NOT obsolete.• Dual receivers are out of favor,

Transceivers are in favor

• What is obsolete is our $99/part. If we need more, they will charge $133/part (or more)

USCMS Mayl 200317


“The random latency problem that comes with using the 8bit/10bit link protocol”

• The “random latency” has to do with the TI Serdes function Two clocks here: incoming data clock and reference clock Serdes part has an internal asynchronous FIFO to implement 8B/10B

protocol But this is NOT the fault of the protocol!

o Any protocol which includes a clock, to be recovered, will have this.

• TI does have a 2-3 clock tick random latency with 50% probability for 2 or 3 We can use VME controllable reset and comparison to achieve the 2

clock tick lesser latency Can readout SLBs and use relative latency to correct pointers Can use FE BC0 signals

USCMS Mayl 200318


“Routing of large no. of stiff cables to the front of the HTRs versus other configurations such as a transition module” Transition module is NOT POSSIBLE. Forget about this.

o Would cost us 6 months at least (time and engineering $)

Strain relief:o Each HCAL rack will have 2 VME 9U crates

o Each 9U crate will have an accompanying 6U strain relief panel

o Changing to 15m quad cables (from 20m 2xdual “Wesley” cables) will greatly reduce torques on SLB cards

We will test these cables this summer – need Wisconsin Vitesse test setup

o Each SLB card will be attached to the HTR front panel, and screwed into HTR motherboard

We believe this will work fine.

USCMS Mayl 200319


“Ensuring appropriate quality assurance and testing at the HTR board fabrication facility”

• We agree, this is a big worry. Have used new high-tech assembler for Rev 3 (pre-production) Note: almost any assembler will have startup issues

o Overall techniques are more important than QA, which comes after the fact

o We have chosen an assembler with very modern (and expensive) equipment.

o An engineering review by the assembler is included in the assembly cost

o Our biggest problem was fine-line BGA (1.0 mm pitch) implementation

Current version uses standard 1.27mm pitch BGA

Given current experience, we believe we have solved this…

USCMS Mayl 200320


“Providing sufficient FPGA excess capability against possible future enhancements to the firmware”

• HTR FPGA change: Virtex/1000E to Virtex2/3000 Current firmware uses

o 83% of all RAM resourcesFIFOs, LUTs, etc. this will not change

o 50% of all Logic resourcesRoom for more logicRoom for more memory (can use distributed memory)

The sky is not the limit, but we think we’re ok hereo Firmware has evolved quite far thanks to Tullio Grassi’s efforts

USCMS Mayl 200321


“Minimizing the trigger latency”

Item LatencyTOF .5

HCAL Optics 1

FE (CCA+QIE) 8-9

GOL 2

Fiber Tx to HTRs 18

Deserializer 2-3

HTR Alignment 6

HTR TPG path 5-10

SLB 3

TPG Cables 4

TOTAL 50 - 57

• Current total 50 – 57 clocks Very rough guesses

o Many numbers have not been measured

• Optimizations: Fiber cables need to be 90m? HTR firmware needs optimization Deserializer random latency fix TPG cables changed to 15m will save

1 tick Others…main efforts over next 6

months

USCMS Mayl 200322

TPG PathTPG PathTPG PathTPG Path

ETcomp

7 10

Muon bit

SumConsecutive

Time-samples9

TP

8

QIE-dataINPUT

LUTLineariz.

and Et

ET[9:0]

2

Compression

LUT

2Muon LUT

1

Delay to synchronize

with BCID

10

L1 Filter

10

Sum in ET

PeakDetection

TP_Bypass

1

0

2 2

2

Mask &Reset

“NO-SHOWER” LUTtake care of cases where showers can leak into a cell and incorrectly set the muon bit.

BCID

BCID avoids to flag as a muon the tail of a

more energetic event

USCMS Mayl 200323

Other ESR ConcernsOther ESR ConcernsOther ESR ConcernsOther ESR Concerns

• Reliability/maintenance Replacement of HTRs not an issue – HTRs not in hi-rad region

• Data link error detection Not difficult to implement, just requires coordination. Under consideration, schemes are evolving, dealing with e.g.

o Loss of synch

o Trigger acceptance violations

o Buffer overflow (actual and warnings so DCC can cause L1 to throttle)

o Use of BC0 from front end

o Inline pedestal determination

o Zero suppression

o DAQ format

uscms hcal

Documents