EIC Workshop21 May 2008Experience with high trigger rates @JLAB

R. Chris CuevasJefferson LabExperimental Nuclear Physics DivisionTopics Cebafs Large Acceptance Spectrometer CLAS Trigger design parametersPerformance Notes -- 1996 to 200812 GeV Upgrade Trigger Requirements & Solutions EIC Trigger requirements New challenges(A few slides from Dec07 Workshop)Future technology

CLAS Trigger Design Parameters Photon & Electron Experiments with polarized targets, polarized beam High Luminosities 1034cm-2s-1: DAQ event rate designed to 10KHz Dead-timeless, low latency Level 1 (

CLAS Overall Trigger Block Diagram

SECTOR1

SECTOR3

SECTOR4

SECTOR5

SECTOR6

EVENT

PROC

TRIGGER

SUPERVISOR

TOF_D

TOF_T

EC

CC

SECTOR2

Forward Carriage Trigger CrateLevel 2 PassLevel 2 FailBUSYLevel 1 AcceptCLEARBranch Cablesto all ROCSLevel 1 TriggerDistributionFCRocsStart/Gate*DG535DELAYDECK1DG535DELAYDECK2DG535DELAYDECK31877ROCSSTOPSDeck11877ROCSSTOPSDeck21877ROCSSTOPSDeck3OR

BUSYLEVEL2LOGICHBTG**FROMDriftChambers[ADBs]* LAC Not Shown ** HBTG == L1 Accept

Cuevas -- Electronics Group 18 May 1999

Level 2 Trigger Block Diagram Track Segment Finders and Majority Logic R1AxialR3AxialR2StereoR3StereoR2AxialeSector 1142Sector2Sector3 Sector4 Sector5 Sector6

DriftChambersSegmentCollectorsSpace FrameDecks 1-3Segment Finders Each Superlayer VME Majority LogicVME Control to Select Majority FunctionCPUNIM/ECLTo TDC {DC4}To Forward CarriageLevel 2 Latch**** Majority Logic Boards designed on VME Flexible I/O format Two NIM outputs per board. Majority Logic function selected by VME control. One output drives a local TDC and the other output goes to the forward carriage Latch module. Cuevas -- Electronics Group 18 May 1999

CLAS Trigger Performance Notes (1996 Present)

All FastBus front end modules 1872A TDC, 1877TDC, 1881 ADC Struck FastBus Interface Motorola VME Cpu No Level 2 implemented Long conversion time (1.8us/chan 1872A single hit TDC) limits trigger rate Events NOT pipelined in the ReadOut Controller (ROC) ATM network to ROC < 3KHz event trigger rate

Level 2 implemented - Drift Chamber regions with majority logic Electron(L1) and a track in a sector can be combined for L2 pass or fail Replace 1872A with VME pipeline TDC(CAEN) Upgrade Motorola Cpu (ROC) 100MB Ethernet network adapted to ROCs Other DAQ methods improve event trigger rate to ~8KHz

LIMITATIONS Trigger Supervisor support of 32 crates Max Triggers are not pipelined ( 1 Trigger ->1 Event readout cycle ) Gated ADC (1881) Analog signal *stored* in delay cable Photon experiment triggers not easily implemented with Level 1 hardware

CLAS Trigger Performance Notes (1996 Present)

Other Performance Notes

Relatively low failure rate of FastBus instrumentation Aging BiRa FastBus crate power supplies will be replaced with Wiener product for 1877 TDC crates only Air flow cooling design has worked well Virtually no hardware failures for custom Level 1 Trigger System *VXI crate and power supply converted to Wiener product Virtually no hardware failures for custom Level 2 Trigger modules (~400 )

Very recent implementation of VME CAEN programmable (FPGA) logic modules for the g12 photon beam experiment. Photon trigger hardware is coupled to original Level 1 modules to create triggers for CLAS. In use since April 2008, with excellent results and new trigger GUI.

12 GeV Trigger Requirements

Hall D Hall B

12 GeV Trigger Requirements

Hall D Reduce total hadronic rate from 350KHz to true tagged hadronic rate of ~14KHz Use Level1 hardware trigger and Level 3 farm to achieve this 25:1 reduction Level 1 hardware trigger efficiently cuts low energy photon interactions Level 1 trigger hardware design goal is 200KHz Level 1 uses:Energy Sum from Barrel CalorimeterEnergy Sum from Forward CalorimeterCharged track counts (Hits) from TOFCharged track counts (Hits) from Start CounterTagger Energy (Hit counts) Simulations show that this Level1 cut method achieves ~150KHz trigger rate Relatively open Level 1 trigger

Physics eventCut backround

12 GeV Trigger Hardware

Hall D How do you perform this Level1 cut with hardware?

Use FLASH ADC for detector signals that are included in the Level 1 trigger Detector signals are stored in front end boards Energy Sum is computed at the board, crate, and subsystem (BCAL,FCAL) Synchronous system and Trigger Supervisor performs event blocking at the ROC level 8us buffer on front end boards allows for trigger decision (latency) Use high speed fiber optic/serial data transfer between front end crates Easily supports 64 readout crates and is easily expandable

12 GeV Trigger Hardware

Block Diagram: Hall D Level 1 TriggerCrateSubSystemGlobalTrigger Supervisor

z

16 channel 250 Msps Flash ADCEnergy Sum ModuleVXSHigh SpeedSerialBackplaneLatest Designs

12 GeV Trigger Requirements

Hall B Photon & Electron Experiments with polarized targets, polarized beam Increase Luminosity to 1035cm-2s-1: DAQ event rate increase 10KHz(25-30MB/s) Retain sector based trigger scheme Add PreCal, Low Threhold Cerenkov counter Add Silicon Vertex Tracker, and Central TOF Upgrade Drift Chamber Level 2 Hardware Replace FastBus ADC modules with FlashADCs (Keep Multi-Hit 1877s TDC)

FlashADC design will be used for Calorimeter Energy Sum and Cluster finding Level 1 trigger will be promptly sent to SVT and the Drift Chamber Level 2 hardware Level 2 will employ a Road Finder to link all three Drift Chamber regions per sector

FlashADC and custom trigger modules will be identical to Hall D for cost savings and efficient use of design and implementation resources!

12 GeV Front End Electronics & Level 1 Trigger Modules- (~400) FADC250- (

12 GeV Trigger Hardware

A few other nice features of JLAB custom Trigger Modules

Collaborative efforts of JLAB Fast Electronics, JLAB DAQ, and Christopher Newport University groups.

Christopher Newport University

Subsystem Processors (SSPs)All subsystem processors reside in Global Trigger CrateAll subsystem processors are same physical PC boards!Each SSP receives up to eight four-lane crate data linksSome SSPs divided into two boards (because of crate count)If so both board Partial Results sent to global processorEight SSPs are needed:Two for BCAL Energy Subsystem Processor (ESP)Two for FCAL Energy Subsystem Processor (ESP)One for Start Counter Hit Subsystem Processor (HSP)One for TOF Hit Subsystem Processor (HSP)One for Tagger Tagger Subsystem ProcessorOne spare!Each subsystem processor sends time-stamped Subsystem Event Reports (SER) to all Global Trigger Processors (as in CTP-SSP link)

Christopher Newport University

SSP

Christopher Newport University

The Global Trigger CrateEight SubSystem Processors (SSPs) on one logical SideOne or more Global Trigger Processors (GTPs) on other SideSSPs are connected to the GTPs via a partial-mesh backplane8 x 2 Mesh Initially (VXS!)Each SSP talks to each GTP via a four-lane Aurora Backplane LinkEach SSP sources two four-lane links to the backplaneEach GTP sinks eight four-lane links from the backplane

Christopher Newport University

The Global Trigger Crate (logical view)SSP ArrayGTP ArraybcalESPsfcalESPstofHPstrtHPphotEPVME/VXSClk/TrigIn

Christopher Newport University

GTP Logic

8 Trigger Bits 2-8 Trigger Bits

Christopher Newport University

Example GTP Trigger Equation ImplementationZ >= TFM*HTOF + EFM*EFCal + RM*((EFCal +1)/(EBCal + 1)) HTOF - Hits Forward TOFEFCal - Energy Forward Calorimeter EBCal - Energy Barrel CalorimeterEquation was implemented in VHDL using Xilinx synthesis tools and Virtex 5 LX220 FPGA All computing done in pipelined, 32bit floating point arithmetic Subsystem processor data was converted from integers to floating point Equation is computed every 4ns and trigger bit is updated if Z is above a programmable threshold Each coefficient is variable can be changed very quickly without having to reprogram FPGA Used Xilinx specific math libraries (+, -, *, /, sqrt) Synthesis and implementation resulted in using 3% of LX220 FPGA Latency was 69 clock cycles => 276ns delay introduced for forming L1 trigger Algorithm was targeted to run at a 300MHz clock speed without significant effort.

EIC Trigger Hardware Goals(Copied Slides)

EIC Trigger Hardware Goals(Copied Slides)

Summary CLAS high rate trigger system has been very reliable for over a decade Evolution of improvements to trigger rate by upgrading aging hardware FLASHADC used for detector signals that create the trigger 250MHz sample rate(4ns) with 8us data buffer Energy summing and other trigger logic created from detector signals Elegant VXS backplane implementation takes advantage of high speed serial links Latest Field Programmable Gate Arrays used to implement trigger equations Full synchronous system managed by Trigger Supervisor Trigger distributionEvent data blocking and ReadOut Controller interupts Flexible system design (same module designs) used for two complex experimental Halls

New FPGA inputs can accept very high (1.2Gbps) input data from higher speed ADC chips.

Questions? Discussion?

VXS Crate with:(16) FADC-250Sum Board(1) Clock/Trig/SyncCpu not shown

36 Deep19 Standard JLAB RackFan Tray/Crate control Examples of physical rack layout drawings ALL equipment must be shown to identify rack space issues (i.e. Network gear, patch panels, splitter panels, etc.) Airflow/Cooling issues will need to be identified and resolved