HEP UCL

Cambridge UniversityCambridge UniversityImperial College LondonImperial College LondonUniversity of ManchesterUniversity of Manchester

Royal Holloway, University of LondonRoyal Holloway, University of LondonUniversity College London University College London

Matthew Warren, UCLMatthew Warren, UCL

22 March 200722 March 2007

EUDET/CALICE DAQ OverviewEUDET/CALICE DAQ Overview

22-Mar-2007

EUDET/CALICE DAQ Overview 2 HEP UCL

PC/s

Ideal DAQ Structural Ideal DAQ Structural OverviewOverview

•Detector ASICs on e.g. ECAL slabDetector ASICs on e.g. ECAL slab

•Front-End (FE)Front-End (FE)-FE-Interface (DIF): FE-Interface (DIF): Detector specificDetector specific-FE Link/Data Aggregator (LDA): FE Link/Data Aggregator (LDA): GenericGeneric

•Data-link (FE to Off-Detector Receiver)Data-link (FE to Off-Detector Receiver)•CCC-link (Clock+Control+Config to FE)CCC-link (Clock+Control+Config to FE)

•DAQ PCDAQ PC-Off-Detector Receiver/s (ODR)Off-Detector Receiver/s (ODR)-Drives CCC-linkDrives CCC-link-Data StoreData Store

LDA

CC

C-li

nk

ODR

Store

Data-link

DIF

AS

ICs

FEDIF

AS

ICs

DIF

AS

ICs

…

22-Mar-2007

EUDET/CALICE DAQ Overview 3 HEP UCL

FE Structure FE Structure DetailDetail

We have 2+ types of detector to readout.We have 2+ types of detector to readout.Divide the FE into a 2 part, tiered systemDivide the FE into a 2 part, tiered system

1) Detector Interface module (DIF)1) Detector Interface module (DIF)-Detector specific interfaceDetector specific interface-Includes power connectorsIncludes power connectors-‘‘Local’ systems (e.g. stand-alone clock)Local’ systems (e.g. stand-alone clock)-Debug connectorsDebug connectors

2) Link/Data Aggregator module2) Link/Data Aggregator module (LDA)(LDA)

-Collects data from many ‘DIF’sCollects data from many ‘DIF’s-Drives data Off detector linkDrives data Off detector link-Receives and distributes C+CReceives and distributes C+C-FPGA Development boardFPGA Development board-

BUT:BUT:We would might like to read-out slabs individually first…We would might like to read-out slabs individually first…

Link/ Data Link/ Data AggregatorAggregator

PCPC

ECAECALL

SlabSlab

ECAL ECAL DIFDIF

Link/ Data Link/ Data AggregatorAggregator

PCPC

HCALHCALLayerLayer

HCAL HCAL DIFDIF

HCALHCALLayerLayer

HCAL HCAL DIFDIF

ECAECALL

SlabSlab

ECAL ECAL DIFDIF

C+C Fanout

22-Mar-2007

EUDET/CALICE DAQ Overview 4 HEP UCL

Data-link Data-link (+CCC)(+CCC)

•Use most common networking fibre-optics: Use most common networking fibre-optics: -Multimode with LC connectorsMultimode with LC connectors-SFP (mini-GBIC) interfacesSFP (mini-GBIC) interfaces-1Gbit rate (maybe tuned to multiple of machine 1Gbit rate (maybe tuned to multiple of machine freq.) freq.)

-EthernetEthernet

•Control up-link NOT via fibre, Control up-link NOT via fibre, initiallyinitially..

22-Mar-2007

EUDET/CALICE DAQ Overview 5 HEP UCL

Off-Detector Receiver Off-Detector Receiver (ODR)(ODR)

•PCI Express CardPCI Express Card•Virtex 4, FX100 FPGA (big!)Virtex 4, FX100 FPGA (big!)•Hosts opto-linksHosts opto-links

-2xSFP, 2xHSSDC2 on board2xSFP, 2xHSSDC2 on board

•Source of C+C (Control link)Source of C+C (Control link)-Initially copper (LVDS)Initially copper (LVDS)-Later fibreLater fibre

•Will use external clock and sync Will use external clock and sync signals for multi-board signals for multi-board synchronous operationsynchronous operation

22-Mar-2007

EUDET/CALICE DAQ Overview 6 HEP UCL

ODR(2) - ODR(2) - Status Status

Firmware AND software well underway:Firmware AND software well underway:•PCIe interface PCIe interface DONEDONE •Register read/write Register read/write DONEDONE

•DMA access DMA access DONEDONE

•Ethernet InterfaceEthernet Interface IN-PROGRESSIN-PROGRESS

•DDR2 Interface DDR2 Interface IN-PROGRESSIN-PROGRESS

•Linux driver Linux driver DONEDONE

•Optimised Disk StoreOptimised Disk Store

IN-PROGRESSIN-PROGRESS

Manager Software Manager Software IN-PROGRESSIN-PROGRESS

•Performance profilingPerformance profiling IN-PROGRESSIN-PROGRESS

•Clock and Control UplinkClock and Control Uplink NOT-STARTEDNOT-STARTED

EthernetInterface

DDR2Interface

PCIeInterface

Control/StatusReg.Block

InternalRAM

TestDataGen

Arbiter

Driver

Manager

Software

Firmware

22-Mar-2007

EUDET/CALICE DAQ Overview 7 HEP UCL

Detector Interface Detector Interface (Cam, IC)(Cam, IC)- Spec + hardwareSpec + hardware

DIF to Link/Data Aggregator DIF to Link/Data Aggregator (Cam/Man)(Cam/Man)- Spec + hardwareSpec + hardware

Data aggregate, format Data aggregate, format (Man)(Man)- Hardware + firmware Hardware + firmware

LDA to ODR opto-link LDA to ODR opto-link (Man, UCL)(Man, UCL)- Hardware + firmwareHardware + firmware

ODR ODR (RHUL, UCL, Cam)(RHUL, UCL, Cam)- firmwarefirmware

ODR to disk ODR to disk (RHUL)(RHUL)- Driver softwareDriver software

Local Software DAQ Local Software DAQ (RHUL)(RHUL)

Full blown Software DAQ Full blown Software DAQ (RHUL, UCL, [IC])(RHUL, UCL, [IC])

LDALDA

PCPC

ECALECALSlabSlab

DIFDIF

ODRODR

DriverDriver

OptoOpto

OptoOpto

UK Read-out work (ECAL UK Read-out work (ECAL FE)FE)

22-Mar-2007

EUDET/CALICE DAQ Overview 8 HEP UCL

Clock + Control, Clock + Control, IntegrationIntegration

Keep it simple!Keep it simple!-System synchronising signals distributed-System synchronising signals distributed-All data tagged with common ‘timestamp’All data tagged with common ‘timestamp’

Clock

Control (Train-start, Sync)

PCODRODR

PCODRODR

Fanout(TLU?)

LDA

Config

Slab

Slab

Slab

Config

LDASlab

Slab

Slab

Slave outMaster In

22-Mar-2007

EUDET/CALICE DAQ Overview 9 HEP UCL

‘‘TLU’ TLU’ RequirementsRequirements

Lets assume 32 ‘Slabs’.Lets assume 32 ‘Slabs’.-Each Slab needs: Clk; Train, (Trigger?)Each Slab needs: Clk; Train, (Trigger?)But could use LDA here. At 8 slabs/LDA = 4 LDAsBut could use LDA here. At 8 slabs/LDA = 4 LDAsWith minimum 1 LDA/ODR = 4 ODRsWith minimum 1 LDA/ODR = 4 ODRs-Each ODR needs: same + Fast Control(data)Each ODR needs: same + Fast Control(data)4 Signals Clock, Train, Trigger, Control4 Signals Clock, Train, Trigger, ControlFanout of 32 slabs + 4 LDA + 4 ODE =40, Fanout of 32 slabs + 4 LDA + 4 ODE =40, OR more likely 4 LDA + 4 ODR = 8.OR more likely 4 LDA + 4 ODR = 8. -TLU acts as Master or Slave for signallingTLU acts as Master or Slave for signalling-TLU generates signals stand-aloneTLU generates signals stand-alone-We presume to use LVDS everywhere.We presume to use LVDS everywhere.

?? Do we try to use the TLU for fanout, or just as a ?? Do we try to use the TLU for fanout, or just as a ‘machine’ interface?‘machine’ interface?

22-Mar-2007

EUDET/CALICE DAQ Overview 10

HEP UCL

Extra:Extra:

Optical SwitchOptical Switch

22-Mar-2007

EUDET/CALICE DAQ Overview 11

HEP UCL

Optical (Layer-1) Optical (Layer-1) SwitchingSwitching

Part of the UK CALICE is to evaluate the use of a “layer-1” switch. Part of the UK CALICE is to evaluate the use of a “layer-1” switch. 1) DAQ PC failover - Redirect data to spare unused DAQ PC on the 1) DAQ PC failover - Redirect data to spare unused DAQ PC on the

fly fly 2) “Router” - Can change data destination per bunch-train. 2) “Router” - Can change data destination per bunch-train.

Regulate load by sending data directly to free resources Regulate load by sending data directly to free resources 3) Programmable optical patch panel (large installation)3) Programmable optical patch panel (large installation)

Manufacturers offering similar products, in same price range e.g. Manufacturers offering similar products, in same price range e.g. Glimmerglass, Polatis - difficult to differentiate between themGlimmerglass, Polatis - difficult to differentiate between them

• Decided on PolatisDecided on Polatis- can switch dark fibre (i.e. not MEMS based)can switch dark fibre (i.e. not MEMS based)- Multimode fibre capableMultimode fibre capable- Fastest switching time (20ms)Fastest switching time (20ms)

• 16x16 array with 5016x16 array with 50μμm multimode LC connectorsm multimode LC connectors