daq/fles summary

DAQ/FLES SummaryDAQ/FLES Summary

Walter F.J. Müller, GSI, Darmstadt

CBM Collaboration Meeting16 April 2010

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 2

Scope of SummaryScope of Summary

Joint summary of DAQ aspects of

FEE/DAQ Workshop, 22/23 February at GSI

DAQ/FLES Working Group this CBM week

FEE/DAQ WSFEE/DAQ WS

DAQ/FLES WGDAQ/FLES WG

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 3

The CBM Generic Read-out The CBM Generic Read-out ChainChainDetectorFront-End

BoardRead-OutController

Active BufferBoard

FEB ROC ABB

Data CombinerBoard

DCB

First LevelEvent Selector

FLES

TNet

Data

Control

Data &Control

Sync

Data

DigitizationCustom ASIC

FEE DAQInterfaceLocal Pre-Processing

BufferingSystem

Synchronization

DAQFLESInterface

FPGA Coprocessor

Event SelectionCPU Farm

DAQDetector Sub-System FLES

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 4

Mission Statement FEE/DAQ Mission Statement FEE/DAQ WorkshopWorkshop

Focus is on system aspects of the final system Interim solutions, like n-XYTER, should only be covered

in a 'lessons learned from' context.

FEE/DAQ WSFEE/DAQ WS

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 5

Agenda FEE/DAQ WorkshopAgenda FEE/DAQ Workshop

Monday morning: TOF (PADI/GET4 readout) afternoon: TRD/MUCH ('Gas'-XYTER readout)

Tuesday morning: STS ('Si'-XYTER readout) afternoon: MVD/RICH

DAQ/FLES

Full program and presentations seehttp://cbm-wiki.gsi.de/cgi-bin/view/DAQ/Workshop-2010-02

FEE/DAQ WSFEE/DAQ WS

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 6

FLESFLES

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 7

Options for CBM FLES LocationOptions for CBM FLES Location

CBM CBM ServiceServiceBuildingBuilding

~350m

linear dist.

Distance is Distance is preliminary!!preliminary!!

'Green Cube''Green Cube'

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 8Slide: J. de CuvelandSlide: J. de Cuveland

DAQ/FLES WGDAQ/FLES WG

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 9Slide: J. de CuvelandSlide: J. de Cuveland

DAQ/FLES WGDAQ/FLES WG

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 10Slide: J. de CuvelandSlide: J. de Cuveland

DAQ/FLES WGDAQ/FLES WG

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 11Slide: J. de CuvelandSlide: J. de Cuveland

DAQ/FLES WGDAQ/FLES WG

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 12Slide: J. de CuvelandSlide: J. de Cuveland

DAQ/FLES WGDAQ/FLES WG

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 13Slide: J. de CuvelandSlide: J. de Cuveland

DAQ/FLES WGDAQ/FLES WG

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 14

DABCInfiniBand performance tests (2006-2007)

Slide: S. Linev Slide: S. Linev

DAQ/FLES WGDAQ/FLES WG

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 15

CBM RadiationCBM RadiationEnvironmentEnvironment

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 16

STS Plane 1 (30cm) – TID STS Plane 1 (30cm) – TID

FEE

Aux

no MUCH

at FEE: ~200krad/yrat Aux: ~50krad/yr

ProvisoProviso: preliminaryFLUKA calculations !

FEE/DAQ WSFEE/DAQ WS

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 17

STS Plane 8 (100cm) – fast STS Plane 8 (100cm) – fast hadron fluxhadron flux

FEE

with MUCH

2·103 h/cm2/s

2·104 h/cm2/s

2·105 h/cm2/s

at FEE: ~105 h/cm2/s

ProvisoProviso: preliminaryFLUKA calculations !

FEE/DAQ WSFEE/DAQ WS

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 18

Physical Location View – Feb Physical Location View – Feb 2222ndnd

FEB ROC/DCS ABBDCB FLES

TNet

Data

Control

Data &Control

Sync

Data

Cu

optical

Data

Control

ST

S,..

..T

OF

, ...?

ROC

ECS

on/near Detector CBM Service Bldg. GSI/FAIR IT Bldg.

Data

~50m~50m ~500m~500m

??

FEE/DAQ WSFEE/DAQ WS

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 19

Closer lookCloser lookat STS readoutat STS readout

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 20

STS Hit Rates per 128 Channel STS Hit Rates per 128 Channel ChipChip

Hit rate [MHz]

Sta0

Sta 1

Sta 2

Sta 3

Sta 4

Sta 5

Sta 6

Sta 7

Total

32-64 21 14 9 1 0 0 0 0 45

16-32 67 72 157 116 73 60 1 1 547

8-16 208 357 477 555 325 430 157 189 2698

4-8 494 521 250 440 964 785 1048

753 5255

2-4 441 376 51 24 219 294 693 1079

3177

1-2 104 51 0 0 3 5 21 90 274

0.5-1 1 1 0 0 0 4 0 0 6

Total 1336 1392 944 1136 1584 1578

1920

2112

12002

Very few chip in inner region have hit rate > 32 MHz >90% of chips have hit rates in 2...16 MHz range

Hit rate per chip StatisticsHit rate per chip StatisticsAu+Au @ 25 AGevAu+Au @ 25 AGev

101077 evt/sec evt/sec

FEE/DAQ WSFEE/DAQ WS

SIM-Data: A. KotyniaSIM-Data: A. Kotynia

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 21

STS QuadrantSTS Quadrant

6 cm6 cm5 cm5 cm

STS-FEBs

Is this enough space for: - 117 copper links - 10 opto converters - 10 ribbon fiber cablesplus other services like - power distribution - control & monitoring - ,,,,,,,

size of asize of a

post cardpost card

FEE/DAQ WSFEE/DAQ WS

CAD: S. Belogurov et alCAD: S. Belogurov et al

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 22

Compact electro-optics Converts Compact electro-optics Converts II

Optically enabled ASIC

Use MT connector:12 x TX

or 12 x RX or 4xTX and 4xRX

Optically enabled FPGA

Extracted from ReflexPhotonicsWhite Papers and Presentations

Claim for 12 channels:

4 cm2 footprint3 mm height

LightABLE from ReflexPhotonicsLightABLE from ReflexPhotonics

FEE/DAQ WSFEE/DAQ WS

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 23

Some Data on Fiber Ribbon Some Data on Fiber Ribbon CableCable Assume 12 fiber ribbon cable

very compact, 20 mil pitch cable cross section: 2.3 x 5.1 mm

Assume MPO/MPT connector

2.3

mm

5.1 mm

Pictures from GORE

FEE/DAQ WSFEE/DAQ WS

24Ulrich Brüning

STS Interface Converter STS Interface Converter BoardBoard

Proposal for STS interfacing2m copper twinax x1, x2, x4 or

x8copper => optical on Postcard

12.7cm x 8.9cmConversion to optical signals

with AOCs

Does electrical and AOC connector fit on one Postcard ?

Connector density analysis

AOCconnectors

twinaxconnectors

twinaxconnectors

Passive BoardPostcard

miniHTconnector

Slide: U. BrueningSlide: U. Bruening

FEE/DAQ WSFEE/DAQ WS

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 25

ProvisoProviso:link counts likely to go up to

account for more background,

overhead, head room,....

All STS data can be transported by ~280 ribbon cables Corresponds to 34 cm2 cable cross section

STS Link & Ribbon Cable STS Link & Ribbon Cable DistributionDistribution

Sta0

Sta 1

Sta 2

Sta 3

Sta 4

Sta 5

Sta 6

Sta 7

total

FEBs 168 176 120 144 200 200 240 264 1512

links 361 417 443 459 425 426 347 336 3214

cables/quadrant 8 9 10 10 9 9 8 7

cables tot 32 36 40 40 36 36 32 28 280

FEB&Cable StatisticsFEB&Cable StatisticsAu+Au @ 25 AGevAu+Au @ 25 AGev

101077 evt/sec evt/sec48 bit message48 bit message

FEE/DAQ WSFEE/DAQ WS

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 26

Where to locateWhere to locatefast serializers fast serializers

andandopto-converters ?opto-converters ?

1212thth – 16 – 16thth April, 2010 April, 2010 1515thth CBM Collaboration Meeting CBM Collaboration Meeting 27

Data Aggregation on separate chip - Data Aggregation on separate chip - Communication hubCommunication hub Motivation:

Data Aggregation: In most cases 2,4, or 8 CBM-XYTER chips (FEBa2, FEBa4, FEBa8, resp.) can be aggregated to fill a 2.5 Gbps link

Increase the bandwidth available per link Reduce the number of Optical Links

Schemes: Embedded (aggregation integrated in CBM-XYTER)

Data aggregation and Traffic Management with a separate Communication Controller Asic

Communication ‘hub’

Clock Distribution Slow control traffic Data Readout traffic

6

DAQ/FLES WGDAQ/FLES WG

Slide: P. BanerjeeSlide: P. Banerjee

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 28

All Combined – Detector & All Combined – Detector & Service Bldg.Service Bldg.

Data

Clock &Control

CBM Service Bldg.

DCB

DCS

on/near Detector

Clock &Sync

febfeb hubhub optoopto

TNetCntlNet

Con

trol

Data

DAQ/FLES WGDAQ/FLES WG

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 29

STS Quadrant – A possible STS Quadrant – A possible arrangementarrangement

6 cm6 cm5 cm5 cm

STS-FEBs

STS-'hub' STS-'opto'

DAQ/FLES WGDAQ/FLES WG

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 30

DCSDCS

DCB

DCS

DCB

DCS

Physical Location View – April Physical Location View – April 1313thth

FEB ABB

DCB

FLES

Data

Control

Clock &Data &Control

Clock &

Sync

Data

ST

S,..

..T

OF

, ...?

ROC

ECS

Cu

optical

on/near Detector CBM Service Bldg. 'Green Cube'

Data

~50m~50m ~500m~500m

Data

Control

'hub' 'opto'Data

Clock & Control

DCB

DCS

CLOSY

DCS

TNet

CntlNet

Clock & Sync

Contro

l

Con

trol

DAQ/FLES WGDAQ/FLES WG

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 31

DCSDCS

DCSDCS

Physical Location View – April Physical Location View – April 1515thth

FEB ABB

DCB

FLES

Data

Control

Clock &Data &Control

Clock &

Sync

Data

ST

S,..

..T

OF

, ...?

ROC

ECS

Cu

optical

on/near Detector CBM Service Bldg. 'Green Cube'

Data

~50m~50m ~500m~500m

Data& Control

clock

'hub' 'opto'

CLOSY

DCS

TNet

CntlNet

Clock & Sync

Contro

l

Con

trol

DPB?

DCS

Clock &Data &Control

DAQ/FLES WGDAQ/FLES WG

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 32

ROC next GenROC next Gen

33

Basic Components on FPGA Module

Xilinx FPGA (Spartan, Virtex)

RAM

Actel Configuration Controller

Flash EEPROM

Voltage Regulators (1.2, 1.5, ...) driven by 3.3 V or 5V

RS232 Interface

1 or 2 SFPs

Jitter Cleaner

FMC-HPC (Vita57) connector with (160 or) 400 pins

Slide: D. GottschalkSlide: D. Gottschalk

FEE/DAQ WSFEE/DAQ WS

34

SysCore 3 Base Board Example

32 RJ45 Connectorswith LVDS

48Volt Power Input

Isolated JTAG forHigh Voltage Applications

External Ref. ClockInputs (LVDS??)

Gigabit LinksSFPs or electrically

USB + GBit Ethernet

Slide: D. GottschalkSlide: D. Gottschalk

FEE/DAQ WSFEE/DAQ WS

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 35

ControlsControls

13.04.201013.04.2010 Burkhard Kolb GSIBurkhard Kolb GSI 3636

Test beam time exampleTest beam time example

Burkhard Kolb GSIBurkhard Kolb GSI 3636

DAQ/FLES WGDAQ/FLES WG

Slide: B. KolbSlide: B. Kolb

13.04.201013.04.2010 Burkhard Kolb GSIBurkhard Kolb GSI 3737Burkhard Kolb GSIBurkhard Kolb GSI 3737

Requirements DBRequirements DB

In collaboration with PANDA we have set up a database with web access

http://nuclear.gla.ac.uk/CBM-DCS/RegisterThen enter requirements:

Process variablesCommon hardware…

In collaboration with PANDA we have set up a database with web access

http://nuclear.gla.ac.uk/CBM-DCS/RegisterThen enter requirements:

Process variablesCommon hardware…

DAQ/FLES WGDAQ/FLES WG

Slide: B. KolbSlide: B. Kolb

16 Apr 2010 CBM Collaboration Meeting -- Walter F.J. Müller, GSI 38

The EndThe End

Thanks for Thanks for your attentionyour attention