d ata a cquisition b ackbone c ore dabc/dabc/dabc.php 27.09.07 j.adamczewski, h.g.essel, n.kurz,...
TRANSCRIPT
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
1Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
Work supported by EU RP6 project JRA1 FutureDAQ RII3-CT-2004-506078
Data Acquisition Backbone CoreJ.Adamczewski, H.G.Essel, N.Kurz, S.LinevGSI, Experiment Electronics, Data Processing group
• Motivation• Data-flow engine, control• Event building network• Performance• To do
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
2Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
CBM data acquisition
data dispatcher
FEEdeliver time stamped
data
CNetcollect data into buffers
Detector Detector Detector
collect
~50000 FEE chips
event dispatcher
switching networkBNetsort time stamped data
~1000 links a 1 GB/sec
HNethigh level selection
to high level computing
and archiving~1 GB/sec Output
processing
PNetprocess events
level 1&2 selection
subfarm subfarm subfarm ~100 subfarms
~100 nodes per subfarm
~10 dispatchers → subfarm
~1000 collectors
~1000 active buffers
TNettime distribution
W.F.J.Müller, 2004
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
4Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
PCIe
FEFEFront end board: sampling ADCs, clock distribution
Active Buffer board *: PCI express card
PC
ABBPCIe
2.5 GBit/s bi-directional (optical) link: data, clock
Use case example: Frontend components test
The goal:
• Detector tests• FEE tests• Data flow tests
FE: Frontend board
ABB: Active Buffer board
* A.Kugel, G.Marcus, W.Gao, Mannheim University Informatics V
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
5Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
GE switch
PC
ABBPCIe
DCB
FE
IB switch
FE
DC
Front end board: sampling ADCs, clock distribution
Data combiner boards, clock distribution to FE
Active Buffer board: PCI express card
8-20 PCs dual/quad
PC
ABBPCIe
8
4
Scales up to 10k channels, 160 CPUs
~625 Mb/s bi-directional (optical) link: data, clock
~2.5 Gb/s data linksThe goal:• Investigate critical technology• Detector tests• Replace existing DAQ
FE: Frontend boardDC: Data combiner boardABB: Active Buffer boardGE: Gigabit EthernetIB: InfiniBandMBS: Multi Branch System
MBS
Use case example: middle-size setup
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
6Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
Driving forces and motivation for DABC
2004 → EU RP6 project JRA1 FutureDAQ*2004 → CBM FutureDAQ for FAIR2005 → FOPI DAQ upgrade (skipped)2007 → NUSTAR DAQ
* RII3-CT-2004-506078
1996 → MBS future 50 installations at GSI, 50 external http://daq.gsi.de
• Detector tests• FE equipment tests• Data transport• Time distribution• Switched event building• Software evaluation• MBS event builder• General purpose DAQ
Data Acquisition Backbone Core
Intermediatedemonstrator
Requirements• connect (nearly) any front-ends• handle triggered or self-trigger front-ends• process time stamped data streams• build events over fast networks• provide data flow control (to front-ends)• interfaces to plug in application codes• connect MBS readout or collector nodes• be controllable by several controls frameworks
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
7Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
DABC key components
• Data-flow engine– memory and buffers management– threads and events management– data processing modules with ports, parameters, timers– transport and device classes, file I/O– back pressure mechanism
• Slow control and configuration– components setup on each node– parameters monitoring/changing– commands execution– state machine logic– user interface
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
8Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
Data flow engine
DABC Module
port
port
DABC Module
port
port
process process
Local transport
A module processes data of one or several data streams.Data streams propagated through ports, which are connected by transports
Queue
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
9Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
Data flow engine
DABC Module
port
port
DABC Module
port
port
process process
Device
Net transport
Device
Net transport
Queue Queue
Network
A module processes data of one or several data streams.Data streams propagated through ports, which are connected by transports and devices
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
10Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
Memory management
Main features:• All memory to be used in modules for transport organized in memory pools• Memory pool consists of one or several blocks of memory, divided on equal
peaces - buffers• Each buffer in memory pool can be referenced once for writing and any times
for reading• Several references may be combined in gather list
Use for transport:• Only data from memory pools can be transported via ports• Each module port associated with only memory pool• Transport between two modules in same application done via pointer• Zero-copy network transport where supported (InfiniBand)• Support of gather lists for all kind of transports
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
11Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
Threads and event management
DABC Module A
processInputprocessCommand
Event manager
Thread with modulesDevice thread
Commands are synchronized with thedata flow also through the Event manager
Data ready event
A device, i.e. socket device controls several ports (transports).Once a queue buffer is filled, the transport signals the Event manager,which in turn calls the processInput function of the associated module.
Transport
QueueDABC Module B
Manager thread
CommandCommand
event
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
12Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
Back pressure mechanism
Basic idea:• sender allowed to send packets only after receiver confirms (with special
acknowledge message) that it has enough resources to receive these packets • Implemented on transport layer (not visible for user)
Impact on module code:• No additional efforts is required• Can be enabled/disabled for any port• To block connection - just do not read packets from it
Pro: easy method for traffic control in small system
Con: easy way to block complete network when single node is hanging
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
13Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
Class diagram as currently implemented
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .10. .1
0. .1
0. .10. .1
0. .1
0. .1 0. .1
0. .1
0. .10. .1
0. .1 0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .10. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. . *
0. . 1
0. .1
0. .1 0. .10. .1
0. .1
0. .10. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
0. .1
dabc::CommandsQueue
#fQ ueue: stl _cmd_queue*#fRepl yQ ueue:bool#fCmdsMutex:Mutex*
+CommandsQueue+~CommandsQ ueue+Push:voi d+Pop:Command*+Front:Command*+Si ze:unsi gned+Cl eanup:voi d
dabc::StandaloneCommandClient
#fCmdsMutex:Mutex*#fNumSumbCommands: i nt#fRepl yCond:Condi ti on*#fRepl yedCmds:CommandsQ ueue
+Standal oneCommandCl i ent+~Standal oneCommandCl i ent+Wai tCommands:bool#Wai tCommandRepl y:bool#PostCommandProcess:bool#ProcessRepl y:bool#AttachCmd:voi d#D ettachCmd:voi d#I sCl i entWorki ng:bool
dabc::CommandThrdConnect
+CommandThrdConnect
dabc::MemoryPool
#fBuffer:voi d *#fTotal Si ze:ui nt64_t#fNumBuffers:BufferI d_t#fBufferSi ze:BufferSi ze_t#fD efaul tO ffset:BufferSi ze_t#fUsage:ui nt32_t*#fSeekCnt:BufferI d_t#fNoFreeBuffers:bool#fPool Mutex:Mutex*#fO wnMutex:bool
+MemoryPool+~MemoryPool+Al l ocate:bool+UseMutex:voi d+Rel ease:voi d+G etTotal Buffer:voi d *+G etTotal Si ze:ui nt64_t+G etNumBuffers:BufferI d_t+G etBufferSi ze:BufferSi ze_t+G etBufferLocati on:voi d *+TakeBuffer:bool+Unl ockBuffer:bool+NewReference:bool+Rel easeBuffer:voi d+I sBufferUsed:bool+TakeMemoryBuffer:MemoryBuffer*+CreateSegmentedBuffer:SegmentedBuffer*+I sFreeBuffers:bool
EBuffersUsage
dabc::ModuleItem
#fModul e:Modul e*#fI temType: i nt#fI temI d: i nt#fPri ori ty: i nt
+Modul eI tem+~Modul eI tem+G etModul e:Modul e*+G etType: i nt+I temI d: i nt+G etPri ori ty: i nt#SetI temI d:voi d#SetPri ori ty:voi d#Fi reEvent:voi d#D oStart:voi d#D oStop:voi d
dabc::CommandDirectConnect
+CommandD i rectConnect
dabc::A ctionListner
+Acti onLi stner+~Acti onLi stner+ProcessActi on:voi d
enumdabc::EModuleEvents
+evntNone: i nt+evntI nput: i nt+evntO utput: i nt+evntPool : i nt+evntCommand: i nt+evntRepl y: i nt+evntTi meout: i nt
dabc::CommandPortConnect
+CommandPortConnect
dabc::ModulesThread
- fQ ueues:EventsI dQ ueue*-fCondi ti on:Condi ti on*-fNumQ ueues: i nt-fModul es:Poi ntersVector-fWorki ng:bool-fCheckTi mers:bool-fTi mers:Poi ntersVector-fTi me:Ti meSource*-fSysCommands:CommandsQ ueue*-fSysMutex:Mutex-fSysCondi ti on:Condi ti on-fExpl i ci tLoop:bool
+Modul esThread+~Modul esThread+AddModul e:voi d+RemoveModul e:voi d+StartModul e:voi d+StopModul e:voi d+SysCommand:voi d+Fi re:voi d+Wai t:EventI d+TestWorki ng:bool+ThrdTi meStamp:doubl e#Worki ngLoop:voi d#D oWakeup:bool#ProcessThreadEvent:voi d#ProcessThreadCommand:bool#D efineNextTi meout: l ong#Fi reModul eCommand:voi d#UpdateModul eD ependentLi sts:voi d
dabc::CommandA llModules
+CommandAl l Modul es
dabc::Timer
#fMutex:Mutex*#fTi meout:doubl e#fLastFi re:doubl e#fTi meoutFi red:bool#fCounter: l ong#fSi ngl eShoot:doubl e
+Ti mer+~Ti mer+G etTi meout:doubl e+D efineNextTi meout: l ong+G etCounter: l ong#Fi reTi meout:voi d#Si ngl eShoot:bool#Compl eteTi meout:bool#D oStart:voi d#D oStop:voi d
dabc::MemoryBuffer
#fI d:BufferI d_t#fBuffer:voi d *#fFul l Si ze:BufferSi ze_t#fO ffset:BufferSi ze_t#fD ataSi ze:BufferSi ze_t
#MemoryBuffer#~MemoryBuffer+G etI d:BufferI d_t+G etD ataLocati on:voi d *+G etD ataSi ze:BufferSi ze_t+G etO ffset:BufferSi ze_t+SetD ataSi ze:voi d+G etMaxD ataSi ze:BufferSi ze_t+MakeReadO nl y:bool+NewReference:MemoryBuffer*+CopyFrom:voi d+G etTotal Si ze:BufferSi ze_t
dabc::Folder
- fChi l ds:Poi ntersVector-fO wner:bool
+Fol der+~Fol der+AddChi l d:voi d+RemoveChi l d:voi d+D el eteAl l Chi l ds:voi d+NumChi l ds: i nt+G etChi l d:Basi c*+Fi ndChi l d:Basi c*+I sChi l d:bool+I sO wner:bool#G etFol der:Fol der*-Chi l dCreated:voi d-Chi l dD estroyed:voi d
dabc::Iterator
#fTop:Basi c*#fCurrent:Basi c*#fI ndexes:std: :vector<i nt>#fFol ders:std: :vector<dabc: : Fol der*>#fFul l Name:Stri ng
+I terator+~I terator+next:Basi c*+current:Basi c*+l evel : i nt+ful l name:const char *+name:const char *+Pri ntH i eararchy:voi d
dabc::CommandCl ient
+CommandCl i ent+~CommandCl i ent#ProcessRepl y:bool#AttachCmd:voi d#D ettachCmd:voi d
enumdabc::EModuleItemType
+mi tPort: i nt+mi tPool : i nt+mi tParam: i nt+mi tTi mer: i nt+mi tCmdQ ueue: i nt
dabc::CommandStartModule
+CommandStartModul e
dabc::CommandReceiver
+CommandRecei ver+~CommandRecei ver+Submi tCommand:bool
dabc::CommandCreateModule
+CommandCreateModul e
dabc::Condition
#fI nternCondMutex:Mutex#fCondMutex:Mutex*#fCond:pthread_cond_t#fFi redCounter: l ong i nt#fWai ti ng:bool
+Condi ti on+~Condi ti on+D oFi re:voi d+D oWai t:bool+Reset: voi d+_D oFi re:voi d+_D oWai t:bool+CondMutex:Mutex*
dabc::A ctionSource
- fLi stners:Poi ntersVector-fMutex:Mutex*
+Acti onSource+~Acti onSource+AddLi stner:voi d+RemoveLi stner: voi d+D oActi on:voi d
enumdabc::EModuleThreadEvents
+mthrtevntH al t: i nt+mthrtevntCmd: i nt+mthrtevntExcl : i nt+mthrtevntCl ean: i nt
dabc::VerbsDevice
#fI bPort:ui nt8_t#fContext: i bv_context *#fPD : i bv_pd *#fD evi ceAttr: i bv_devi ce_attr#fPortAttr: i bv_port_attr#f_mtu:ui nt16_t#fO sm:VerbsO sm*#fRecvThrd:VerbsRecvThread*#fConnThrd:VerbsConnThrd*#fVerbsMutex:Mutex#fChannel : i bv_comp_channel *#fMai nCQ :VerbsCQ *
+VerbsD evi ce+~VerbsD evi ce+I sMul ti cast:bool+I bPort: i nt+context+pd+l i d:ui nt16_t+mtu:ui nt16_t+G etG i dI ndex: i nt+CreateAH+Regi sterMul ti CastG roup:bool+UnRegi sterMul ti CastG roup:bool+CreateMAH+ServerConnect:bool+Cl i entConnect:bool+Fi l l ConnServerI d:bool#O penVerbs:bool#Cl oseVerbs:bool#CreatePortQP:VerbsQ P*#CreateTransport:voi d
dabc::VerbsTransport
#fVerbs:VerbsD evi ce*#fQ P:VerbsQP*#f_mr: i bv_mr *#f_rwr: i bv_recv_wr *#f_swr: i bv_send_wr *#f_sge: i bv_sge *#fH eadersPool :VerbsMemoryPool *#fSegmPerO per: i nt
#_Submi tSend:voi d#_Submi tRecv:voi d#G etRecH eader:voi d *+VerbsTransport+~VerbsTransport+MaxSendSegments: i nt
std: :vector<NetworkTransport*>dabc::NetTransportVector
bnet::B netManagerPlugin
+BnetManagerPl ugi n+CreateModul e:dabc: :Modul e*+CreateSubeventsG eneratorWorker:SubeventsG eneratorWorker*+CreateSubeventsCombi nerWorker:SubeventsCombi nerWorker*+CreateSenderWorker:SenderWorker*+CreateEventBui l derWorker:EventBui l derWorker*+ReadoutBufferSi ze:ui nt64_t+ReadoutPool Name:const char *+TransportBufferSi ze:ui nt64_t+TransportPool Name:const char *+EventBufferSi ze:ui nt64_t+EventPool Name:const char *+Control BufferSi ze:ui nt64_t+Control Pool Name:const char *
bnet::EventBui lderModule
#fI npPool :dabc: : Pool Envel ope*#fO utPool :dabc: :Pool Envel ope*#fNumSenders: i nt#fWorker:EventBui l derWorker*
+EventBui l derModul e+NumSenders: i nt+ProcessEvent:voi d
bnet::EventBui lderWorker
+EventBui l derWorker+AddPacket: voi d+NumPackets: i nt+I sCompl eteEvent:bool+ProduceEvent:bool
bnet::EventFi lterModule
#fI npPool :dabc: : Pool Envel ope*
+EventFi l terModul e+ProcessI nputEvent:voi d
bnet::GlobalDFCModule
#fNumNodes: i nt
+G l obal D FCModul e+NumNodes: i nt
bnet::LocalDFCModule
#fNodeI d: i nt
+Local D FCModul e+NodeI d: i nt
bnet::ReceiverModule
#fNumSenders: i nt#fPool :dabc: :Pool Envel ope*#fRecvRate:dabc: :Ratemeter#fLoopCounter: i nt
+Recei verModul e+ProcessEvent:voi d+AfterModul eStop:voi d+NumSenders: i nt
bnet::SenderModule
#fNumRecei vers: i nt#fPool :dabc: :Pool Envel ope*#fWorker:SenderWorker*#fTranportBufferSi ze: i nt#fSendRate:dabc: :Ratemeter
+SenderModul e+ProcessEvent:voi d+NumRecei vers: i nt
bnet::SenderWorker
+SenderWorker+D efineTargetNode: i nt
bnet::SubeventCombinerModule
#fNumReadout: i nt#fI npPool :dabc: : Pool Envel ope*#fO utPool :dabc: :Pool Envel ope*#fWorker:SubeventsCombi nerWorker*
+SubeventCombi nerModul e+ProcessEvent:voi d+NumReadout: i nt
bnet::SubeventsCombinerWorker
+SubeventsCombi nerWorker+AddPacket: voi d+NumPackets: i nt+I sCompl eteSubevent:bool+ProduceSubevent:boolbnet::SubeventsGeneratorWorker
+SubeventsG eneratorWorker+G enerate:bool
dabc::CommandStopModule
+CommandStopModul e
std: :excepti ondabc::Exception
#fWhat:std: :stri ng
+Excepti on+Excepti on+~Excepti on+what: const char *
dabc::SegmentedBuffer
#fCapaci ty:unsi gned#fNumSegments:unsi gned#fSegments:Segment*
#Rel l ocate:voi d#SegmentedBuffer#~SegmentedBuffer+AddSegment:bool+AddSegment:bool+G etTotal Si ze:BufferSi ze_t+NumSegements:unsi gned+G etI d:BufferI d_t+G etD ataLocati on:voi d *+G etD ataSi ze:BufferSi ze_t+G etO ffset:BufferSi ze_t
Segment
dabc::TimeSyncModule
#fMasterConn:bool#fNumSl aves: i nt#fSyncTi mes:doubl e *#fPool Name:Stri ng#fTi meSource:Ti meSource*#fWorkBuf:MemoryBuffer*#fCurrCmd:Command*#fSl aveCnt: i nt#fPktCnt: i nt#fLoopTi me:Average#fMasterToSl ave:Average#fSl aveToMaster:Average#fSetShi f t: doubl e#fSetScal e:doubl e
#Fi l l MasterPacket:bool#Fi l l Sl avePacket:bool#ProcessFastRecv:bool#ExecuteCommand:bool#ProcessTi merEvent:voi d#BeforeModul eStart: voi d#AfterModul eStop:voi d+Ti meSyncModul e+~Ti meSyncModul e
dabc::CommandChannelModule
#fPool : Pool Envel ope*#fI sMaster:bool#fNumSl aves: i nt#fCmdO utQ ueue:CommandsQ ueue*
+CommandChannel Modul e+~CommandChannel Modul e+I sMaster:bool+NumSl aves: i nt+G etPool :MemoryPool *+ExecuteCommand:bool+ProcessI nputEvent:voi d+ProcessO utputEvent:voi d#SendSubmi ttedCommands:voi d
dabc::StandaloneManager
#fNodeI d: i nt#fNumNodes: i nt#fCmdChannel :CommandChannel Modul e*#fCmdThread:Modul esThread*
+Standal oneManager+~Standal oneManager+NodeI d: i nt+NumNodes: i nt+ConnectCmdChannel : voi d+Submi tRemoteCommand:bool+H al tAl l Nodes:voi d+ParameterEvent:voi d#D efineNodeI d: i nt#D efineNodeName:std: : stri ng#D efineNumNodes: i nt#Val i dNodeName:bool#CanSendCmdToManager:bool#SendRemoteCommand:bool#SendRemoteRepl y:bool#D oCommandRecv:voi d
dabc::TimeSource
#fCl ockShi f t:cycl es_t#fTi meScal e:doubl e#fTi meShi f t:doubl e
+Ti meSource+Ti meSource+operator=:Ti meSource&+~Ti meSource+G etTi meStamp:doubl e+Cal cTi meStamp:doubl e+SetCal i br: voi d+Adj ustTi meStamp:voi d+Adj ustCal i br: voi d
dabc::CpuStatistic
#system1:unsi gned l ong#system2:unsi gned l ong#user1:unsi gned l ong#user2:unsi gned l ong#i dl e1:unsi gned l ong#i dl e2:unsi gned l ong#fStatFp:FI LE*
+CpuStati sti c+~CpuStati sti c+Reset:bool+Measure:bool+CPUuti l : doubl e#G et:bool
dabc::PoolEnvelope
#fPool :MemoryPool *#fRequi redNumber:ui nt32_t#fRequi redSi ze:ui nt64_t
+Pool Envel ope+~Pool Envel ope+SetRequi rements:voi d+I ncrementRequi rements:voi d+G etRequi rements:bool+I sPool Assi gned:bool+Assi gnPool :voi d+D i sconnectPool :voi d+TakeBuffer:MemoryBuffer*+ProcessActi on:voi d
pool :MemoryPool *
dabc::ManagerPlugin
+ManagerPl ugi n+~ManagerPl ugi n+CreateModul e:Modul e*
dabc::ManagerThread
#fManager:Manager*#fCondi ti on:Condi ti on#fEvents:EventsI dQ ueue
+ManagerThread+~ManagerThread+Worki ngLoop:voi d+D oWakeup:bool+Fi reEvent:voi d
dabc::EventsIdQueue
#fQ ueue: stl _eventsi d_queue*
+EventsI dQ ueue+~EventsI dQueue+Push:voi d+Pop:EventI d+Si ze:unsi gned
dabc::Module
#fModul eThrd:Modul esThread*#fWorki ng: i nt#fI tems:Poi ntersVector#fI nputPorts:PortsI ndexVector*#fO utputPorts:PortsI ndexVector*#fPorts:PortsI ndexVector*#fCommands:Modul eCommandsQ ueue*#fRepl yes:Modul eCommandsQ ueue*#fO wnMai nLoop:bool#fModul eI d: i nt
#i ni t:voi d+Modul e+Modul e+~Modul e+Assi gnToThread:voi d+G etModul eThread:Modul esThread*+Start: voi d+Stop:voi d+Submi tCommand:bool+NumI nputs: i nt+NumO utputs: i nt+NumPorts: i nt+I nput:Port*+O utput:Port*+I O Port: Port*+Fi ndParam:Parameter*+SetParamStr:bool+SetParamD oubl e:bool+G etPool sFol der:Fol der*+G etPortsFol der:Fol der*+G etO bj Fol der: Fol der*+G etParsFol der: Fol der*+G etTi mersFol der:Fol der*+G etI tem:Modul eI tem*+TakeBuffer:MemoryBuffer*#I sWorki ng:bool#I sO wnMai nLoop:bool#SetO wnMai nLoop:voi d#Mai nLoop:voi d#TestWorki ng:bool#BeforeModul eStart: voi d#AfterModul eStop:voi d#ProcessEvent:voi d#ProcessI nputEvent:voi d#ProcessO utputEvent:voi d#ProcessPool Event:voi d#ProcessTi merEvent:voi d#ProcessUserEvent:voi d#ProcessFastRecv:bool#ExecuteCommand:bool#Repl yCommand:bool#Assi gnTransport:voi d#CreatePool :Pool Envel ope*#CreateI nput: i nt#CreateO utput: i nt#CreatePort: i nt#CreateParameter:Parameter*#CreateTi mer:Ti mer*#Fi ndTi mer:Ti mer*#ShootTi mer:bool#G etPortI tem:Port*#I temCreated:voi d#I temD estroyed:voi d#ProcessRepl y:bool#ThrdTi meStamp:doubl e#G etThrdTi meSource:Ti meSource*-Modul eMai nLoop:voi d-Fi reEvent:voi d-PreprocessEvent:bool-D oTransportAssi gn:voi d-D oStop:voi d-D oStart: voi d
dabc::ObjectsQueue
#fQ ueue: stl _obj s_queue*#fI sO wner:bool#fQ ueueMutex:Mutex*
+O bj ectsQ ueue+~O bj ectsQueue+Push:voi d+Pop:Basi c*+Si ze:unsi gned
T: cl ass
dabc::CleanupEnvelope
#fO bj :T*
+Cl eanupEnvel ope+~Cl eanupEnvel ope
dabc::B asic
- fParent:Basi c*-fName:Stri ng-gNumI nstances: l ong
+Basi c+~Basi c+G etParent:Basi c*+MakeFul l Name:voi d+G etFul l Name:Stri ng+G etName:const char *+I sName:bool+Fi l l I nfo:voi d+G etManager:Manager*+NumI nstances: l ong#SetName:voi d#D el eteThi s:voi d-Chi l dCreated:voi d-Chi l dD estroyed:voi d
dabc::Runnable
+Mai nLoop:voi d *+~Runnabl e
dabc::Thread
#fThrd:pthread_t
+Thread+~Thread+Start: voi d+J oi n:voi d+Ki l l :voi d+SetPri ori ty:voi d+I sI tsel f :bool dabc::RunnableThread
- fWorkStatus:EWorkStatus-fO wnMutex:Mutex-fStartCond:Condi ti on*-fSyncCond:Condi ti on*-fWai ti ngJ oi n:bool-fWorkMutex:Mutex*
-Mai nLoop:voi d *#Worki ngLoop:voi d#D oWakeup:bool#SetWorkMutex:voi d#G etWorkMutex:Mutex*#_I sWorki ng:bool#ConfirmWakeup:voi d+Runnabl eThread+~Runnabl eThread+I sWorki ng:bool+G o:bool+Stop:voi d+D oThreadSync:bool+Wai tForJ oi n:bool+SetThreadPri ori ty:voi d
EWorkStatus
dabc::PointersVector
#fVector: stl _voi dstar_vector*
+Poi ntersVector+~Poi ntersVector+push_back:voi d+pop_back:voi d *+i ndex_of : i nt+remove:bool+remove:bool+at:voi d *+si ze: i nt+operator[]:voi d * &+cl ear: voi d
dabc::Manager
#fWorki ng:bool#fCmdThread:ManagerThread*#fUseThreadForRepl y:bool#fCommandsQ ueue:CommandsQ ueue#fRepl yesQ ueue:CommandsQ ueue#fCl eanupQ ueue:O bj ectsQ ueue#fSendCmdsMutex:Mutex*#fSendCmdCounter: i nt#fSendCommands:Poi ntersVector
#Manager#~Manager#ExtractManagerName:const char *+SetUseThreadForRepl y:voi d+Wai tCmdThreadJ oi n:bool+H al tManager:voi d+G etPl ugi nFol der: Fol der*+G etD evi ceFol der:Fol der*+G etThreadsFol der:Fol der*+G etModul esFol der: Fol der*+G etPool sFol der:Fol der*+G etConnFol der: Fol der*+Fi ndModul e:Modul e*+Fi ndPort:Port*+Fi ndNonconnectedPort: Port*+Fi ndD evi ce:G eneri cD evi ce*+Fi ndPool :MemoryPool *+CreateMemoryPool s:voi d+Submi tCommand:bool+Submi tCommand:bool+Modul eExecpti on:voi d+Sampl ePacket:voi d+ParameterEvent:voi d+StartModul e:voi d+StartAl l Modul es:voi d+D el eteO bj ect: voi d+D el eteAnyObj ect:voi d+Cl eanupManager:voi d+D el eteAl l Modul es:voi d+G etThread:Modul esThread*#ProcessRepl y:bool#ExecuteCommand:bool#ExecuteSpeci al Command:bool#ExecuteManagerCommand:bool#CreateStandardModul e:Modul e*#PostCommandProcess:bool#ProcessEvent:voi d#I sCl i entWorki ng:bool#Val i dNodeName:bool#CanSendCmdToManager:bool#SendRemoteCommand:bool#SendRemoteRepl y:bool#RecvRemoteCommand:voi d#RecvRemoteRepl y:voi d
MgrEvents
enumdabc::EMemoryBufferType
+mbt_G eneri c: i nt+mbt_I nt64: i nt+mbt_TymeSync: i nt+mbt_AcknCounter: i nt+mbt_User: i nt
dabc::B uffer
#fPool :MemoryPool *#fReadO nl y:bool#fTypeI d:ui nt32_t#fUserI d:ui nt32_t#gNumI nstances: l ong
#Buffer#~Buffer#SetReadO nl yFl ag:voi d+G etPool :MemoryPool *+I sReadO nl y:bool+SetTypeI d:voi d+G etTypeI d:ui nt32_t+SetUserI d:voi d+G etUserI d:ui nt32_t+G etTotal Si ze:BufferSi ze_t+Rel ease:voi d+NumI nstances: l ong
dabc::SocketConnThread
#fMai n:SocketD evi ce*#fServerH ost:Stri ng#fServerPort: i nt#fServerSocket: i nt#fConnCounter: i nt#fMutex:Mutex#fConnRecs:Poi ntersVector
+SocketConnThread+~SocketConnThread+TryServerPort:bool+ServerH ost:const char *+ServerPort: i nt+Worki ngLoop:voi d+D oWakeup:bool+StartServerConnect:bool+StartCl i entConnect:bool#ServerWorki ngLoop:voi d#Cl i entWorki ngLoop:voi d#Cl eanupTi meout:bool
SocketConnRec
dabc::SocketRecvThread
#fMai n:SocketD evi ce*#fWakeupMutex:Mutex#fWakeupCounter: i nt#fWakeupPi pe: i nt [2]
#Worki ngLoop:voi d#D oWakeup:bool+SocketRecvThread+~SocketRecvThread
dabc::SocketTransport
#fSocket: i nt#fSocketRecvQ ueue:SocketRecvQ ueue*
#_Submi tSend:voi d#_Submi tRecv:voi d#RecvD ataFromSocket: i nt#SendD ataO verSocket:bool+SocketTransport+~SocketTransport+MaxSendSegments: i nt
dabc::SocketSendThread
#fQ ueue:EventsI dQ ueue#fCondi ti on:Condi ti on*
#Worki ngLoop:voi d#D oWakeup:bool#Fi reEvent:voi d+SocketSendThread+~SocketSendThread
dabc::IntParameter
#fVal ue: i nt
+I ntParameter+Ki nd:EParamKi nd+G etPtr: voi d *+G etStr:bool+SetStr:bool+SetI nt:bool+G etI nt: i nt+G etD oubl e:doubl e+SetD oubl e:bool
dabc::DoubleParameter
#fVal ue:doubl e
+D oubl eParameter+Ki nd:EParamKi nd+G etPtr: voi d *+G etStr:bool+SetStr:bool+SetD oubl e:bool+G etD oubl e:doubl e
dabc::Parameter
#fFi xed:bool
+Parameter+~Parameter+Ki nd:EParamKi nd+G etPtr: voi d *+I sFi xed:bool+SetFi xed:voi d+G etStr:bool+SetStr:bool+G etD oubl e:doubl e+SetD oubl e:bool+Fi l l I nfo:voi d#Changed:voi d#Rai seEvent:voi d
dabc::StrParameter
#fVal ue:Stri ng
+StrParameter+Ki nd:EParamKi nd+G etPtr: voi d *+G etStr:bool+SetStr:bool+G etD oubl e:doubl e+SetD oubl e:bool
enumdabc::EParamK ind
+parNone: i nt+parStri ng: i nt+parD oubl e: i nt+parI nt: i nt
dabc::LocalTransport
#fPai r: Local TransportPai r*#fO ther: Local Transport*#fRecvQ ueue:Local TransportRecvQ ueue*
#PortChanged:voi d+Local Transport+~Local Transport+Provi desI nput:bool+Provi desO utput:bool+Recv:voi d+RecvQ ueueSi ze: i nt+Send:voi d+SendQ ueueSi ze: i nt+D i rectSend:voi d+MaxSendSegments: i nt
dabc::PCITransport
#fMutex:Mutex#fRecvQ ueue:PCI TransportRecvQ ueue*#fRecvQ ueueLength: i nt#fD MARecvQueue:PCI D MATransportRecvQ ueue*#fPool :MemoryPool *
#PushBuffer:bool#PushD MABuffer:bool#PortChanged:voi d+PCI Transport+~PCI Transport+Provi desI nput:bool+Provi desO utput:bool+Recv:voi d+RecvQ ueueSi ze: i nt+Send:voi d+SendQ ueueSi ze: i nt+MaxSendSegments: i nt+D i rectSend:voi d
std: :queue<ui nt32_t>dabc::SocketRecvQueue
dabc::AbbDevice
- fD MAengi ne:mprace: :D MAEngi neWG *-fCurrentBuffer:mprace: :D MABuffer*-fExpl i ci tPol l i ng:bool
+AbbD evi ce+~AbbD evi ce+Wri teBuffer:bool+ReadBuffer:bool+SetExpl i ci tPol l i ng:voi d
std: :queue<mprace: :D MABuffer*>dabc::PCIDMATransportRecvQueue
dabc::GenericTransport
#fPortMutex:Mutex#fPort: Port*#fD evi ce:G eneri cD evi ce*#gNumTransports: l ong
#G eneri cTransport#Fi reI nput:voi d#Fi reO utput:voi d#PortChanged:voi d#ProcessFastRecv:bool+~G eneri cTransport+Assi gnPort:voi d+D ettachPort: voi d+I sPortAssi gned:bool+Provi desI nput:bool+Provi desO utput:bool+Recv:voi d+RecvQ ueueSi ze: i nt+Send:voi d+SendQ ueueSi ze: i nt+D i rectSend:voi d+MaxSendSegments: i nt+NumTransports: l ong
dabc::CommandDoTimeSync
+CommandD oTi meSync
dabc::CommandPCIBoardDeviceConnect
+CommandPCI BoardD evi ceConnect
dabc::Command
#fParams:CommandParametersLi st*#fCl i ent:CommandCl i ent*
+Command+~Command+SetPar:voi d+G etPar: const char *+RemovePar:voi d+SetStr:voi d+G etStr: const char *+SetBool : voi d+G etBool :bool+SetI nt:voi d+G etI nt: i nt+SetPtr:voi d+G etPtr: voi d *+SaveToStri ng:voi d+ReadFromStri ng:bool+AddVal uesFrom:voi d+SetResul t: voi d+G etResul t:bool+Cl earResul t:voi d+SetCl i ent: voi d+I sCl i ent:bool+D oRepl y:voi d
std: :queue<MemoryBuffer*>dabc::LocalTransportRecvQueue
dabc::LocalTransportPair
#fO wnMutex:Mutex#fMutex:Mutex*#fMemCopy:bool#fTr1:Local Transport*#fTr2:Local Transport*
#CheckCl eanup:voi d+Local TransportPai r+~Local TransportPai r+I sActi ve:bool
dabc::LocalDevice
#fCounter: l ong
+Local D evi ce+~Local D evi ce+Submi tCommand:bool+Cl eanupD evi ce:voi d+ConnectPorts:bool
dabc::GenericDevice
#G eneri cD evi ce+~G eneri cD evi ce+Cl eanupD evi ce:voi d
dabc::NetworkDevice
#fD evi ceMutex:Mutex#fVector:NetTransportVector*#fVectorChanged:bool
#NetworkD evi ce#~NetworkD evi ce#AddTransport: voi d#G etThrdsFol der:Fol der*#I ni ti ateCl eanup:voi d#_VectorSi ze:unsi gned#_VectorAt:NetworkTransport*#Cl oseNetD evi ce:voi d#D el eteTransports:voi d+Cl eanupD evi ce:voi d+G etTransport:NetworkTransport*+Submi tCommand:bool+ServerConnect:bool+Cl i entConnect:bool+Fi l l ConnServerI d:bool
dabc::VerbsConnThrd
#fMai n:VerbsD evi ce*#fUseMul ti :bool#fMul ti LI D :ui nt16_t#fMul ti AH : i bv_ah *#fQ ueueLength: i nt#fBufferSi ze: i nt#fConnCQ :VerbsCQ *#fConnQ P:VerbsQ P*#fConnCounter: i nt#fConnPool :VerbsMemoryPool *#fH andshakePool :VerbsMemoryPool *#fMutex:Mutex#fConnRecs:Poi ntersVector
#Fi ndRec:VerbsConnRec*#Fi ndRecH S:VerbsConnRec*#ProcessH SPacketSend:voi d#ProcessUD PacketRecv:voi d#ProcessH SPacketRecv:voi d+VerbsConnThrd+~VerbsConnThrd+Worki ngLoop:voi d+D oWakeup:bool+D oServer:bool+D oCl i ent:bool
VerbsConnRec
dabc::VerbsMemoryPool
#fUD :bool#f_mr: i bv_mr *#f_rwr: i bv_recv_wr *#f_swr: i bv_send_wr *#f_sge: i bv_sge *
+VerbsMemoryPool+~VerbsMemoryPool+mr+G etRecvWR+G etSendWR+G etRecvWR+G etSendWR+G etSendBufferLocati on:voi d *
dabc::VerbsQP
#fQ PCounter:ui nt32_t#fVerbs:VerbsD evi ce*#fType: i bv_qp_type#f_qp: i bv_qp *#f_l ocal _psn:ui nt32_t#f_remote_l i d:ui nt16_t#f_remote_qpn:ui nt32_t#f_remote_psn:ui nt32_t#fNumSendSegs: i nt
+VerbsQ P+~VerbsQ P+qp+qp_type: i bv_qp_type+i s_ud:bool+qp_num:ui nt32_t+l ocal _psn:ui nt32_t+NumSendSegs: i nt+Connect:bool+I ni tUD :bool+remote_l i d:ui nt16_t+remote_qpn:ui nt32_t+remote_psn:ui nt32_t+Post_Send:bool+Post_Recv:bool+AttachMcast:bool+D etachMcast:bool
dabc::VerbsCQ
#f_cq: i bv_cq *#f_channel : i bv_comp_channel *#f_ownchannel :bool#f_wc: i bv_wc#f_srq: i bv_srq *#f_pi pe: i nt [2]
+VerbsCQ+~VerbsCQ+channel+cq+wc+srq+wr_i d:ui nt64_t+i mm_data:ui nt32_t+i mm_data_host:ui nt32_t+qp_num:ui nt32_t+src_qp:ui nt32_t+WasRecvO per:bool+CreateWakeUpPi pe:voi d+WakeUp:voi d+Pol l : i nt+Wai t: i nt+arg:ui nt64_t+StatusStr: const char *
dabc::VerbsRecvThread
#fChannel : i bv_comp_channel *#fCQ :VerbsCQ *#fLoopBackQ P:VerbsQ P*#fLoopBackPool :VerbsMemoryPool *#fPol l i ngCnt: i nt
+VerbsRecvThread+~VerbsRecvThread+Worki ngLoop:voi d+D oWakeup:bool#TestPol l i ng:bool#Wai tChannel Event:bool
structVerbsConnectData
+ki nd:ui nt64_t+node_LI D :ui nt64_t+node_Port:ui nt64_t+conn_Q PN:ui nt64_t+conn_PSN:ui nt64_t+tr_Q PN:ui nt64_t+tr_PSN:ui nt64_t+hand_Q PN:ui nt64_t+hand_PSN:ui nt64_t+ConnType:ui nt64_t+ConnI d:char [512]
dabc::NetworkTransport
#fTransportI d:ui nt32_t#fI sI nput:bool#fI sO utput:bool#fPool :MemoryPool *#fUseAckn:bool#fI nputQ ueueLength: i nt#fO utputQ ueueLength: i nt#fMutex:Mutex#fNumRecs:ui nt32_t#fRecsCounter:ui nt32_t#fRecs:NetI ORec*#fO utputQ ueueSi ze: i nt#fAcknAl l owedO per: i nt#fAcknSendQ ueue:NetI O RecsQ ueue*#fAcknSendBuf:MemoryBuffer*#fAcknSendBufBusy:bool#fI nputQ ueueSi ze: i nt#fI nputQ ueue:NetI O BuffersQ ueue*#fFi rstAckn:bool#fAcknReadyCounter: i nt#fErrorState:bool
#NetworkTransport#~NetworkTransport#I ni t:voi d#Cl ose:voi d#ErrorCl ose:voi d#PortChanged:voi d#I sActi ve:bool#_TakeRec:ui nt32_t#_Rel easeRec:voi d#Fi l l RecvQ ueue:voi d#CheckAcknReadyCounter:bool#_Submi tAl l owedSendO perati ons:voi d#_Submi tRecv:voi d#_Submi tSend:voi d#PackH eader:bool#UnpackH eader:bool+G etI d: i nt+ProcessSendCompl :voi d+ProcessRecvCompl :bool+Provi desI nput:bool+Provi desO utput:bool+Recv:voi d+RecvQ ueueSi ze: i nt+Send:voi d+SendQ ueueSi ze: i nt+D i rectSend:voi d+ProcessActi on:voi d
ENetworkO perTypes NetI O Rec
dabc::NetworkDeviceThread
#fD evi ce:NetworkD evi ce*
+NetworkD evi ceThread+~NetworkD evi ceThread
dabc::SocketDevice
#fSendThrd:SocketSendThread*#fRecvThrd:SocketRecvThread*#fServerThrd:SocketConnThread*#fCl i entThrd:SocketConnThread*
+SocketD evi ce+~SocketD evi ce+ServerConnect:bool+Cl i entConnect:bool+Fi l l ConnServerI d:bool#CreateTransport:voi d#Rebui l dUFD S:bool
dabc::Port
#fPool : Pool Envel ope*#fI nputQ ueueCapaci ty: i nt#fI nputPendi ng: i nt#fO utputQ ueueCapaci ty: i nt#fO utputPendi ng: i nt#fUseAcknol edges:bool#fTransport:G eneri cTransport*
#D ecO utputPendi ng:voi d#I ncI nputPendi ng:voi d#D i rectSend:bool#ProcessFastRecv:bool+Port+~Port+Pool :Pool Envel ope*+I nputQ ueueCapaci ty: i nt+O utputQ ueueCapaci ty: i nt+I sUseAcknol edges:bool+ChangeUseAcknol edges:voi d+NumO utputBuffersRequi red: i nt+NumI nputBuffersRequi red: i nt+Assi gnTransport:voi d+I sConnected:bool+D i sconnect:voi d+I sI nput:bool+I sO utput:bool+Fi reI nput:voi d+Fi reO utput:voi d+O utputQ ueueSi ze: i nt+O utputPendi ng: i nt+O utputBl ocked:bool+MaxSendSegments: i nt+Send:bool+Send:bool+I nputQ ueueSi ze: i nt+I nputPendi ng: i nt+I nputBl ocked:bool+Recv:bool+Recv:bool
dabc::PCIB oardDevice
#fCancel I O :bool#fEnabl eD MA:bool#fBoard:mprace: :Board*#fPool :MemoryPool *#fTransport:PCI Transport*#fThread:PCI Worki ngThread*#fReadBAR:unsi gned i nt#fReadAddress:unsi gned i nt#fReadLength:unsi gned i nt#fWri teBAR:unsi gned i nt#fWri teAddress:unsi gned i nt#fWri teLength:unsi gned i nt-fD MABuffers:std: : vector<mprace: :D MABuffer*>-fFreeBuffersQueue: std: :queue<mprace: :D MABuffer*>-fMutex:Mutex
+PCI BoardD evi ce+PCI BoardD evi ce+~PCI BoardD evi ce+Submi tCommand:bool+Cl eanupD evi ce:voi d+Wri teBuffer:bool+ReadBuffer:bool+ProcessI O :voi d+Bui l dI O Buffers:voi d+NextD MABuffer:mprace: :D MABuffer*+Rel easeD MABuffer:voi d+SetReadBuffer:voi d+SetWri teBuffer:voi d+I sCancel I O :bool+Cancel I O :voi d#ForwardBuffer:bool#ForwardBuffer:bool std: :queue<MemoryBuffer*>
dabc::PCITransportRecvQueue
dabc::PCIWorkingThread
#fD evi ce:PCI BoardD evi ce*
#Worki ngLoop:voi d#D oWakeup:bool+PCI Worki ngThread+~PCI Worki ngThread
std: :queue<MemoryBuffer*>dabc::NetIOBuffersQueue
std: :queue<ui nt32_t>dabc::NetIORecsQueue
Test2SendModule
#fPool :dabc: :Pool Envel ope*#fCanSend:bool#fSendRate:dabc: :Ratemeter#fPortCnt: i nt
+Test2SendModul e+ExecuteCommand:bool+ProcessO utputEvent:voi d+ProcessTi merEvent:voi d+AfterModul eStop:voi d
Test2RecvModule
#fPool :dabc: :Pool Envel ope*#fSl eepTi me: i nt#fRecvRate:dabc: :Ratemeter#fReal Sl eepTi me:dabc: :Average
+Test2RecvModul e+ExecuteCommand:bool+ProcessI nputEvent:voi d+AfterModul eStop:voi d
Test2WorkerModule
#fCounter: i nt
+Test2WorkerModul e+ExecuteCommand:bool+Mai nLoop:voi d
bnet::ReadoutModule
#fSubI d:SubD etector_t#fPool :dabc: :Pool Envel ope*#fWorker:SubeventsG eneratorWorker*#fEventCnt:ui nt64_t#fAbbI nput:bool#fAbbI ndependent:bool#fLastTm:doubl e#fMaxD i ff:doubl e#fAverD i ff:dabc: :Average#fAbbRate:dabc: :Ratemeter
+ReadoutModul e+BeforeModul eStart: voi d+AfterModul eStop:voi d+Mai nLoop:voi d+SubI d:SubD etector_t+ProcessO utputEvent:voi d+ProcessI nputEvent:voi d#G eneratePacket:bool
dabc::Average
#num: l ong#sum1:doubl e#sum2:doubl e#mi n:doubl e#max:doubl e
+Average+~Average+Reset:voi d+Fi l l :voi d+Number: l ong+Mean:doubl e+Max:doubl e+Mi n:doubl e+D ev:doubl e+Show:voi d
dabc::Ratemeter
#firstoper:doubl e#l astoper:doubl e#numoper: l ong#total packetsi ze: l ong#fMeasureI nterval :doubl e#fMeasurePoi nts: l ong#fPoi nts:doubl e *
+Ratemeter+~Ratemeter+D oMeasure:voi d+Packet:voi d+Reset:voi d+G etRate:doubl e+G etTotal Ti me:doubl e+AverPacketSi ze: i nt+G etNumO per: i nt+SaveI nFi l e:voi d+SaveRatesI nFi l e:voi d
Test2Plugin
+Test2Pl ugi n+CreateModul e:dabc: :Modul e*
D ABC 10-J ul -2007(Note: Mutex and Lockguardhi dden)
SocketInfiniBand
PCIboard
Port
Bnet
Module
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
14Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
Slow control
Tools for control
• Configuration via XML files• State machines, Infospace, message/error loggers, monitoring• Communication: Webserver, SOAP, DIM• Connectivity through DIM to: LabView, EPICS, Java, any DIM client/server• Java with NetBeans (Matisse GUI builder) maybe soon in Eclipse• Front-end controls?• Mix of cooperating control systems• First LabView and Java GUIs operable
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
15Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
Controls & monitoring communication
XDAQ Executive (process, address space)
Web server (SOAP)
DIM client:Java GUILabView GUIEPICS GUI
SOAP client:Java GUI
Infospace*
XDAQ Application
Modules, command queue
DABC data-flow
DIM server
State machine
Web browser
GRIDCC
* Infospace – remotely accessible parameters
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
16Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
LabView-DIM Control GUI*
Generic construction of fixed parameter table from DIM servers
* Dietrich Beck, EE
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
17Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
Java-DIM Control
Generic construction from commands and parameters offered by DABC DIM servers
Applications create the commands and parametersand publish
Ratemeter, trending, statistics
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
18Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
Generic Java DIM GUI controls
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
19Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
Event building network (BNet)
data dispatcher
FEEdeliver time stamped
data
CNetcollect data into buffers
Detector Detector Detector
collect
~50000 FEE chips
event dispatcher
switching networkBNetsort time stamped data
~1000 links a 1 GB/sec
HNethigh level selection
to high level computing
and archiving~1 GB/sec Output
processing
PNetprocess events
level 1&2 selection
subfarm subfarm subfarm ~100 subfarms
~100 nodes per subfarm
~10 dispatchers → subfarm
~1000 collectors
~1000 active buffers
TNettime distribution
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
20Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
collectingsortingtagging
buildingfilteringanalysis
collectingsorting tagging
buildingfilteringanalysis
IB
Linux
Linux
GE
analysisarchive
archive
Linux
GE: Gigabit EthernetIB: InfiniBand
frontendDataDispatcher
frontendother
frontendMBS readout Sender
Sender
BNet dataflow – bidirectional approach
DABC
Receiver
Receiver
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
21Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
plugin
input
BNet – modules view
plugin*
Readout
plugin
SubeventCombiner
DataReceiver
DataSender
Network NxM
plugin
EventBuilder
plugin*
EventFilter
M sender nodes
6 different modules
N receiver nodes
N outputs
M inputs
Ana
lisys
/ S
tora
ge
* optional
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
23Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
InfiniBand – testbench for BNet
InfiniBand - reliable low-latency zero-copy high-speed data transport
Aim: Prove of principal as event building network candidate for CBM
Tests last year:
• GSI cluster - 4 nodes, SDR
• Forschungszentrum Karlsruhe* (March 2007) – 23 nodes DDR
• UNI Mainz** (August 2007) - 110 nodes DDR
DDR – double data rate (up to 20 Gb/s), SDR – single data rate (10 Gb/s)
SDR (GSI) DDR (Mainz)
Unidirectional 0.98 GB/s 1.65 GB/s
Bidirectional 0.95 GB/s 1.3 GB/s
Point-to-point tests
Rate per node
Gb Ethernet ~100 MB/s
InfiniBand ~800 MB/s
BNet prototype (GSI, 4 Nodes)
* thanks to Frank Schmitz, Ivan Kondov and Project CampusGrid in FZK
** thanks to Klaus Merle and Markus Tacke at the Zentrum für Datenverarbeitung in Uni Mainz
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
24Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
Scaling of asynchronous traffic
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
25Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
Chaotic (async.) versus scheduled (sync.)
27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev
26Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php
DABC further tasks
Achieved:• Control infrastructure
– setup, configure– communication – very first Java GUI
• Data flow engine– multi-threading– InfiniBand– Sockets (Gigabit Ethernet)– first PCIexpress board– good performance– back pressure
To do:• Data formats• Error handling/recovery• MBS event building• Time-stamped data• Final API definitions• Documentation