shine offline framework: status and plans

NA61 collaboration meeting, Zagreb, October 2011 1

Shine Offline Framework:Status and Plans

The Shine Offline Group


Shine: Organization

• Framework: event (object persistence), detector & mag. field classes, central configuration, coordinate systems, data-source managers

• Event I/O: raw event (BOS), DS, SHOE (import/export)

• Modules: reading/writing, calibration, reconstruction, simulation

• NA49: clients, interfaces

• eventBrowser

• Validation: framework, core, event structures, modules

• Utilities: misc. helpers, STL (containers), Math, Geometry

• Documentation: SHOE, I/O, modules, reconstruction, physics, tracking

• Build system: CMakeModules, CMakeLists


Overview

Run Control

Central ConfigManager Registry

NA61:

TPC

TOF

Trigger

PSD

NA61:

TPC

TOF

Trigger

PSD

Mag. Field


Framework

• Event structures: SHOE

• Detector description

• strict separation of framework classes and (physics) code

• one class per header file

• non-deletable: processing only adds things, Has/Make/Get interfaces

• prevent accidental processing of a copy: private ctor/dtor

• data-source Managers: config defined queues of Managers; easy source

replacement, wrappers for MySQL, SQLite, Xerces

• central configuration: Detector, Managers, Modules

• Registry of Random number streams, seeding conf


Framework: Data Managers

SQLiteMySQL Xerces

TPC

VTPC

MTPC

GTPC

TOF

BPD

Trigger

PSD

Magnetic field

Drift velocity

etc.

and now also TGeo


Framework: Configuration

• XML

• run-time XML Schema validation: XSD

• automatic conversion of physical quantities into internal units

<energy unit=”GeV”> 137.23 </energy><position unit=”cm”> 14.3 </position>

• default configuration check-summed, modifications reported in run logs

• full configuration saved with data


Event I/O

Currently supports:

• raw BOS reading

• DS reading

• SHOE read/write (ROOT persistent Event)

• (pending) reading of event generators UniGen, HepMC, ...


SHOE

• output files (ROOT) are Event objects (no persistent/transient separation)

•mega/mini/micro SHOEswitching-off increasing level of Event details (branches)

• GUIevent/detector browser, run configuration inspector, simple selection tools


SHOE


Modules

• By detector: BPD, Trigger, TPCs, TOF, PSD etc.

• calibration, reconstruction, simulation (Geant4)

• readers/writers: native event/detector (SHOE)

• modules replaceableimplementing several alternative or competing approaches is easy

• (if needed) Modules communicate (only) through the Event structure


Modules: Run Control

• rudimentary control-language implemented in XML

<sequenceFile>

<enableTiming/>

<moduleControl>

<loop numTimes="unbounded">

<module> EventFileReaderSG </module>

<module> EventCheckerSG </module>

<module> BPDCalibratorSG </module>

<module> BPDEventSelectorSG </module>

<module> TPCCalibratorSG </module>

<module> TOFCalibratorSG </module>

<module> TPCClusterFinderFG </module>

<module> TPCTrackFinderWG </module>

<module> TOFTimeFinderDG </module>

<module> TPCTrackPlotterSG </module>

<module> ShineFileExporterSG </module>

</loop>

</moduleControl>

</sequenceFile>

intermediate Event states can be streamed at any time by inserting Writers


Modules: Interface

class ModuleName : public fwk::VModule {

public:

fwk::VModule::EResultFlag Init();

fwk::VModule::EResultFlag Process(evt::Event& event,

const utl::AttributeMap& attr);

fwk::VModule::EResultFlag Finish();

REGISTER_MODULE(”ModuleName”, ModuleName, ”$Id$”);

};


NA49 Clients as Modules<sequenceFile>

<enableTiming/>

<moduleControl>

<loop numTimes="unbounded">

<module> EventFileReaderSG </module>

<module> ClientInitializerSG </module>

<module config="many"> MiniModuleSG </module>

<module config="vdinit"> MiniModuleSG </module>

<module> VdriftCorrModuleSG </module>

<module config="VTPCs"> Dipt256NewModuleSG </module>

<module config="MTPCs"> Dipt256NewModuleSG </module>

<module config="VT1"> EdistoModuleSG </module>

<module config="VT1"> VtNcalcModuleSG </module>

<module config="VT2"> EdistoModuleSG </module>

<module config="VT2"> VtNcalcModuleSG </module>

<module config="MTL"> EdistoModuleSG </module>

<module config="MTR"> EdistoModuleSG </module>

<module config="ALL"> ResCorrModuleSG </module>

<module config="TrckM1"> MtracModuleSG </module>

<module config="I2RTrck"> MtracModuleSG </module>

<module config="MTPCs"> MpatModuleSG </module>

<module config="VT-2"> MpatModuleSG </module>

<module> NA49ToShineConverterSG </module>

<module> ShineFileExporterSG </module>

</loop>

</moduleControl>

</sequenceFile>

current module sequence with wrapped clients

DSPACK assimilated into a shared library (DSHACK), runs in local mode (still needs shm)


Client-Module Configuration

<ResCorrModuleSG>

<commandLine config="ALL">

-vt1_chris [DOCUMENTATION_PATH]/SampleNonEventData/RES_CORR_ZG/vt1.corr

-vt2_chris [DOCUMENTATION_PATH]/SampleNonEventData/RES_CORR_ZG/vt2.corr

-mtl_chris [DOCUMENTATION_PATH]/SampleNonEventData/RES_CORR_ZG/mtl.corr

-mtr_chris [DOCUMENTATION_PATH]/SampleNonEventData/RES_CORR_ZG/mtr.corr

-p [DOCUMENTATION_PATH]/SampleNonEventData/RES_CORR_ZG/vdrift_2007.txt

</commandLine>

</ResCorrModuleSG>

example: XML configuration of RES_CORR_ZG client,run as Shine module


Utilities

• XML Reader (Xerces library wrapper)

• SI units, internal units definition

• Mathematics, Statistics & Physics functions

• Traces, trace algorithms

• Linear Algebra, Geometry

• ODE Runge-Kutta Integrator, Mag. Field Tracker

• MD5 for configuration checksums

• custom exceptions

• logging, generic file I/O

• special template libraryspecialized event & detector containers, shadow pointer, Meyers singleton, counted objects, safe bool cast etc.


Build System

• CMakeseems to be preferred by large projects, multiplatform

• Shapefor external dependencies and production releasessmall dependency resolution engine; pythonfor installation follow instructions in twiki pages

• buildbotsautomated compile, test, validation after each commitseveral linux flavors, email blame notifications


Validation• Unit testing: CppUnit

• I/O validation (test ROOT schema evolution)

• detect changes in reconstructed quantities on example datasets


Status of Client Porting

Roland & Oskar are mostly done with the technicalities of client porting and inclusion in Shine.

more in Roland’s and Oskar’s talk

To do:

• fix all bugs related to the operational difference: clients were single processes before, initialization/run/exit once per event (nobody cared about memory management); now initialization is done only once at Shine start, call client main() for each event in the chain(most clients written in a really unprofessional way -> a lot of bug fixes, will have to be ported back to old source)

• output verification (done for some)for some can be done only on statistical level, numerical differences (e.g patrec) mostly due to the pgf/gfortran differences in executable code

• event comparison


Status of Client Portingmini ported module

vdrift_corrported module

dipt ported module

edisto ported module

vt_ncalc ported module

res_corr ported module

mtrac ported module

mpat ported module

patrec ported module

tpc_error ported module

r3d ported module

tpc_flight_time ported module

vtx ported module

ppoints ported module

gtpc_trackported module

reform ported module

ftof, tofl, tofr ported module

dedxna61 ported module

v0find ported module

v0fit ported module


News

• Migration of NDST Online Monitor to Shine

• first native module (TriggerSimulatorSG )

• MC event fully implemented in SHOE with RecEvent SimEvent cross linking

• buildbot integration (mostly done)

• first tags and a release

• several physics analysis done with SHOE


First Physics Analyses with SHOE

T61 – Tomek’s analysis

SHOE – Martin’s analysissame event selection

pC @ 31GeV 2007 (07H)h- analysis



12 events out of ~200,000

already done: MC corrections; next: acceptance, feed down


First Physics Analyses with SHOEAnd much more:

• Krypton calibration (Adam)

• S4 trigger bias studies with TriggerSimulatorSG (Michael H)

• T2 trigger probabilities with BOSToShine (Michael H)

• charged hadron analysis with SHOE (Martin)

• V0 analysis with SHOE (Michael U)



And much more...

• short lived resonances (Tanja)

• PSD (Oleg)

• (Maja, Anar, ... ?)


Development in Progress

• Shoe event: still thinking about alternative to ShoeLaces(current track/vertex linking implemented with indices)

• warning: SHOE still under development

• current production with beta release

• backward compatibility not guaranteed (yet)might affect files and user code!


Monte Carlo Status

• see Marek’s talk


Suggestions for Help

• detector description (especially “new” detectors)

• access to databases (currently DSPACK soon SQLite or ROOT)

• reconstruction modules for new detectors: PSD, LMPDthese Modules can run after the client-wrappers or, since they will not be ready soon, even in the DSToSHOE-phase:EventFileReaderSG (read and convert standard reconstruction output) -> PSDReconstructorSG -> LMPDReconstructorSG -> ShineFileExporterSG

• SHOE validationeverybody who runs old analysis on T61 or DSPACK:please re-write your progs for SHOE and check if physics results are the same

• utilities from old framework need to run natively:PadTrans: (x,y,z) given (pad,time) and vice versapotential point client: ideal for first moduleKalmanTracker?

• simulation modulesadvances with VMC, simHits are ready, need e.g. native MTSIM


End

shine offline framework: status and plans

Documents