updates from sphenix tk hemmick. sphenix? one year from now, phenix will be destructively...

Updates from sPHENIXTK Hemmick

sPHENIX?• One year from now, PHENIX will be

destructively decommissioned (blowtorch…)

• sPHENIX is a concept originating w/in PHENIX for a new experiment in the 8 O’Clock Hall.

• Steps to form a brand new collaboration (which will not be called sPHENIX) started in June 2015.

• Steps to divorce the desirable portions of the PHENIX code from the “PHENIX Universe” were starting Spring 2015.

• Meeting to discuss divorced code base held 3 weeks ago…these slides are partly from there.

Intended/ Not Intended• This talk is a description of sPHENIX code.

• This talk is NOT a sales-pitch!

• As always, there are bad/bad features.

• I will both summarize what is there and also summarize what maps well (or not) onto our evolving model for SoLID.

• Slides mostly from C. Pinkenburg.

NOTE: ENTIRE codebase (offline, simulations, nightly rebuilds) is available via github and is likely most valuable while most generic!

Code: https://github.com/sPHENIX-Collaboration Meeting: https://indico.bnl.gov/conferenceDisplay.py?confId=1237

https://github.com/sPHENIX-Collaboration

https://indico.bnl.gov/conferenceDisplay.py?confId=1237

Fun4All

Coding Paradigm• There is ONLY ONE executable:

root.exe (unaltered from root.cern.ch)

• Root becomes useful by two methods:– Compile system/user code into *.so

• rootcint makes dictionaries of user objects so that these can be instantiated and manipulated at the root prompt.

– Execute macro at the root prompt.

• User *.so attaches to core *.so via libadd in their package’s Makefile.am file.

• Core codebase maintained fresh by nightly rebuilds that include complete compilation and test runs of the core code over old datafiles.

Structure of Fun4All

That’s all there is to it (as long as you don’t look under the hood)

Output ManagersInput Managers Fun4AllServer

Analysis Modules

DST

Raw Data (PRDF)

Simulated PRDF

Histogram Manager

Root File

DST

Raw Data (PRDF)

HepMC/Oscar

Empty

The Phool Node Tree

• The Node Tree is at the center of the Phenix software universe (but it’s more or less invisible to you). It’s the way we organize our data.

• It is NOT a Root TTree• We have 3 different Types of Nodes:

– PHCompositeNode: contains other Nodes– PHDataNode: contains any object– PHIODataNode: contains objects which can be written out to DST

• PHCompositeNodes and PHIODataNodes can be saved to a DST and read back• This DST contains root TTrees, the node structure is saved in the branch names. Due

to Roots limitations not all objects can become PHIODataNodes (e.g. anything containing BOOST).

• We currently save 2 root trees in each output file, one which contains the eventwise information, one which contains the runwise information

• Input Managers put objects as PHIODataNodes on the node tree, output managers save selected PHIODataNodes to a file.

• Fun4All can manage multiple independent node trees

**ALL items in PHIODataNode move seamlessly to Database and File storage.AUTOMATIC FORMATING

**Two Tickets to Paradise• “Streaming” is root’s name for taking a live

object in memory to and from static content.

• Root considers static content as TFile().

• PHENIX adds psql IO for root streamers.1. Inherit from TObject2. Add to header:

• User only defines CONTENT of object.

• IO to Database and TFile() automatic.

• Huge burden lifted.

• Analysis code identical for:– You are running GEANT and

reconstruction at same time.

– You saved simulation to file.

– You read data from file.

It’s all about choices - Input• Fun4AllDstInputManager: Reads DST’s, if reading multiple

input files it makes sure all data originates from the same event

• Fun4AllNoSyncDstInputManager: Reads DST’s but doesn’t check events for consistency (needed when reading simulated input together with real data for embedding or if you just feel like screwing up)

• Fun4AllHepMCInputManager: Read HepMC and Oscar formatted ASCII files

• Fun4AllPrdfInputManager: Reads PHENIX Raw Data Format (PRDF)

• Fun4AllDummyInputManager: just drives the event loop

NOTE: Should be easy to write Fun4AllEVIOInputManager()

• Fun4AllDstOutputManager: Write DST’s• Fun4AllEventOutputManager: Writes

Events in PRDF (packet selection possible)• Fun4AllPrdfOutputManager: Write

simulated raw data file (needs probably work).

• Fun4AllHepMCOutputManager: Write HepMC ASCII files

It’s all about choices - Output

Caveat:You can only use the Fun4AllEvenOutputManager if you have an Event object and a Fun4AllHepMCOutputManager if you have a

HepMC object on the node tree

Your Analysis Module

• Init(PHCompositeNode *topNode): called once when you register the module with the Fun4AllServer

• InitRun(PHCompositeNode *topNode): called whenever data from a new run is encountered

• Process_event (PHCompositeNode *topNode): called for every event

• ResetEvent(PHCompositeNode *topNode): called after each event is processed so you can clean up leftovers of this event in your code

• EndRun(const int runnumber): called before the InitRun is called (caveat the Node tree already contains the data from the first event of the new run)

• End(PHCompositeNode *topNode): Last call before we quit

You need to inherit from the SubsysReco Baseclass (offline/framework/fun4all/SubsysReco.h) which gives the methods which are called by Fun4All. If you don’t implement all of them it’s

perfectly fine (the beauty of base classes)

If you create another node tree you can tell Fun4All to call your modulewith the respective topNode when you register your modue

Okay, How do I navigate the Node Tree which is in every argument????

#include <g4hit/PHG4HitContainer.h>#include <fun4all/getClass.h>

Myanalysis::process_event(PHCompositeNode *topNode){ PHG4HitContainer *g4hits = findNode::getClass<PHG4HitContainer>(topNode,”G4HIT_HCALIN”); if (g4hits) …}

You need to know the name of the node and the class of theobject you want (e.g. some PHG4HitContainer version in the node called G4HIT_HCALIN (that’s where the Fun4All printoutof the Node Tree comes in handy)

topNode of the node tree yourmodule is registered with

Caveat: getClass will return the first node of a given name, if you havehave multiple identically named nodes you need to search differently

Blah Blah Blah – Very nice, but how do I run the show?

Fun4All provides only building blocks, you need to put them togetheryourself in your root macro according to your needs.

void G4HitTTree(const char *fname, const int nevnt = 0, const char *outfile="hits.root"){ gSystem->Load("libg4histos"); // load library (Makefile.am makes sure dependencies are resolved) Fun4AllServer *se = Fun4AllServer::instance(); // Get pointer to the instance of the Master of the Beast G4HitTTree *tt = new G4HitTTree(); // Make my analysis module tt->Detector(“InnerHcal”); // set my module up using it’s methods se->registerSubsystem(tt); // register with Fun4All – this determines the order in which modules are called

Fun4AllInputManager *in = new Fun4AllDstInputManager("DSTin"); // Make an input manager – we read a DST in->fileopen(fname); // open the input file se->registerInputManager(in); // register the Input Manager with Fun4All Fun4AllOutputManager *out = new Fun4AllDstOutputManager("DSTout",outfile); // Make an output manager – we write a DST out->AddNode(“MyInnerHcal”); // We only want to write out this node se->registerOutputManager(out); // register the Output Manager with Fun4All

se->run(nevnt); // Run over N events (0 means all events of input file) se->End(); // Call the End() method of analysis modules, close input and output files, write out runwise T1 TTree delete se; // cleanup, free up all memory}

Detector class design

PHG4Subsystem: SubsysReco, interface to Fun4All

PHG4Detector: Detector setup – implements GEANT construct method

PHG4SteppingAction: implements stepping action for each detector

PHG4EventAction: implements actions taken at end of event (e.g. removing G4Hits with zero energy loss)

GEANT steps

GEANT propagates particles one step at a time. The step size isdetermined by the physics processes associated with the currentparticle or when a boundary between volumes is crossed

After each step the user stepping method is called with a pointer to the current volume which has access to the full information

(energy loss, particle momentum at beginning and end of step, …)

Our G4HitsIn our stepping method we add the energy loss in each volume andstore the entry and exit coordinates (and for tracking detectors themomentum at the entry and exit)

We also keep the ancestry for G4Hits so any hit can be traced backto a primary particle. To reduce size we do not store particles

which do not leave G4Hits and are not in the ancestry of a particle which created a G4Hit

G4Hit

PHG4Reco

• Fun4All Interface to GEANT4

• Sets features of the world (size, shape, material, magnetic field, physics list)

• Provides interface to GEANT command line (especially useful for event display)

• Manages our detectors

• totally configurable on macro level

Why is it called Fun4AllServer when it’s not a Server?

The parent of Fun4All is the PHENIX online monitoring andthat one is a real server to distribute histograms. If it turnsout to be useful some day this capability can be put back

History• Development started in 2002, in use by PHENIX from

2003 on (reconstruction and analysis of Run3 data)• Underlying data structure – the phool node tree – was

part of day1 of PHENIX software (PHENIX Object Oriented … it predates stl)

• Needed to get many subsystems who developed their code independently and without coordination under one umbrella

• Development driven by reconstruction and analysis needs – not by “beauty” or some abstract design considerations (or “rules”)

Phool Node Tree for sPHENIX

Node Tree under TopNode TOPTOP (PHCompositeNode)/ DST (PHCompositeNode)/ G4HIT_HCALIN (PHIODataNode) G4HIT_ABSORBER_HCALIN (PHIODataNode) G4HIT_HCALOUT (PHIODataNode) G4HIT_ABSORBER_HCALOUT (PHIODataNode) SVTX (PHCompositeNode)/ SvtxHitMap (PHIODataNode) SvtxClusterMap (PHIODataNode) SVTX_EVAL (PHCompositeNode)/ SvtxClusterMap_G4HIT_SVTX_Links (PHIODataNode)RUN (PHCompositeNode)/ CYLINDERGEOM_SVTX (PHIODataNode) CYLINDERGEOM_SVTXSUPPORT (PHIODataNode) CYLINDERGEOM_EMCELECTRONICS_0 (PHIODataNode) CYLINDERGEOM_HCALIN_SPT (PHIODataNode)PAR (PHCompositeNode)/ SVTX (PHCompositeNode)/ SvtxBeamSpot (PHIODataNode)

Print it from the cmd line withFun4AllServer *se = Fun4AllServer::instance();se->Print("NODETREE");

TOP: Top of Default Node TreeCreation and populating of othernode trees is possible (used forembedding)




DST and RUN Node: default for I/O •DST – eventwise•RUN - runwise

Objects under the DST node are reset afterevery event to prevent event mixing. Youcan select the objects to be saved in theoutput file. Subnodes like SVTX are savedand restored as well. DST/RUN nodes canbe restored from file under other TopNodes ROOT restrictions apply:Objects cannot be added while running toavoid event mixing




Users can add their own branches.Resetting the objects is theirresponsibility.

The PAR node will probably turninto the node for calibrations whichwe want to keep on the DST




Caveat: You loose ownership oncean object is put on the node tree.Fun4All deletes the node tree when cleaning up. Deleting nodesis not supported (yet – but ROOTprevents this for the DST node)

Features

• Reading of various formats (DST’s, PHENIX raw data, HepMC, Oscar)

• Synchronization, supports parallel reading of multiple input files with only common events handed to the event loop

• User modules inherit from common base class and have to implement the methods they want to use

• Writing various output formats (DSTs, raw data, HepMC)

It’s all about choices - InputCaveat:

If you read from more than one input file containing identical nodes, nodes from the last input manager will overwrite the others.

Input Managers can run user modules on “their data” allowing you todeal with identical nodes somehow before they get wiped

Input Managers can be given multiple files or file lists. They will transparently open a new file when the previous file is exhausted

But files have to be run over in ascending segment orderotherwise the synchronization will get lost

Histogram OutPutRoot’s pathetic TDirectory handling makes it unlikely that youfigure out how to create and save histograms on your own reliably(at least that’s my experience). Use the Fun4AllHistoManager for this#include <fun4all/Fun4AllHistoManager.h>MyAnalysis::MyAnalysis(){ HistoManager = new Fun4AllHistoManager(“MyHistos"); HistoManager->setOutfileName( “myhistos.root”);}MyAnalysys::Init(PHCompositeNode *topNode){ myhist1 = new TH1F(….); // myhist1 should be declared in MyAnalysis.h HistoManager->registerHistogram(myhist1);}

This will save your histograms when executing the Fun4AllServer::End() inthe file “myhistos.root”. If you don’t set the filename it will construct aname from the name of your analysis module. If you don’t register it withFun4All you have to call the writing out yourself – e.g. saving snapshots in separatefiles or in the End() method and delete it in your dtor.

Create your own TTreeMy biased opinion: creating my own root TTrees is a very painful experience and analyzing them is even worse it’s so much fun to

Write your own eventloop and try to debug those crappy CINT macros

But aren’t our node trees saved as root trees? Wouldn’t it be nice tojust piggy back on its very generic mechanism to do this?

It is actually possible – the only thing you need to do is write aclass which contains the variables you want to save (on an eventby event basis, but you can select which event should be saved).

You put this object on the node tree and tell an output manager towrite it out.

Now you’ve got your own root TTree which you can either analyzelike you analyzed your old tree or you can feed it back intoFun4All and use another SubsysReco module to analyze it

Create your own TTree

An example how to do this is in thesPHENIX tutorials under MyOwnTTree

Don’t worry about the ugly TClonesArray gymnasticsyou just have to modify MySimpleTree

So you finally admitted to yourself that writing bugfree code is just impossible

and you are tired of adding couts

And if your code just bombs out on you, don’t use print statements, usegdb instead which will get you to the problem in no time:http://www.phenix.bnl.gov/WWW/offline/tutorials/debuggingTips.html(this one is old and needs to be brought up to date but it is still valid)

The ever popular and easy to use and understand valgrind (what morethan the line number were your code is doing the dirty deed do you want):https://wiki.bnl.gov/sPHENIX/index.php/Tools#valgrind

Here are some short tutorials how to run code checkers:

Insure is more cryptic but it does find problems valgrind cannot pick up:https://wiki.bnl.gov/sPHENIX/index.php/Tools#insure

What’s next?• Fun4All is a work in progress – it will continue to

evolve for use by sPHENIX – the current version is already incompatible with what we have in PHENIX

• Keep it simple – it’s a tool – not a monument• Next project is a port of our calibrations DB

(PdbCal) and development of DB capabilities based on it

updates from sphenix tk hemmick. sphenix? one year from now, phenix will be destructively...

Documents

root trees

root prompt

root streamers

root ttrees

root ttreewe

phenix universe

output file

node structure