John Marshall, 1

John Marshall,University of Cambridge

LCD-WG2, May 6 2010

Redesign of Pandora PFA

John Marshall, 2

Pandora Redesign Whilst Pandora works very well, old code has reached a point where it is extremely difficult

to extend. It is not flexible enough to try out new ideas and improvements...

ILD Letter of Intent version of Pandora has been frozen and a new version is being written from scratch.

This is much more than just a re-implementation, it is a major redesign; Pandora is now a framework for running decoupled particle flow algorithms:

Increased flexibility, designed to make it easy to try out new ideas Independent of any specific software framework and any specific detector details Properly designed code, taking findings from previous PFAs into account, makes it easier

to maintain Easier for other people to get involved; simple for users to create and run their own

algorithms Pandora framework helps separate physics in particle flow algorithms from C++ memory

management

The new Pandora is a separate library, with no dependencies. A user application, in any framework, accesses the library via a simple C++ API (application programming interface).

Will now give a very brief overview of new Pandora framework and summarise the current status (for more detail, please see slides presented at LCWS10)...

John Marshall, 3

New Pandora Structure

Specify Geometry

Create Calo Hits

Create Tracks

Create MC Particles

Register User Algorithms

Clustering Algorithm

Topological Association Algorithms

Statistical Reclustering Algorithm

Photon ID Algorithm

Fragment Removal Algorithms

Track-cluster Association Algorithms

PFO Construction Algorithm

Pandora

Algorithm

Manager

Calo Hit Manage

r

Cluster Manag

er

MC Manag

er

Geometry

Helper

Pandora Settings

Track Manag

er

Particle Flow

Object Manager

Get Particle Flow Objects

User Application: Pandora Framework, can

treat as “black box”:

Pandora Algorithms:

Pan

dora

AP

IPan

dora

Con

ten

t AP

I

John Marshall, 4

Pandora API To run Pandora, a user needs to write a

small application in their chosen software framework.

This application uses the PandoraAPI to supply Pandora with details of the detector geometry and of the calo hits and tracks in each event.

Pandora then builds its own simple objects.

Construction of these objects is simple; the user makes a Parameters class, fills the member variables and then calls the API Create function.

Example member variables for a track: d0, z0, track state at start, track state at ECal, etc.

All member variables must be specified, or an exception will be thrown when Create is called.

The user can provide this information in any order, then call the API ProcessEvent function.

Finally, user calls the API GetParticleFlowObjects function.

Pandora API

Pandora

Calo Hit Manager

Cluster Manager

Track ManagerMC

ManagerParticle

Flow Object Manager

User Application, e.g. ILDPandora

PandoraApi::Track::Parameters parameters;parameters.m_d0 = ...; ...PandoraApi::Track::Create(pandora, parameters);

John Marshall, 5

Pandora ObjectsCalo Hit

• Position + normal vectors

• Calorimeter cell size

• Absorber material in front of cell

• Time of first energy deposition

• Calibrated energy (mip equivalent, EM, Had)

• Layer + pseudolayer

• Hit type + detector region

• Density weight

• Surrounding energy

• IsDigital, IsIsolated + IsPossibleMip flags

• Associated MC particle

• Associated user object

Track

• 2D impact parameters

• Momentum at d.c.a

• Particle mass

• Charge sign

• Start track state

• End track state

• ECal track state

• ReachesECal flag

• List of track state projections to calorimeter surfaces

• Associated cluster

• Associated MC particle

• Associated user object

• PFO formation flag

• “Clusterless” PFO formation flag

Cl

uster

• List of constituent calo hits, ordered by pseudolayer

• Mip fraction

• EM energy measure

• Had energy measure

• Corrected energy estimates

• Initial direction

• Current direction

• Result of linear fit to all hits in cluster

• Energy-weighted centroid

• ShowerStart layer

• Shower profile properties

• List of associated tracks

Particle

Flow Objec

t

• PDG Code

• Charge

• Mass

• Energy

• Momentum

• List of tracks

• List of clusters

Mixture of properties specified by user and value-added properties, but all simple and well defined physics quantities for use in particle flow algorithms.

John Marshall, 6

Pandora Managers

Pandora

Calo Hit Manager

Cluster Manager

Track ManagerMC

ManagerParticle

Flow Object Manager

Pandora Managers are designed to store named lists of their respective objects.

These objects can be accessed by the Pandora Algorithms, which perform the reconstruction.

The algorithms interact with the Managers in a controlled way, via PandoraContentAPI, and the Managers perform the memory management.

At any instant each Manager has a “current” list, which can be accessed by an algorithm.

Parent algorithms can manipulate the current list in order to control scope and behaviour of daughter algorithms.

The Managers store information about currently running algorithms so they can keep track of lists.

Algorithms can use the PandoraContentAPI to modify lists and/or save new lists.

Algorithms can use the API without worrying about how the managers work – separation of

physics and C++ memory management!

Pandora Content API

e.g. Clustering Algorithm

John Marshall, 7

Pandora Algorithms In the new Pandora framework, the algorithms contain almost exclusively physics-driven code,

alongside the following typical usages of the PandoraContentAPI:

Create new clusters and particle flow objects Modify clusters, by adding hits, merging or deleting Access the current lists of Pandora objects Save new lists of clusters, calo hits or tracks Run a daughter algorithm, etc...

Static helper functions are provided to perform tasks that are useful to multiple algorithms, and the Pandora algorithms are configured via xml and can be swapped in/out without recompiling.

Pandora will provide a comprehensive library of built-in algorithms. However, we want other people to get involved:

Easy for a user to create their own algorithm and we provide a template. User then calls API functions and implements their own algorithm logic.

Once the algorithm settings are added to the xml file (an XmlHelper makes this easy), the algorithm will then run in the Pandora framework, without needing to rebuild the Pandora library.

At LCWS10, presented details of re-implementation of Pandora algorithms, together with first low energy (91GeV) results. Will now quickly summarise these results, before discussing the most recent changes...

John Marshall, 8

Completed Algorithms

Looping tracks

Cone associations

Back-scattered

tracks

9 GeV

9 GeV

6 GeV

Layers in close contact

9 GeV

6 GeV

3 GeV

Fraction of energy in cone

Neutral hadron Charged hadronPhoton

Clustering Algorithm

Topological Association Algorithms

Photon ID Algorithms

Fragment Removal Algorithms

Track-cluster Association Algorithms

PFO Construction Algorithm

Cluster firstlayer position

Projected track position

Cone-based forward projective method

John Marshall, 9

Results at 91GeV

First tests were performed for the ILD detector concept, using MC samples of approximately 10,000 Z uds generated with the Z decaying at rest with Ez = 91.2GeV.

At this energy, all newly implemented code and framework is exercised, without (a strong) need for statistical reclustering and/or leakage corrections.

For fair comparison, reclustering, leakage corrections and all use of track-relationship information was turned off in old Pandora (at the time, was still work in progress for new Pandora).

Excellent agreement observed, by construction.

sE / E (rms90) = 3.79%

All Events, 91GeV

PFO energy sum, old vs. new

John Marshall, 10

Reclustering

Algorithms that “steer” the reclustering have now been fully implemented and validated.

Have deviated from old Pandora here, tidying the algorithms and more clearly defining the role of each.

Introduces some small differences between new and old Pandora output. However, extensive validation shows that results remain statistically identical to old Pandora.

12 GeV 32 GeV

18 GeV

30 GeV Track

38 GeV

If these hits are clustered together withthese, lose energy deposit from neutral hadron (now part of track particle) and ruin jet energy measurement.

e.g.

At high jet energies, performance degrades due to increasing overlap between hadronic showers from different particles:

Pandora addresses this problem with statistical reclustering: Clusters that have been incorrectly merged together are identified via consistency of cluster

energy and associated track momentum. Attempts are made to redistribute the hits by using different clustering parameters or entirely

different clustering algorithms.

John Marshall, 11

Reclustering

As expected, reclustering

algorithms are very important at higher

energies.

All Events, 200GeV

All Events, 360GeV

All Events, 500GeV

John Marshall, 12

Track Relationships

Pass/fail track pfo selection cuts

Kink (or Split)

Prong

V0 V0

Cluster associated with track

E = Edaughter

P = Pdaughter

q= qdaughter

PID = PIDparent

E = SEgood daughters

P = SPgood daughters

q= qparent

PID = PIDparent

E = Egood daughter

P = Pgood daughter

q= Sqboth daughters

PID: examine track PIDs

E = SEdaughters

P = SPdaughters

q= Sqdaughters

PID: examine track PIDs

New Pandora code allows track parent, daughter and sibling relationships to be specified.

Uses this information when determining which tracks can be associated to clusters and when building charged PFOs.

For ILD, extract track relationships from v0 and kink finders in MarlinReco.

Charged PFOs are built using illustrated rules. Require at least one track with a cluster association, or a track which is deemed to be low pT

John Marshall, 13

Photon Recovery

Photon-like cluster

Mip-like cluster Have recently implemented a cluster fragmentation mechanism. This can be considered as a “local reclustering”:

A cluster can be broken up into different combinations of smaller fragments.

The original cluster and all fragment combinations can still be accessed and their properties queried.

When the best fragment combination has been identified, call EndFragmentation API.

ClusterLists will be tidied accordingly, any temporary clusters created will be deleted.

This functionality has been used in a newly implemented PhotonRecovery algorithm, which splits clusters up into photon-like fragments and mip-like fragments.

Further photon id helper functions, based on shower-profiles, have also been implemented.

John Marshall, 14

Remaining Work1. Leakage correction For high energy jets, non-containment of showers is

significant. To address this issue, have successfully reproduced old

Pandora identification of clusters exiting the detector. Currently use this information when e.g. considering track-cluster consistency in reclustering.

Now need to reproduce old Pandora use of muon chamber information to estimate leakage and energy deposition in the coil. However, this treatment is rather crude and will eventually be replaced.

2. Photon clustering

3. Consolidation, profiling and cpu/memory usage improvements

Peter has been working on an algorithm that runs the main clustering algorithm, applies a shower-profile based selection and saves identified photon clusters as a separate named cluster list.

The removal of these clusters allows for improved identification of hadronic showers when the clustering algorithm is called again. Can simply “plug-in” this algorithm to provide additional photon id, without any other changes.

Would like to include this in first official release of PandoraPFANew, but it still requires thorough validation.

John Marshall, 15

Plans for next few months Pandora provides some simple cluster energy corrections, which remove effects of calo hits with

anomalously high energies. The corrected cluster energy is used in comparisons with track momenta and when creating neutral PFOs.

Energy correction functionality is designed to be modular. If a function asks for corrected energy, cluster will redirect to a helper function to provide the correction. Corrected energy is then cached in the cluster until cluster changes.

We will provide a simple API so that a user can change a function pointer in the helper class, allowing them to use their own Energy Correction function.

Work is already under way to investigate a software-compensation based correction (Katja and Frank).

Energy Corrections

Particle identification Just as a user can change the energy correction function, the modular structure of

PandoraPFANew will allow a user to register different particle identification functions. As the functions may be compiled as part of the user application, these can be detector specific

functions.

We encourage people with existing particle id functions to get involved; we are interested in both “fast” (an indication) and “full” (accurate) versions of these functions.

Modular PandoraPFANew allows the user application to change the energy corrections and particle id functions without even needing to rebuild the

Pandora library.

John Marshall, 16

Summary The new Pandora framework is complete and has been rigorously tested. It is stable and has

been unchanged for over six months now.

Most of the Pandora algorithms have been fully implemented and validated, reproducing the old Pandora results.

To complete the reconstruction, need to address leakage. Then want to consolidate this first complete version of code, which renders old Pandora code

obsolete.

Current applications that use the Pandora library: ILD Pandora application completed (our default test application), Norman Graf and Jeremy McCormick working on SlicPandora application, Every Pandora algorithm is automatically and instantly available for both ILD and SiD concepts.

Have many ideas for moving forwards and improving Pandora within the new framework: New clustering algorithms, Long list of topological association improvements, Improved treatment of leakage, Particle identification and energy correction helper functions,

Easy for other people to get involved and write new algorithms or contribute ideas...