further development of the geant4 simulation and the

CAN UNCLASSIFIED

Defence Research and Development Canada Contract Report DRDC-RDDC-2017-C164 October 2017

CAN UNCLASSIFIED

Further Development of the Geant4 Simulation and the Analysis Package for the Compton Gamma-Ray Camera Christian Van Ouellet Nicholi Shiell Ryuichi Ueno Calian Group Ltd Prepared by: Calian Group Ltd 340 Legget Drive Suite 101 Ottawa, ON, K2K 1Y6 Project Manager: Jetske Goslinga Project Agreement Number: PA15003 Technical Authority: Pierre-Luc Drouin, Defense Scientist Contractor's date of publication: February 2017

CAN UNCLASSIFIED

© Her Majesty the Queen in Right of Canada (Department of National Defence), 2017 © Sa Majesté la Reine en droit du Canada (Ministère de la Défense nationale), 2017

CAN UNCLASSIFIED

IMPORTANT INFORMATIVE STATEMENTS

Disclaimer: This document is not published by the Editorial Office of Defence Research and Development Canada, an agency of the Department of National Defence of Canada, but is to be catalogued in the Canadian Defence Information System (CANDIS), the national repository for Defence S&T documents. Her Majesty the Queen in Right of Canada (Department of National Defence) makes no representations or warranties, expressed or implied, of any kind whatsoever, and assumes no liability for the accuracy, reliability, completeness, currency or usefulness of any information, product, process or material included in this document. Nothing in this document should be interpreted as an endorsement for the specific use of any tool, technique or process examined in it. Any reliance on, or use of, any information, product, process or material included in this document is at the sole risk of the person so using it or relying on it. Canada does not assume any liability in respect of any damages or losses arising out of or in connection with the use of, or reliance on, any information, product, process or material included in this document.

This document was reviewed for Controlled Goods by Defence Research and Development Canada (DRDC) using the Schedule to the Defence Production Act.

Table of contents

Table of contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i

List of tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii

List of figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv

1 Introduction to Compton Imager and DRDC Simulation . . . . . . . . . . 1

2 Discrepancy in the ARM Distribution between Geant4 Version 4.9.6 and4.10.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

3 Quantitative Analysis of Image Resolution . . . . . . . . . . . . . . . . . . 5

3.1 Gaussian Overlap . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3.2 Contrast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3.3 Modulation Transfer Function . . . . . . . . . . . . . . . . . . . . . 9

4 Simulation of Background Events . . . . . . . . . . . . . . . . . . . . . . . 11

4.1 Background Simulation Model . . . . . . . . . . . . . . . . . . . . . 11

4.2 Geant4 Simulation of the Background with the SCoTSS Imager . . . 11

5 Likelihood Ratio Method For Source Detection . . . . . . . . . . . . . . . 14

5.1 Definition of Signal and Background Region using Likelihood Ratio . 14

5.2 PDF and Likelihood Formulation . . . . . . . . . . . . . . . . . . . . 15

5.2.1 Chi-Square Image Space . . . . . . . . . . . . . . . . . . . . 16

5.2.2 Multinomial Angular Space . . . . . . . . . . . . . . . . . . 16

5.3 Population of Test Statistic and Hypothesis Testing . . . . . . . . . 17

5.4 Additional Tests to Confirm Findings . . . . . . . . . . . . . . . . . 17

6 User’s Manual for DRDC Simulation and Analysis Package . . . . . . . . . 20

6.1 DRDC Simulation Package . . . . . . . . . . . . . . . . . . . . . . . 20

6.1.1 The main simulation . . . . . . . . . . . . . . . . . . . . . . 20

DRDC-RDDC-2017-??? i

6.1.2 Batch submission system . . . . . . . . . . . . . . . . . . . . 23

6.2 DRDC Imaging and Analysis Package . . . . . . . . . . . . . . . . . 23

7 Summary and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

ii DRDC-RDDC-2017-???

List of tables

Table 1: Description of diagnostic histograms produced by theCISimulation binary. . . . . . . . . . . . . . . . . . . . . . . . . . 21

Table 2: Description of TLeaf data contained in the EventInfo datastructure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Table 3: Description of data stored in the TBranchm OutputSummedHitList. This is the raw simulation data used asinput to the analysis package. . . . . . . . . . . . . . . . . . . . . 22

Table 4: Values extracted from the raw simulation data by the SkimTreebinary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Table 5: Contents of output file produced by the CIAnalysis binary. . . . . 26

Table 6: Contents of output file produced by the BuildPDF binary. . . . . 26

DRDC-RDDC-2017-??? iii

List of figures

Figure 1: Schematic illustration of a Compton camera . . . . . . . . . . . . 1

Figure 2: Geant4 visualization of the detector geometry . . . . . . . . . . . 2

Figure 3: Comparison of angular distributions for two different Geant4versions before fix . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Figure 4: Comparison of angular distributions for two different Geant4versions after fix . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Figure 5: Simulation of five point sources separated by 5 degrees . . . . . . 5





Figure 10: Overlap as a function of degree of separation . . . . . . . . . . . . 8

Figure 11: Contrast as a function of degree of separation . . . . . . . . . . . 9

Figure 12: MTF of the image reconstruction algorithms . . . . . . . . . . . . 10

Figure 13: Background emission spectrum from the model . . . . . . . . . . . 12

Figure 14: Detected background energy spectrum . . . . . . . . . . . . . . . 12

Figure 15: Comparison of background energy spectrum . . . . . . . . . . . . 13

Figure 16: Signal and background regions . . . . . . . . . . . . . . . . . . . . 15

Figure 17: Resulting distribution of a sample test statistic . . . . . . . . . . . 18

Figure 18: Impact of larger samples on likelihoods . . . . . . . . . . . . . . . 19

Figure 19: Backprojection visualized 1/500 source salting and associatedChi-square . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Figure 20: Flow chart of analysis pipelines . . . . . . . . . . . . . . . . . . . 23

iv DRDC-RDDC-2017-???

1 Introduction to Compton Imager andDRDC Simulation

A Compton gamma imager is useful for detecting and localizing the source of gamma-emitting radioactive sources. A project to develop the imager for security, emergencyresponse and military use was funded by Department of National Defence (DND),and with collaboration with National Research Council (NRC) and Natural ResourcesCanada (NRCan).

A Compton imager mainly consists of two layers of gamma-ray detectors, each layersegmented in x-y plane for position localization. If it is assumed that the gamma rayfrom the source scattered once in the front plane (scatter layer), and fully absorbedin the back plane (absorber layer), then the angle θC between the incident directionand the direction of the scattered gamma ray can be calculated up to a cone: [1]

cos(θC) = 1 +mec2

[1

Etotal

− 1

Eabsorber

](1)

Overlap of several of these “Compton cones” gives the location of the source (SeeFigure 1 for illustration).

Figure 1: Schematic illustration of a Compton camera. Two layers of gamma-raydetectors and several overlaps of Compton cones can provide the location of initialgamma-ray source.

NRC has assembled a laboratory set up consisting of two scatter layers, each con-sisting of 9 by 9 array of thallium-doped cesium iodide (CsI(Tl)) crystals, and anabsorber layer of 10 by 10 array of thallium-doped sodium iodide (NaI(Tl)) crystals.It was successful in localizing several known sources (137Cs, 113Sn and 22Na of 1 mCiactivity at a distance of 10 m away from the detector), as well as to measure and

DRDC-RDDC-2017-??? 1

characterize detector responses. [2, 3] A Geant4 simulation was written to simulatethis experimental setup, with intention to assess the detector response at differentgeometry configurations and various scenarios. [4, 5]

The simulation framework can model either the NRC lab imager and mission-readymodule, as seen in Figure 2.

(a) (b)

Figure 2: A Geant4 visualization of the detector geometry. (a) NRC lab imagermodule; (b) A single, mission-ready portable module.

The previous report [5] outlined several areas where further investigations are needed.Some of these have been addressed in this contract report. Investigation of the dis-crepancy in the ARM distribution is detailed in Section 2; Results of a quantitativeanalysis of the images are shown in Section 3; Modelling of the background eventsfrom soil and other naturally occurring radiation is discussed in Section 4; and initialdevelopment of the likelihood ratio method for source detection is contained in Sec-tion 5. Section 6 is intended as a reference for the simulation and analysis code usedfor this project. Section 7 concludes the report.

2 DRDC-RDDC-2017-???

2 Discrepancy in the ARM Distributionbetween Geant4 Version 4.9.6 and 4.10.1

In the previous report, a discrepancy in the ARM distribution was observed betweenthe two Geant4 versions. The discrepancy only appeared for high statistical sample,and impact was deemed to be minor. However it was left for further investigation.

Figure 3 shows the discrepancy in the ARM distribution for the high statistical sam-ple (75 million initial gamma rays). There is a systematic deviation between thetwo Geant4 versions, where version 4.10.1 has narrower Gaussian distribution, whileno deviation was found in Compton angle distribution, nor the energy spectra (notshown). Identical simulation set up was used, and sane random number seeds wereused to ensure statistical independence between the two samples.

Through further investigation, the difference was found to be in the Compton modelused within the simulation. Version 4.9.6 uses Livermore model, in which the pre-collision momentum is constrained in a plane defined by the incident and scatteredphoton. In version 4.10.1, Monash model [6] is used by default that assumes two-body,fully relativistic three-dimensional model.

Figure 4 shows the same comparison as Figure 3, but using Monash model for version

Figure 3: Comparison of angular distributions: ARM (left) and reconstructed Comp-ton angle (right), for the two Geant4 versions: unpatched 4.9.6 (red) and 4.10.1 (blue).Statistically significant disagreement is observed for the ARM distribution. No dis-agreement is visible for Compton angle distribution.


Figure 4: Comparison of angular distributions: ARM (left) and reconstructed Comp-ton angle (right), for the two Geant4 versions: patched 4.9.6 (red) and 4.10.1 (blue).Both distributions show good agreement.

4.9.6. As it can be seen, the discrepancy has disappeared and the two versions agreewell within statistical uncertainties.

Comparison of a real experimental result and the Geant4 simulation has been doneusing version 4.9.6 and found to have a good agreement. Another comparison wasperformed between the experimental result and the EGSnrc simulation that uses theLivermore model. However, these studies had limited statistics in the experimentalresult and thus did not allow the impact of the different models to be visible.

As mentioned previously, the impact of this discrepancy is expected to be almostnegligible, therefore it was agreed that an upgrade of Geant4 to 4.10.1 is suggestedfor future work.


3 Quantitative Analysis of Image Resolution

Qualitative analysis of the image quality and resolving power of the imager in termsof single and multiple source, as well as moving and extended source scenario wasdiscussed in [5]. Here, a more quantitative approach to discuss the image resolutionis presented.

Three different metrics of resolution or image quality are investigated: Gaussian over-lap (3.1), contrast (3.2) and modulation transfer function (3.3). For each metric,three image reconstruction algorithms (simple backprojection, two-cone backprojec-tion, and list-mode MLEM method) as described in [5] are compared and evaluated.

3.1 Gaussian OverlapSCoTSS Geant4 simulation package was used to simulate five 137Cs point sourcesof same activity, using the lab imager configuration. Each source was placed at adistance of 10 m away from the face of the detector and they were separated bya known amount in angular space, ranging from 5 to 20 degrees from each other.Figures 5-9 shows the results of the simulation.

− − − −−

−

−

−

− − − −

− − − −−

−

−

−

− − − −

− − − −−

−

−

−

− − − −

Figure 5: Simulation of five point sources separated by 5 degrees, reconstructed bysimple backprojection (left), two-cone backprojection (centre), and list-mode MLEM(right). Bottom plot shows the intensity map projected along the x-axis.


− − − −−

−

−

−

− − − −

− − − −−

−

−

−

− − − −

− − − −−

−

−

−

− − − −


− − − −−

−

−

−

− − − −

− − − −−

−

−

−

− − − −

− − − −−

−

−

−

− − − −


Overlap of the neighbouring peaks is calculated as:

Overlap =(μ2 − FWHM2/2)− (μ1 + FWHM1/2)

(μ2 − μ1)(2)


− − − −−

−

−

−

− − − −

− − − −−

−

−

−

− − − −

− − − −−

−

−

−

− − − −


− − − −−

−

−

−

− − − −

− − − −−

−

−

−

− − − −

− − − −−

−

−

−

− − − −


where μ1,2 and FWHM1,2 are the mean and the full-width-half-maximum of the fittedGaussian between the two neighbouring peaks (For each image, a simultaneous fit offive Gaussians over a quadratic “background” contribution was done). The average


−

−

Figure 10: Overlap as a function of degree of separation, compared between thethree image reconstruction algorithms. The overlap could not be calculated for 5-degree separation.

of the overlaps can be plotted as a function of degree of separation, as depicted inFigure 10. The line Overlap = 0 indicates the threshold where two peaks are fullyoverlapped (not separable). As it can be seen, LM-MLEM can separate sources 10degrees apart, but the other two methods need at least 12 degree separation to resolvetwo neighbouring sources.

3.2 ContrastContrast of an image is defined as:

Contrast =Imax − Imin

Imax + Imin

(3)

where Imin and Imax are the minimum and maximum intensity in the image of interest.For each of Figures 5-9, Imax is calculated as the average of the three central peaks,and Imin is calculated as the average of the two central valleys.

Figure 11 shows the comparison of the three algorithms as a function of degree of sep-aration. As expected, the contrast increases when the sources are placed farther awayfrom each other. Contrast is greatly improved in the case of LM-MLEM at around 12degree separation while the backprojection methods only show gradual improvement.From Figure 11, LM-MLEM can resolve neighbouring peaks at 12 degree separationat 50% contrast level.


degree separation4 6 8 10 12 14 16 18 20

Ave

rage

Con

trast

0

0.2

0.4

0.6

0.8

1Simple BP2Cone BPLM-MLEM

Figure 11: Contrast as a function of degree of separation for the three image re-construction algorithms. Only LM-MLEM can resolve sources 12 degrees apart with50% contrast level.

3.3 Modulation Transfer FunctionModulation Transfer Function (MTF) is often used in optical systems as a measureof image quality, where it can be computed as a Fourier Transform of a Point SpreadFunction (PSF):

MTF = |FFT{PSF}| (4)

where PSF is simply the projected image of a point source. The frequency in theFourier space is the inverse of degree separation.

Figure 12 shows the comparison of the three algorithms as a function of inverse degreeof separation. Taking the same 50% contrast level as before, the plot shows that onlyLM-MLEM can resolve at 10 degree separation. The MTF is expected to over-estimatethe resolution compared to the direct contrast calculation from Equation 3 becauseit neglects detector efficiencies that are spatially dependent.


1(degree separation)0 0.05 0.1 0.15 0.2 0.25 0.3

MTF

0

0.2

0.4

0.6

0.8

1Simple BP2Cone BPLM-MLEM

Power Spectrum (MTF)

Figure 12: Modulation Transfer Function of the SCoTSS Imager calculated usingFast Fourier Transform of a Point Spread Function. Only LM-MLEM can resolvesources 10 degrees apart with 50% contrast level.

10 DRDC-RDDC-2017-???

4 Simulation of Background Events

In order to simulate the background in Geant4, a model developed by Naval ResearchLaboratory [7] was adopted and implemented. The model is detailed in Subsection 4.1,and the implemented Geant4 simulation is shown in Subsection 4.2.

4.1 Background Simulation ModelThe model uses Geant4 simulation with a prior knowledge of background spectra,and iteratively match the experimental data collected using high-purity Germanium(HPGe) detector. The general steps are (See [7] for details):

• From the experimental data, identify the most prominent energy peaks, andcross check with other literature.

• Each of the identified lines were modelled separately, i.e. simulate a monochro-matic γ-ray in Geant4 and record the response with simulated HPGe detector,including the energy resolution of the detector.

• The scaling coefficient for the individual lines are determined by comparing thesimulated result with the relative height of the peaks in the experimental dataabove the continuum.

• The Geant4 simulation was re-run using the whole collection of spectral lines.

• Using iterative process, the continuum component of the emission spectrum wasderived by comparing the simulation result with the experimental data.

The spectral information are stored as XML file, and they are converted to rootfile which can then be included in the SCoTSS Geant4 simulation. The resultingemission spectra are depicted in Figure 13. The background emission spectrum variesdepending on a specific location, however, for the purpose of this project, a simplegeneric scenario suffices. The figure shows an open coastal scenario and urban scenariodominated by concrete structures.

4.2 Geant4 Simulation of the Background with theSCoTSS Imager

The emission spectra from the above background model is implemented into Geant4simulation platform. The background source is randomly placed within a spheresurrounding the detector, and events are generated with the momentum randomlypointed toward the detector and with energy following the emission spectra from the

DRDC-RDDC-2017-??? 11

−

−

−

(a)

−

−

−

−

(b)

Figure 13: Final background emission spectrum from the model. (a) Open, coastalscenario; (b) Urban, concrete scenario. For both plots, black indicates the total flux,and red flux indicates the continuum portion.

Figure 14: Detected background energy spectrum for urban scenario. Total (left),scatter (centre), and absorber (right) spectra are shown.

model. This assumes the “worst case scenario” where the background events couldcome from anywhere, while in reality it only comes from the ground below. This alsoneglects any down scatter from the air.

Figure 14 shows the detected energy spectrum from the simulated lab imager for allcoincidence events. It portrays an expected continuum shape with the peaks spreadand smeared due to the detector responses.

Figure 15 shows the comparison of the detected energy spectrum between the simu-lated events (black) and the experimental data (blue), normalized to the height of thehighest peak. It can be seen that the overall shape portrays similar features, howevera difference in the continuum rate can be seen. This difference could be explained

12 DRDC-RDDC-2017-???

Figure 15: Comparison of the simulated background events from Geant4 events(black) and the experimental data (blue) from the lab imager, in linear scale (left),and in logarithmic scale (right).

by the fact that the model is for a “generic” background and does not reflect thetrue shapes and the proximity of the building walls an their arrangement. Electronicresponses and triggers are only approximated and this may also contribute to thedifference shown in the spectrum seen in the figure.

DRDC-RDDC-2017-??? 13

5 Likelihood Ratio Method For SourceDetection

To determine if a source of radiation exists in the field of view, one needs to observe anexcess of event counts in the range of interest over expected background. A windowabout the characteristic radiation peak in the detected energy spectrum needs tobe chosen. This energy window needs to be not too broad, as a real signal could beswamped with excessive background events, and not too tight as this could reject validsignal. To this end a likelihood ratio method has been implemented. Furthermore thecompton imager also records angular information to reconstruct directionality and, inprinciple, this angular information could be used to supplement the energy spectrumto enhance detection of weak signals. Within the constraints of our particular designfor the imager it was however found that additional angular information could notsignificantly improve weak signal detection.

5.1 Definition of Signal and Background Region usingLikelihood Ratio

A commonly used signal strength statistical test is the likelihood ratio method. Thesignal purity is defined as follows:

L =nsig true

nsig true + nbkg true

where nsig true is the number of true signal events and nbkg true is the number of truebackground events, from the MC truth event information. All background events aregenerated following the model described in Section 4. The signal region is selectedby maximizing the product of efficiency (= nsig true) and purity. The backgroundregion is selected from defining an arbitrary purity > 95%. The background regionis further limited in the total energy (Eabs + Escat) around the photopeak, since theshape of the background distribution is expected to be different depending on thespecific location (however the background shape around the peak should be stablewithin statistical uncertainties). A binary mask is thus constructed to sort eventsas either signal or background, this mask can be visualized in Figure 16, which isselected for the primary photopeak of 137Cs, 662 keV ± 65 keV was selected as thebackground region. The global signal to background ratio used to generate the maskis 1/500.

These signal and background regions are used to build the probability density func-tions (PDF) necessary for the calculating the likelihood values, as will be describedin the next section.

14 DRDC-RDDC-2017-???

Figure 16: Visualization of the 2D mask as defined by the likelihood ratio. Greenindicates the signal region, red indicates background region. Events with a high frac-tion of their energy deposition in the scatter plane are clearly filtered out of the signalregion.

5.2 PDF and Likelihood FormulationEvents are further sub divided:

• If they are useful for imaging, having a single coincidence with a properly re-constructed Compton cone, they are labelled “coincidence”.

• If they are not single coincidence, but otherwise deposit all their energy in thedetector, they are labelled “other”.

Two PDFs are constructed through the normalized sum of high statistics simulatedisotropic background: a multidimensional PDF containing all the angular informationfor coincidence events and a 1D PDF for other events.

The test statistic is constructed as follows:

P (Eventssig|nbkg) =∏

coinc,other

P (nsig|nbkg)× P (Eventssig|nsig) (5)

Where the subscript “sig” refers to signal region and “bkg” to background region.Eventssig is the set of observables for the signal box events contained in the ana-lyzed sample (for instance pixels hit, compton angles, etc...). The full details of themathematics behind this test statistic will be presented in a forthcoming paper byP.-L. Drouin. The product of Equation 5 symbolizes how the energy information from“other”, non-coincidence events is folded into the likelihood for signal detection.

DRDC-RDDC-2017-??? 15

The first term,

P (nsig|nbkg) =nbkg

nsig + nbkg

(nsig + nbkg

nsig

)psig

nsig (1− psig)nbkg (6)

is a binomial term characterized by the probability psig. The value of psig for eithercoincidence or other is determined by calculating nsig/(nsig +nbkg) from the simulatedbackground that is used to construct the PDF. It is this binomial term that containsthe energy information within the likelihood formulation.

The second term, P (Eventssig|nsig) contains the angular information of the likelihoodformulation. Over the course of the research multiple formulations of this second termwere investigated, primarily a Chi-Square and a Multinomial.

5.2.1 Chi-Square Image Space

One natural approach to improve detection is to use reconstructed image pixel strengthas an indicator of a potential source. Visualized as i-horizontal and j-vertical pixelsthe back projected compton cones will intersect each pixel with a differing arc-length.Thus each event will come with its own bin weighting across the pixel space, simplyby virtue of how much arc-length is in any given bin. This complicates what wouldotherwise be a simple multinomial, an approximation must be used, and the Chi-Square distribution was decided upon as it approximates a multinomial when thesample size is large.

P (Eventssig|nsig) = P (ni,j|n, pi,j) =1

2n2 Γ(n

2)(z

n2−1

1 e−z12 ) · · · (z

n2−1

i,j e−zi,j2 ) (7)

where zi,j is constructed out of the pi,j intensity map of the backprojection image.It is possible to also incorporate energy in this image space parametrisation, result-ing in an additional multinomial term for Other events. However with inclusion ofanother dimension this greatly lowered statistics within individual pixels, ultimatelythe decision was made to ignore this component.

5.2.2 Multinomial Angular Space

Another approach to improve detection is to use the angular variables directly, thetwo angular coordinates of the cone centre and the Compton opening angle. Thisleads to a multinomial in angular space:

P (Eventssig|nsig) = P (ncx,cy ,r|n, pcx,cy ,r) =n!

n1!n2! · · ·ncx,cy ,r

pn11 p

n22 · · · p

ncx,cy,r

cx,cy ,r (8)

16 DRDC-RDDC-2017-???

where cx, cy, r, are defined in a convenient system for a pixelated detector. Theenergy Other multinomial additionally reduces to unity for this particular choice ofparameter space.

5.3 Population of Test Statistic and HypothesisTesting

Comparison is then performed between statistically independent sets of high statis-tics isotropic background and a similar set of background salted with a pre-definedratio of source events. Typical salting ratios used are 1/500, where one backgroundevent is at randomly replaced by a source event for every 500 events. The null hy-pothesis distribution of the likelihood values are populated using samples to makethe histograms labelled background in Figure 17. The salted samples contained 137Cssource data with the source positioned off axis at 20 degrees in both (cx,cy) and pro-duce the remaining histograms in Figure 17. A sum of all the likelihood componentscan be computed and based on a pre-established p-value (false alarm rate), the nullhypothesis maybe rejected.

5.4 Additional Tests to Confirm FindingsOne possibility for explaining why the angular term of the likelihood does not pro-vide significant resolving power relative to the binomial energy term may be that thesample size used to build the likelihood distribution was too small. With additionalstatistics presumably the pixels with excess source events could be more noticeablerelative to the background PDF. The results can be seen in Figure 18. A multino-mial angular likelihood component could perhaps supersede the simple binomial withvery large samples however this is beyond all practical application of the envisionedCompton imager.Figure 19 demonstrates the difficulty of detection in image space. The PDF wasconstructed from 194 thousand signal region coincidences, compared with 56 billionother events within the signal region. Within the background region there were 757thousand coincidence events compared to 292 billion other events.

DRDC-RDDC-2017-??? 17

Figure 17: Breakdown of likelihood components of Equation 5 for (background)vs (background with 1/500 salting of source data). The clearest separation can beseen between the Salted Binomial Other (green) and the Background Binomial Other(pink). The Chi-Square term unfortunately does not exhibit enhanced separation. Itis the much greater statistics (≈300x) of the Other samples that is the primary reasonthe likelihoods built out of purely Coincidence events have less discrimination power.

18 DRDC-RDDC-2017-???

Figure 18: Larger samples improves the resolving power of all of the likelihood com-ponents. The difference of the means of salted and background likelihood distributionsare here weighted by their RMS widths. The binomial Other component remains thestrongest discriminator for the presence of a source even using large samples.

(a) (b) (c)

Figure 19: Back projection visualized pixel space (a) The background PDF. Sincethe background is isotropically generated the shape of this distribution is entirely theresult of the detector geometry, efficiency. (b) A point like 137Cs source located 5 mfrom the scatter plane at 20 degrees. (c) With 1/500 salting the source can barelybe distinguished by eye in image space from null hypothesis, under a sigma in pixelto pixel variation for the entire dataset. Within individual samples the statisticalfluctuations are considerably higher.

DRDC-RDDC-2017-??? 19

6 User’s Manual for DRDC Simulation andAnalysis Package

6.1 DRDC Simulation PackageThe DRDC simulation package is hosted on the head node DRDC cluster and trackedusing SVN version control system. There are two major components which are de-tailed below.

6.1.1 The main simulation

The main Geant4 simulation code can be found in /Analysis/Compton/NRCPixe-lImager. The simulation package has the following directory structure:

• include/ : Includes the C++ header files for the simulation

• log/ : Includes the log files produced during the batch submission stages

• macros/: Includes the macro files used for Geant4 simulation

• obj/: Houses the object files that are produced during the compilation process

• Outputs/, RunData/: The simulation outputs are redirected to these folders,instead of populating the root directory

• src/: Includes the C++ source files for the simulation

– src/Geometries/: This directory includes different detector geometries (labimager, modular imager, and stacked imager). src/FullDetectorConstruc-tion.cpp is symlinked to one of these geometries, and the program shouldbe recompiled each time the detector geometry is changed.

– src/Event/: This directory includes sources related to the Event class (seebelow). It is separated from the main simulation code.

In order to simplify the compilation package, a makefile is written. Simply running

> make

will produce the simulation executable (called CISimulation). To run the simulation,issue the following command

> . / CISimulat ion −−macro macros/ m a c r o f i l e \[−−seed x ] [−−proce s s y ] [−−batch ] [−−background ]

20 DRDC-RDDC-2017-???

Table 1: Description of diagnostic histograms produced by the CISimulation binary.Histogram(s) DescriptionEventPrim[X/Y/Z] These histograms contain the initial position of primary par-

ticles in the simulation. Histograms EventPrim[X/Y/Z] con-tain the X, Y, Z Cartesian coordinates respectively.

EventPrimXY EventPrimXY plots the (X,Y) position of the initial parti-cle.

EventBeamE This histogram records the initial energy of the primaryparticle.

EventBeam[XY/ZY] Beam distribution at origin in the XY/ZY plane. Axis unitsare mm.

EventPrimMomTheta The initial theta angle of the primary particles momentum.EventPrimMomPhi The initial phi angle of the primary particles momentum.

The arguments in brackets are optional, and used for parallel computing, batch sub-mission, and background simulation.

The output from the simulation is stored in a ROOT tree format with a specificstructure. In order for the analysis code to read it, a shared library needs to becreated. In order to produce this shared library, run:

> make so

This will produce a shared library object called libEvent.so.

The output Root file contains both histograms and a TTree data structure namedEventInfo. The TTree data structures can be further broken down into TBranch andTLeaf objects. The histograms contain diagnostic data about the simulation run andare described in Table 1. The TLeaf data contained in EventInfo are summarizedin Table 2. The contents of the TBranch m OutputSummedHitList are described inTable 3. This TBranch contains results of the simulation which are used as inputsto the analysis package (See Section 6.2). All other TBranch and histograms notincluded in the tables are obsolete and should be removed.

In addition, a helper executable called GAIAMaker makes a GAIA output file fromthe Geant4 simulation output. This is useful to compare the output from NRC’sGAIA analysis package. In order to produce these executables, run:

> make ga ia

Finally, to clean the executables and compiled objects, issue:

> make c l ean

DRDC-RDDC-2017-??? 21

Table 2: Description of TLeaf data contained in the EventInfo data structure.Leaf Descriptionm dEventNum Internal simulation value.m dEventTime Time of event using simulation time.m dSourceEnergy Initial energy of primary particle.m SourcePosition[X/Y/Z] The (X,Y,Z) coordinates of primary particles initial locationm bEventSlabVeto Obsoletem bEventSurroundVeto Obsoletem dRunNum Obsolete

Table 3: Description of data stored in the TBranch m OutputSummedHitList. Thisis the raw simulation data used as input to the analysis package.Variable Name DescriptionfUniqueID Obsolete.fBits Obsolete.m evTime Obsolete.m dEnergy The sum deposited energy in the pixel.m dSmearedEnergy Obsolete.m iCell The pixel ID (0-15 scatter plane, 16-32 absorber plane).m dXPos X coordinate of pixel centre.m dYPos Y coordinate of pixel centre.m dZPos Z coordinate of pixel centre.m dTime Time that the **first** hit is recorded in pixel.m SubDetector Obsolete.m iLayer The layer ID (0 = Scatter plane 1, 1 = Scatter plane 2, 2 =

Absorber Plane)m dInEnergy The energy incident on the pixel. (Incoming energy)m dFirstOutEnergy The energy of the incident particle after first interaction in

pixel.m dOutEnergy Set to (m dInEnergy - m dEnergy) which is the energy leav-

ing the pixel.

22 DRDC-RDDC-2017-???

6.1.2 Batch submission system

In order to produce a big sample, it is often useful to have a batch submission system.Fortunately, DRDC head node cluster has a framework that allows parallel computingon it’s 150-core cluster. The home directory of the user “rueno” on head node housesthe bash scripts used for batch submission.

6.2 DRDC Imaging and Analysis PackageThe Compton analysis package is hosted on the head node DRDC cluster and trackedusing SVN versioning control system, under /Analysis/Compton/Analysis. The anal-ysis pipeline is broken down into two streams (detection, and imaging), each withmultiple steps carried out by separate binaries.

Figure 20: The analysis pipeline has two streams each with multiple steps. The topbranch illustrates the imaging stream, and the bottom branch shows the detectionstream.

A makefile is used to compile the whole analysis package,

make a l l

This command will produce four binary files SkimTree, CIAnalysis, LRatioCut, andBuildPDF. These executables make up the components of both the imaging anddetection analysis streams.

SkimTree. The first step in either type of analysis (detection or imaging) is to runSkimTree. This binary preprocesses the raw simulation data for use by the rest of theanalysis chain. The program is executed by issuing the following command

. / SkimTree −−in i n F i l e −−out ou tF i l e [−−debug ]

The output is stored in a Root file whose content is summarized in Table 4. Theoptional [–debug] flag outputs information about each event to the screen, adds ad-ditional branches to the root tree, and pauses after the analysis of each event.

DRDC-RDDC-2017-??? 23

Table 4: Values extracted from the raw simulation data by the SkimTree binary.Data Type Leaf Name DescriptionDouble t mcthetax The azimuthal angle of the primary particles initial location.Double t mcthetay The polar angle of the primary particles initial location.Double t mcsene The sum of all energy deposited into the scatter plane.Double t sene The sum of all smeared energy deposited into the scatter

plane. The smearing function was measured empirically.Double t mcaene The sum of all energy deposited into the absorber plane.Double t aene The sum of all smeared energy deposited into the absorber

plane. The smearing function was measured empirically.Double t (sx,sy,sz) The central coordinates of the pixel hit in the scatter plane.

(If there are multiple hits, the time used is that of the pixelwith the highest ID number.

Double t (ax,ay,az) The central coordinates of the pixel hit in the absorberplane. (If there are multiple hits, the time used is that ofthe pixel with the highest ID number.

Double t time Currently not set.UInt t mcflags MC truth table flags.UInt t hitflags Event selection flagsUInt t vetflags Obsolete

LRatioCut. The definition of signal and background regions detailed in Section 5 iscarried out by the LRatioCut binary. This code produces two binary mask histograms(CutMask1D, and CutMask2D) which are used to systematically place events intothe background box or signal box based on the energy deposited into the scatter andabsorber planes. The 2D mask is used for events which deposit energy into a singlepixel of both the scatter and absorber plane (called single coincidence events). The1D mask is used to sort all other events which deposit energy into either plane (calledOther events). The results from LRatioCut are used in CIAnalysis when running thedetection stream of the analysis pipeline.

The output file also contains a number of histograms of intermediate results whichare useful for debugging.

To execute LRatioCut the following command is issued,

> . / LRatioCut −−s i g s i g n a l F i l e s t a r t end \−−bkg bkgFi l e s t a r t end \−−out f i l ename −−mixing value

The –sig/–bkg flags specify the path and the file name base of the signal/backgrounddata files. The start and end values identify which files to include. In order for LRati-

24 DRDC-RDDC-2017-???

oCut to find the correct files in the range [start, end ] the input files must end with anumber padded with up to four zeros (ie. signalFile00001).The –out flag specifies thename of the output file. The output file is created in the same directory as the LRa-tioCut binary. The –mixing flag specifies the ratio between signal and background.The default value is 1:1000 (sig:bkg) or 0.001.

CIAnalysis. The CIAnalysis code calculates the Compton cone parameters, placesevents into either the signal or background box (by appending the events hitflags), andimaging of the source using various reconstruction algorithms. To run the CIAnalysiscode, the following command is used,

> . / CIAnalys is −−in i n F i l e −−out ou tF i l e [−−mask mask f i l e ] \[−−source sourceName ] [−− o f f s e t x y z ] \[−−cones ] [−−arm ] [−−sbp ] [−−tc ] [−−lm ]

where the arguments in brackets are optional. The –mask flag specifies the root filecontaining the CutMask1D and CutMask2D masks produced by LRatioCut (onlyrequired for detection). The –source flag names an isotope contained in the fileknown sources.txt. If –source is not used then the total energy used in the calcu-lation of the Compton cones is the sum of the energy deposited in the scatter andabsorber planes. The –offset option specifies the known (x,y,z) position of the sim-ulated source. This option is required if the –arm flag is used. The remaining flagscontrol what type of analysis is conducted. The –cones option calculates only Comp-ton cone parameters, –arm calculates the angular resolution based on the true sourceposition specified by –offset. The last three flags –sbp, –tc, and –lm specify the typeof imaging algorithm to apply (if any) and correspond to simple back projection,two cone back projection, and list mode maximum likelihood estimation method,respectively.

The package also saves histograms and output to a Root tree for further analysis. Thefile contains a TTree named cones and up to three histograms. The exact contents ofthe output file depends on the optional flags specified at run-time. The contents aredescribed in Table 5.

BuildPDF. The BuildPDF binary executes the source detection technique detailedin Section 5. The binary is run by issuing the following command

> . / BuildPDF −−bkg bkgInputFi le −−source s r c I n p u t F i l e \−−pdf p d f F i l e −−out o u t f i l e −−s a l t i n g value

The –bkg and –src flags specify the input data used in the analysis. The –salting valueneeds to be an integer and specifies the signal to background ratio used in generatingsalted toy samples for hypothesis testing. The background input is split in to two

DRDC-RDDC-2017-??? 25

Table 5: Contents of output file produced by the CIAnalysis binary.Name Descriptioncones.Time Obsolete.cones.ETot Total energy deposited in scatter plane plus absorber plane

[MeV].cones.centre x Azimuthal angle of Compton cone centre as measured from

origin [radians].cones.centre y Polar angle of Compton cone centre as measured from origin

[radians].cones.radius Compton angle [radians].cones.radius error Error on Compton angle [radians]cones.hitflags Event selection flags. Used by BuildPDF binary.hBackProjectionImage Image produced by simple back projection algorithm.hTwoConeImage Image produced by two cone back projection algorithm.hLMMLEMImage Image produced by LM-MLEM algorithm.

Table 6: Contents of output file produced by the BuildPDF binary.Name DescriptionhFinalL[ bkg] Test statistic (t.s) distribution for alter/null hypothesishBinCoincL[ bkg] Binomial component of t.s for alter/null hypothesis (co-

incidence events)hBinOtherL[ bkg] Binomial component of t.s for alter/null hypothesis

(other events)hChi2CoincL[ bkg] Chi-squared component of t.s for alter/null hypothesis

(coincidence events)hMultiOtherL[ bkg] Multinomial component of t.s for alter/null hypothesis

(other events)hEnergySpectra [salted/bkg] Energy distribution of events in alter/null hypothesis

equal parts. The first half is used to generate the model PDFs and to compute therate of ’single coincidence’ to ’other’ events in the background box. The other halfgoes towards generating the toy samples for null hypothesis testing. The salting isdone on the fly and the signal events are incorporated into background randomly.When either the background or source data runs out the program terminates and thesamples are then used to generated likelihood distributions for hypothesis testing. The–pdf flag specifies the file name of an interim pdf file. The output from BuildPDF issummarized in Table 6.

Finally, to clean the executables and compiled objects, issue:

make c l ean

26 DRDC-RDDC-2017-???

7 Summary and Conclusions

The Compton gamma imager is an on-going effort at DRDC, and this report continuesthe work presented in [5]. In particular, the items which were identified as needingfurther investigations were addressed. Namely:

• The discrepancy in the ARM distribution between Geant4 version 4.9.6 and4.10.1 was identified as the different Compton model used between the twoversions.

• Initial effort to develop a quantitative approach to measure the image qualityand resolution was attempted, using Gaussian overlap, contrast, and modulationtransfer function.

• A model to generate background gamma events has been implemented in Geant4simulation following a method developed by the Naval Research Laboratory.

• The likelihood ratio method for source detection was successfully implemented.Initial hypothesis that angular information could improve low level source de-tection proved to be incorrect given the specifics of the envisioned comptonimager.

• Code framework for the Monte Carlo model and source detection algorithmshas been formalised and thoroughly documented.

The focus of the research will shift away from the solved problem of detection to-wards improving imaging algorithms. There is a need for optimized implementationof imaging algorithms (such as LM-MLEM) on an embedded system, possibly utilizingGraphical Processor Unit (GPU).

DRDC-RDDC-2017-??? 27

This page intentionally left blank.

28 DRDC-RDDC-2017-???

References

[1] Phillips, G., Gamma-ray imaging with Compton cameras, Nuclear Instrumentsand Methods in Physics Research B, 99(1995), 674–677.

[2] Saull, P. et al., First demonstration of a Compton gamma imager based onsilicon photomultipliers, Nuclear Instruments and Methods in Physics ResearchA, 679(2012), 89–96.

[3] Sinclair, L. et al., Silicon Photomultiplier-Based Compton Telescope for Safetyand Security (SCoTSS), Proc IEEE Trans. Nucl. Sci., 61(2014), 2745 – 2752.

[4] Lam, J. (2015), SCoTSS Geant4 Design and Simulation,(DRDC-RDDC-2015-C113) Defence Research and Development Canada –Ottawa Research Centre.

[5] Ueno, R. (2016), Development of the GEANT4 Simulation for the ComptonGamma-Ray Camera, (DRDC-RDDC-2016-C138) Defence Research andDevelopment Canada – Ottawa Research Centre.

[6] Brown, J., Dimmock, M., Gillam, J., and Paganin, D. (2014), A low energybound atomic electron Compton scattering model for Geant4, NuclearInstruments and Methods in Physics Research B, 338, 77 – 88.

[7] Novikova, E., Phlips, B., and Wulf, E., A gamma-ray background model forMonte Carlo simulations, Nucl. Inst. & Meth. in Phys. Res. A, 579(2007),279–283.

DRDC-RDDC-2017-??? 29

This page intentionally left blank.

30 DRDC-RDDC-2017-???

CAN UNCLASSIFIED

CAN UNCLASSIFIED

DOCUMENT CONTROL DATA (Security markings for the title, abstract and indexing annotation must be entered when the document is Classified or Designated)

1. ORIGINATOR (The name and address of the organization preparing the document.Organizations for whom the document was prepared, e.g., Centre sponsoring a contractor's report, or tasking agency, are entered in Section 8.)

DRDC – Suffield Research CentreDefence Research and Development CanadaP.O. Box 4000, Station MainMedicine Hat, Alberta T1A 8K6Canada

2a. SECURITY MARKING (Overall security marking of the document including special supplemental markings if applicable.)

CAN UNCLASSIFIED

2b. CONTROLLED GOODS

NON-CONTROLLED GOODS DMC A

3. TITLE (The complete document title as indicated on the title page. Its classification should be indicated by the appropriate abbreviation (S, C or U) in parentheses after the title.)

Further Development of the Geant4 Simulation and the Analysis Package for the ComptonGamma-Ray Camera

4. AUTHORS (last name, followed by initials – ranks, titles, etc., not to be used)

Van Ouellet, C.; Shiell, N.; Ueno, R.

5. DATE OF PUBLICATION (Month and year of publication of document.)

February 2017

6a. NO. OF PAGES (Total containing information, including Annexes, Appendices, etc.)

34

6b. NO. OF REFS (Total cited in document.)

7 7. DESCRIPTIVE NOTES (The category of the document, e.g., technical report, technical note or memorandum. If appropriate, enter the type of report,

e.g., interim, progress, summary, annual or final. Give the inclusive dates when a specific reporting period is covered.)

Contract Report

8. SPONSORING ACTIVITY (The name of the department project office or laboratory sponsoring the research and development – include address.)

DRDC – Suffield Research CentreDefence Research and Development CanadaP.O. Box 4000, Station MainMedicine Hat, Alberta T1A 8K6Canada

9a. PROJECT OR GRANT NO. (If appropriate, the applicable research and development project or grant number under which the document was written. Please specify whether project or grant.)

9b. CONTRACT NO. (If appropriate, the applicable number under which the document was written.)

PA15003

10a. ORIGINATOR’S DOCUMENT NUMBER (The official document number by which the document is identified by the originating activity. This number must be unique to this document.)

DRDC-RDDC-2017-C164

10b. OTHER DOCUMENT NO(s). (Any other numbers which may be assigned this document either by the originator or by the sponsor.)

11a. FUTURE DISTRIBUTION (Any limitations on further dissemination of the document, other than those imposed by security classification.)

Public release

11b. FUTURE DISTRIBUTION OUTSIDE CANADA (Any limitations on further dissemination of the document, other than those imposed by security classification.)

CAN UNCLASSIFIED

CAN UNCLASSIFIED

12. ABSTRACT (A brief and factual summary of the document. It may also appear elsewhere in the body of the document itself. It is highly desirable that the abstract of classified documents be unclassified. Each paragraph of the abstract shall begin with an indication of the security classification of the information in the paragraph (unless the document itself is unclassified) represented as (S), (C), (R), or (U). It is not necessary to include here abstracts in both official languages unless the text is bilingual.)

___________________________________________________________________________

13. KEYWORDS, DESCRIPTORS or IDENTIFIERS (Technically meaningful terms or short phrases that characterize a document and could be helpful in cataloguing the document. They should be selected so that no security classification is required. Identifiers, such as equipment model designation, trade name, military project code name, geographic location may also be included. If possible keywords should be selected from a published thesaurus, e.g., Thesaurus of Engineering and Scientific Terms (TEST) and that thesaurus identified. If it is not possible to select indexing terms which are Unclassified, the classification of each should be indicated as with the title.) 3D imaging, Compton imager, radiation detection

further development of the geant4 simulation and the

Documents