glast large area telescope: status of system-level performance simulations

GLAST LAT Project 17 August, 2001

S. Ritz 1

GLAST Large Area Telescope:GLAST Large Area Telescope:

Status of System-level Performance SimulationsUPDATE (to 26 July presentation)

S. RitzGoddardLAT Instrument [email protected] the PDR/Baseline simulation group

For status see:http://www-glast.slac.stanford.edu/software/PDR/

Gamma-ray Large Gamma-ray Large Area Space Area Space TelescopeTelescope


S. Ritz 2

System-level Performance SimulationsSystem-level Performance Simulations

Outline Brief review of July 26 presentation, actions

Work done since July

background fluxes settled

large generation runs

trigger rate studies

CAL bug fix, ACD segmentation final update

tools for peeling off events, readback of events

Status of actions

The LIST (what’s left to do now that the tools are in place)

Result of much direct work by many people: Toby, Richard, Heather, Sasha, Ian, Karl, Leon, Tracy, Eduardo, Arache, Regis, …

and based on the foundation provided by many others.


S. Ritz 3

Summary of RequirementsSummary of Requirements

We use the simulation to evaluate the expected performance of the instrument design:

• Effective Area as a function of energy

• FOV (Effective Area as a function of angle)

• Energy resolution

• PSF (68%, 95%) as a function of energy and angle. Make parameterization for physics studies.

• Background rejection (and all of the above after background rejection selections)

• Trigger rates and data volume after L1T and L3T

• Failure mode effects on performance

We use the beam tests and other measurements to verify the simulation.


S. Ritz 4

Work done for proposalWork done for proposal

• We did this work for the proposal:

• Why do it again for PDR/Baseline?– quantitative assessment of performance impact of incremental design changes– better modeling of backgrounds– check results and make improvements. move analysis forward.– side benefits: opportunity to use and improve tools. simulation and reconstruction

have undergone major architectural changes -- lost some functionality but gained much more solid foundation. this is an opportunity to pull everything back together.

Source % rateAvg L1T Rate [Hz]

Chime 36 2019

albedo 4 196

Electron 1 30

Albedo p 59 3224

TOTAL 5470


S. Ritz 5

For PDR, we must present the basic performance parameters of the instrument and show they meet the relevant LAT Performance Specifications.

The following elements must be in place to begin this evaluation process:

•updated, validated, and documented geometry to match the current design, along with configuration control of the design parameters; review noise and threshold parameters;•updated source fluxes, incorporating the improved understanding; •a documented release of the simulation and reconstruction in the new framework, including the new event storage format; •a version of the event display that is compatible with the new simulation, reconstruction, and event storage format; •machinery in place to generate the necessary statistics of signal and background events. We specify a requirement of >10 million background events, with a goal of 100 million background events generated and analyzed. The machinery must include a simple mechanism to identify each event uniquely, and the capability to produce from an event list files with a subset of the full event sample; •an updated version of merit, or an equivalent tool, to run in the new environment. Ideally, this tool could run both on the stored events and on a new standard tuple. Depending on the size of the full events, a standard tuple may be a practical necessity. •a basic analysis framework to operate on the new event stores.

PDR Simulation Work Requirements PDR Simulation Work Requirements (2/2001)(2/2001)


S. Ritz 6

• new carbon cell design implemented

• detailed description of top and bottom supporting frames

• detailed description of cell closeout and electronics compartment at the sides of towers.

• all calorimeter dimensions are up-to-date.

CAL Geometry UpdateCAL Geometry Update

Work done by Sasha (CAL group)

CsI

Support frame

cylindrical end supports


S. Ritz 7

41.5 mm

CAL Geometry Update (II)CAL Geometry Update (II)


S. Ritz 8

ACD and GRID Geometry UpdatesACD and GRID Geometry Updates

• ACD support structure consists of an approximated core material and two face sheets.

• The gap between the tiles and the towers reflects the current design.

• Thermal blanket is modeled as it was for the AO, using one average density material.

• Back-most side rows now single-tiles

• GRID flange between TKR and CAL (treated as a separate volume) is undergoing modification to reflect the current design.

• Also adding ACD base frame.

Work done by Heather (ACD group) with help from Eduardo

NEWNEW


S. Ritz 9

New Features:

Dimensions correspond to latest design

Better treatment of top and bottom trays

More accurate composite materials

MCM boards included

Better segmentation of tray faces

MCM Boards

Silicon

Bias board, tray face, glue

Bias board, glue

tray face, glue

Tungsten

TKR Geometry UpdateTKR Geometry Update

Work done by Leon (TKR group)

closeoutscarbon-fiber walls + screws


S. Ritz 10

Level 1 TriggerLevel 1 Trigger

• TKR 3-in-a-row• CAL-LO = any single log with recorded energy > 100 MeV• CAL-HI = any tower with 3-layers-in-a-row each with >0 logs with

recorded energy > 1 GeV (NEW! see LAT-TD-00245-01, 16 July)• L1 Trigger word:

___ ___ ___ ___ ___ <- (LSB)

• Will continue to study details of performance of new CAL-HI proposal in this round of simulations.

• ACD throttle of L1T now included in tuple.

TK

R

CA

L-L

O

CA

L-H

I

AC

D L

O

AC

D (

un

use

d)

NEWNEW


S. Ritz 11

Background FluxesBackground Fluxes

• Implementation by source, comparisons and justifications

• Sanity checking – EGRET A-Dome rates• L1T rates

see• LAT-TD-00250-01 Mizuno et al• Note by Allan Tylka 12 May 2000, and presentations by Eric Grove• AMS Alcaraz et al, Phys Lett B484(2000)p10 and Phys Lett B472(2000)p215

NEWNEW


S. Ritz 12

New Orbit-max FluxesNew Orbit-max Fluxes(the new tools are great!!)

total

Integrates to ~10 kHz/m2


S. Ritz 13

New Orbit-avg FluxesNew Orbit-avg Fluxes

Integrates to ~4.2 kHz/m2


S. Ritz 14

Sanity CheckSanity Check

• The measurements of the flux components may have unsubtracted backgrounds – adding them up could result in double-counting. Also, fluxes below 100 MeV are mostly blind guesses. Need a unitarity check – look at EGRET A-dome rates.

• The proposed orbit-max flux amounts to ~10 kHz/m2

• Also sent fluxes to Eric Grove, Ormes and Kamae for comments, and discussed fluxes with experts at Goddard.


S. Ritz 15

EGRET A-dome Rates (courtesy of Dave Bertsch)EGRET A-dome Rates (courtesy of Dave Bertsch)

A-dome has an area of ~6 m2, so orbit max rate (outside SAA and no solar flares) corresponds to ~16 kHz/m2

This represents a conservative upper-limit for us, since the A-dome was sensitive down to 10’s of keV.

Note peak rate is at (24.7,260)

SAA


S. Ritz 16

Orbit Max Rates – Orbit Max Rates – Preliminary!Preliminary!

all [inc albedo_]

chimemax albedo_p_max CR e- max albedo e+e-

flux (kHz/m2) 9.9 4.2 2.6 0.043 2.2

L1T (Hz) 13,025 7,383 3,499 84 1,803

L1T frac 1 0.57 0.27 0.01 0.14

L1TV (Hz) 5,597 2890 1,729 38 739

L1TV frac 1 0.52 0.31 0.01 0.13

Notes:• as we expected, the unthrottled L1T rate is now > 10 kHz• with the ACD throttle on the TKR trigger, the total max rate is 6 kHz. If the max A-dome rate is due entirely to particles that trigger us (it isn’t), naïve scaling results in an orbit-max throttled rate of 9 kHz. Appears we have some margin.• we have a ceiling, finally.


S. Ritz 17

Data set generation planningData set generation planning

• Main challenge is generating backgrounds ~ 50M events (~half-hour equiv.)

– Would take months to generate on a PC– Use SLAC batch (Linux/Solaris) system, requires 2-3 days

• Linux build of pdrApp now running (7/24/01)– Disk space (800 GB fileserver) is in place– DataManager ready to generate and book-keep events

• Since early June, we have been generating increasingly larger photon Since early June, we have been generating increasingly larger photon and background data sets:and background data sets:– iterate to find the bugs and missing pieces (inspect sets of events,

study distributions, effects of cuts, use tools, …)– complete and exercise the infrastructure– Generated 5M event data set ~1 week ago. Generated 5M event data set ~1 week ago. DemonstratedDemonstrated. 50M . 50M

event run gearing up. Using avg flux. [one minor glitch last night]event run gearing up. Using avg flux. [one minor glitch last night]– Exercised tuple preselections and chaining; subset selectivity Exercised tuple preselections and chaining; subset selectivity

(remaining backgrounds at any stage of filtering); readback/re-(remaining backgrounds at any stage of filtering); readback/re-analysis/event display of event subsets last week. Need to finish analysis/event display of event subsets last week. Need to finish event history display.event history display.

NEWNEW

NEWNEW


S. Ritz 18

Failure Modes ModelingFailure Modes Modeling

• Now part of our regular planning (see, e.g., May study on using ACD as L1T throttle). Loss of functionality will be handled in the simulation as an “after-burner” analysis on the large data set. Tools to do this not yet tested.Tools to do this not yet tested. Difficult part is putting awareness into the reconstruction (as we would if failure really happens) – results for PDR will be rudimentary.

•ACD: loss of a single tile in three locations (front corner, front middle, side).

•TKR: impact of loss of a layer on trigger efficiency.

•TKR+CAL loss of a tower.

•Not possible to explore the infinite combinatorics. Highest priority prior to PDR is to do one tile and 1 TKR layer failure: L1T, L3T, effective area impacts.


S. Ritz 19

Actions from 26 July PresentationActions from 26 July Presentation

• “Please make sure low energy positrons (100-300 MeV) are included in a worst-case background analysis.” Done, extended below 100 MeV.• “Please simulate the loss of ½ the light from 18 PMTs on one of the 12 electronics boards.” Withdrawn.• “Please run the simulation with isotropic electrons of 10 GeV and show that no more than 3 in 104 are accepted as gammas, along with the resulting effective area.” On the list.• “Will the trade study to identify potential ACD extension impact the grid design, and will it be complete by September? If not, when? It may impact S/C.” The essential gaps and geometries are now in the new simulation, along with the potentially problematic fluxes. Already see not a likely problem for on-board operations. Answer on ultimate ground-based background rejection analysis will come with the full analysis automatically. First hand-scan of residual events not worrisome, but new cuts are necessary and must be carefully evaluated. The analysis is only part of the issue. The issue should be systematically addressed by the IDT.• Check that the physics of the simulation code to simulate low energy proton interactions. On the list. If this is a problem, it will show up in the analysis of the 50M event run shortly. Tools exist to evaluate.


S. Ritz 20

Much Left To DoMuch Left To Do

• Re-do overall ground-based background rejection selections validation. Deal with new fluxes. Recalculate Aeff and FOV.

• Specifically evaluate combined effect of ACD gap and low energy particle fluxes.

• Demonstrate 10 GeV electron rejection.• PSF evaluation vs. energy, angle, front/back.• Energy resolution.• Finish baselining L3 algorithms

Tools now ready. Help is needed.

Join the fun.


S. Ritz 21

BackupsBackups


S. Ritz 22

L3 Filtering (orbit max case)L3 Filtering (orbit max case)

chim

e

alb

eo_p

alb

deo

_

elec

tro

n

alb

edo

e+

e-

Open L1T rate: 13,025 Hz Throttled L1T rate: 5,968 Hz

ACD_DOCA

cut at 25 || CAL_Hi

caveat: the current version of ROOT interactive cuts is FLAKY! Beware.

Not checked yet!


S. Ritz 23

L3 Filtering (Orbit Max Case)L3 Filtering (Orbit Max Case)

DOCA cut, (#tracks>0||CAL>5 GeV visible): 167 Hz

Track-CAL match

Cal_Fit_errNrm<15.

72 Hz at orbit max => ~30 Hz avg

#xtals

note!>30 Hz even at orbit max due to <100 MeV e+e-. Is that real? No matter, we can deal with it.

Not checked yet!


S. Ritz 24

L3 Filtering (Orbit Max Case)L3 Filtering (Orbit Max Case)

Visible CAL energy CAL Hi Rate: 40 Hz “L3” inspection of CAL_Hi not implemented yet

Fraction of energy in front3 CAL layers

Require fraction in 1st 3 CAL layers >10% OR Cal_Energy > 10 GeV: 34 Hz at orbit max.

Just first looks with new fluxes and geometry. Gamma distributions on the way.

Not checked yet!


S. Ritz 25

Results: statusResults: status

• First peek at PSF looks ~not terrible (much to do!)

total 68%

total 95%

front 68%

front 95%

back 68%

back 95%

100 MeV normal inc.

3.443.44 2.32.3 4.45 5.8

1 GeV normal incidence

0.590.59 1.87 0.470.47 1.62 0.84 2.08

Note: normal incidence PSF is not particularly relevant for physics – just a well-defined comparison point for performance changes and code checking. “Normal” in Table means 0-8 deg bin.

results arepre-pre-pre-preliminary!


S. Ritz 26

Aside: some definitionsAside: some definitions

0 1 2 3 4 50

1000

2000Histogram of Data

lower upper

5 0 55

0

5

y

x

68% 95%

Effective area (total geometric acceptance) • (conversion probability) • (all detector and reconstruction efficiencies). Real rate of detecting a signal is (flux) • Aeff

Point Spread Function (PSF) Angular resolution of instrument, after all detector and reconstruction algorithm effects. The 2-dimensional 68% containment is the equivalent of ~1.5 (1-dimensional error) if purely Gaussian response. The non-Gaussian tail is characterized by the 95% containment, which would be 1.6

times the 68% containment for a perfect Gaussian response.


S. Ritz 27

Experimental TechniqueExperimental Technique

Instrument must measure the direction, energy, and arrival time of high energy photons (from approximately 20 MeV to greater than 300 GeV):

- photon interactions with matter in GLAST energy range dominated by pair conversion: determine photon direction

clear signature for background rejection

e+ e– calorimeter (energy measurement)

particle tracking detectors

conversion foil

anticoincidenceshield

Pair-Conversion Telescope

• instrument must detect -rays with high efficiency and reject the much higher flux (x ~104) of background cosmic-rays, etc.;

• energy resolution requires calorimeter of sufficient depth to measure buildup of the EM shower. Segmentation useful.

Energy loss mechanisms:

- limitations on angular resolution (PSF) low E: multiple scattering => many thin layerslow E: multiple scattering => many thin layers high E: hit precision & lever armhigh E: hit precision & lever arm


S. Ritz 28

e+ e–

Primary Design Impacts of Science RequirementsPrimary Design Impacts of Science Requirements

Energy range and energy resolution requirements set thickness of calorimeter

Effective area and PSF requirements drive the converter thicknesses and layout. PSF requirements also drive the design of the mechanical support.

Field of view sets the aspect ratio (height/width)

Time accuracy provided by electronics and intrinsic resolution of the sensors.

Electronics

Background rejection requirements drive the ACD design (and influence the calorimeter and tracker layouts).

On-board transient detection requirements, and on-board background rejection to meet telemetry requirements, drive the electronics, processing, flight software, and trigger design.

Instrument life has an impact on detector technology choices.Derived requirements (source location determination and point source sensitivity) drive the overall system performance.


S. Ritz 29

Design and SimulationsDesign and Simulations

The GLAST baseline instrument design is based on detailed Monte Carlo simulations.

Two years of work was put into this before any significant investment was made in hardware development.

– Cosmic-ray rejection of >105:1 with 80% gamma ray efficiency.

– Solid predictions for effective area and resolutions (computer models now verified by beam tests). Current reconstruction algorithms are existence proofs -- many further improvements are possible.

– Practical scheme for triggering.

– Design optimization.

Simulations and analyses are all OO (C++), based on GISMO toolkit.

Zoom in on a corner of the instrument

First TKR module plane

module walls

scintillators

front scintillators

gaps, dead areas included

The instrument naturally distinguishes most cosmics from gammas, but the details are essential. A full analysis is important.

proton

gamma ray


S. Ritz 30

X Projected Angle3-cm spacing, 4% foils, 100-200 MeV

Data

Monte Carlo

Simulations validated in detailed beam tests

Experimental setup in ESA for tagged photons:

101 102 103 104

Energy (MeV)

0.1

1

10

Con

tain

men

t S

pace

Ang

le (

deg)

68% Containment95% Containment

(errors are 2)

GLAST Data

Monte Carlo


S. Ritz 31

Instrument Triggering and Onboard Data FlowInstrument Triggering and Onboard Data Flow

Hardware trigger based on special signals from each tower; initiates readout

Function: • “did anything happen?”

• keep as simple as possible

• TKR 3 x•y pair planes in a row**

workhorse workhorse triggertriggerx

xx

Instrument Total L1T Rate: <5 kHz>

Function: • reject background efficiently & quickly with

loose cuts,

• reduce computing load

• remove any noise triggers

xx

x

• tracker hits ~line up

• track does not point to hit ACD tile

L3T: full instrumentFunction: reduce data to fit within downlink

Total L3T Rate: <30 Hz>

• complete event reconstruction

• signal/bkgd tunable, depending on analysis cuts: cosmic-rays~ 1:~few

Spacecraft

OR

(average event size: ~7 kbits)

rates are orbit averaged; peak L1T rate isapproximately 10 kHz. L1T rate estimate being revised. **ACD may be used to throttle this rate, if req.

Upon a L1T, all towers are read out within 20s

Level 1 Trigger

• CAL: LO – independent check on TKR trigger. HI – indicates high energy event ground

Level 2 Processing Level 3 Processing

L2 was motivated by earlier DAQ design that had one processor per tower. On-board filtering hierarchy being redesigned.

On-board science analysis:On-board science analysis: transient detection (AGN

flares, bursts)


S. Ritz 32

LAT Instrument BasicsLAT Instrument Basics

• 4x4 array of identical towers4x4 array of identical towers Advantages of modular design.

• Precision Si-strip Tracker (TKR) Precision Si-strip Tracker (TKR) Detectors and converters arranged in 18 XY tracking planes. Measure the photon direction.

• Hodoscopic CsI Calorimeter(CAL)Hodoscopic CsI Calorimeter(CAL) Segmented array of CsI(Tl) crystals. Measure the photon energy.

• Segmented Anticoincidence Detector Segmented Anticoincidence Detector (ACD(ACD First step in reducing the large background of charged cosmic rays. Segmentation removes self-veto effects at high energy.

• Central Electronics System Central Electronics System Includes flexible, highly-efficient, multi-level trigger.

Systems work together to identify and measure the flux of cosmic gamma Systems work together to identify and measure the flux of cosmic gamma rays with energy 20 MeV - >300 GeV.rays with energy 20 MeV - >300 GeV.


S. Ritz 33

Background Rejection OverviewBackground Rejection Overview

• First developer of background rejection analysis: Bill Atwood (lots of work also by Jay, Toby, Heather, Cathie, Sawyer, Jose, Paul, Taro, SR, …

•Analysis done thus far for two main reasons:

(1) A reasonable way to quote our effective area.

(2) A proof of principle and a demonstration of the power of the instrument design.

• Not the final background analysis! Other techniques are available to reduce the backgrounds further with good efficiency (particularly using TKR pattern recognition). The analysis for the AO response was done in triage mode, and there is much to do now.

• Some science topics may require less stringent background rejection than others. Issues of duration, visible energy range, etc.

Same points also hold for the event reconstruction we have thus far.

Same points also hold for the event reconstruction we have thus far.


S. Ritz 34

Ideally (and usually) cut variable distributions are examined several ways, first to check the distribution is sensible and then for implementing the selections:

1) raw (after triggers, depending on tuple)

2) cumulative - the distribution of the next cut variable after all previous cuts. Note, this order is arbitraryarbitrary (mostly) and the distributions can be misleading, so….

3) “all but”: look at each variable distribution with every cut but this one applied.

4) niche areas: check for effects of each cut in different energy ranges and different angles of incidence. (usually done with merit first)

5) interplay with track quality cuts: the effects of the track quality cuts and the background rejection cuts are not orthogonal: track quality cuts usually help in background rejection somewhat, and background cuts sometimes help clean up PSF. In one case, an “all but” background distribution was empty! Optimize these together.

6) n-dimensionally (usually 2 at a time) : look for correlations and domains of well-clustered S/B for like variables.

Note that a neural net very well addresses (3), (5) and (6). These cuts are not orthogonal, and there is a better space in which to make them.

Background AnalysisBackground Analysis


S. Ritz 35

• try to keep the cuts away from steep areas, or right next to individual events (avoid fine-tuning).

• process is iterative:

With each variable, look at distributions for gammas and background and choose a preliminary cut value.

Scan remaining background events and lost gamma events for adjusting cut and to determine potentially new cut variables.

Check for cut redundancy and correlation. Check impact on instrument performance. Merit is particularly useful here.

As a practical matter, some days are spent mostly improving the rejection and other days are spent mostly improving the gamma efficiency.

Background Analysis (cont)Background Analysis (cont)


S. Ritz 36

AO Overview: Visible (CAL) Energy Distributions at Various Stages

Although the cosmic ray spectrum peaks around 4-20 GeV, the deposited energy is typically much lower.

The region below 1 GeV is the most difficult for background rejection for several reasons.

(38, actually, but who’s counting?)

Note, after all selections, no background events remain with visible energy greater than 200 MeV. This wasn’t easy.

Important: at the time of the large data set generation for the AO, the albedo proton flux was not implemented. It was implemented for the rate studies, but the energy spectrum was wrong. We thought this was pessimistic.


S. Ritz 37

Some references (beyond meeting presentations):

1) DoE proposal (1998)

2) Note of 9 August 1999 (describes cuts and problem areas fairly well, needs distributions included)

3) AO response

however, better documentation is needed and will be done in this round of studies.

STEPSSTEPS

• The famous VETO_DOCA (only for CsI_Energy_Sum<20) - getting better, but still somewhat broca. Needs improvement.

• “Hit pattern” - Surplus_Hit_Ratio, with an energy-dependent application.Surplus_Hit_Ratio>2.25 || (CsI_Energy_Sum>1&&fst_X_Lyr>13) || CsI_Energy_Sum>5.

(note: description uses AO tuple variable names)


S. Ritz 38

• “CAL info” - CsI_Fit_errNrm, CsI_Xtal_Ratio -- keep events w/no keep events w/no CAL info whenever possible.CAL info whenever possible.

• “Track quality” (most recent selections developed by Jose)

• “S/C induced event cuts” - designed to remove cosmics whose primary interaction is in the S/C. This is our single largest residual background!

STEPS (continued)STEPS (continued)

CsI_Xtal_Ratio>0.25||CsI_No_Xtals<1

(CsI_Energy_Sum<1.&&CsI_Fit_errNrm<10.)||CsI_Fit_errNrm<4.||CsI_No_Xtals<1

No_Vetos_Hit<1.5 || (CsI_Energy_Sum>1. && No_Vetos_Hit<2.5) || CsI_Energy_Sum>50.

Quality_Parm>10 (composite track quality parameter, cut effective against low-energy stubs from splash-up)

CsI_eLayer8/CsI_Energy_Sum<0.08 || CsI_eLayer1/CsI_Energy_Sum>0.25 || CsI_Energy_Sum>0.35||CsI_No_Xtals<1

CsI_moment1<15. || CsI_moment1<80.&&CsI_Energy_Sum>0.35||CsI_Energy_Sum>1.||CsI_No_Xtals<1

CsI_Z>-30.||CsI_No_Xtals<1CsI_No_Xtals_Trunc<20.||CsI_Energy_Sum>75.||fst_X_Lyr<12 Only needed in BACK

Surprisingly efficient even at high energy

(Will

work

on m

aki

ng t

his

one c

onti

nuous

in

energ

y w

ith a

norm

aliz

ed m

om

ent)

Background Analysis Steps (II)Background Analysis Steps (II)


S. Ritz 39

Next steps:Next steps:•Use better background flux model. Low energy p and e albedo must be dealt with. Demonstrate high-energy electron rejection.

•Improve low energy Aeff, work on inefficiencies (Surplus_Hit_Ratio, CsI_Fit_errNrm)

• VETO_DOCA: needs work. Mainly a tracking issue. Seed tracks with hit tiles, track quality selections for loop.

• Document, put correct implementation into merit.

• Simplify analysis (make prettier, simpler). Bring in neural net. More sophisticated tracking (downward “ ”) & CAL pattern recognition. (post-PDR)

•Further improvements in rejection (at time of AO, integrated residual background rate was ~ 6% of extragalactic diffuse rate). Also, study background rate differentially (by visible energy bin). More work on upward-going energy events.

• Evaluate impact of limited set of instrument failure modes (see end of talk)

V

Background Analysis To DoBackground Analysis To Do

glast large area telescope: status of system-level performance simulations

Documents