Propagation of detector systematics

in T2K-ND280

DUNE Near Detector meeting24/02/2016

Anselmo Cervera

IFIC-Valencia

The T2K near detector•0.2 Tesla magnetic field

2

An event in ND2803

FGD1 TPC2 FGD2 TPC3 DsECALTPC2

P0D

P0D-ECAL

SMRD

BrECAL

Systematics4

•tracking eff•hit efficiency•PID

muon kinematics•charge confusion•momentum resol and scale•B field distortions

•tracking eff•michel e- eff•PID

•TPC-FGD matching eff

µ-

π+

π+

external bkg:•out of FV•sand muons•cosmic muons

Items affecting the detector systematic error

FGD1 TPC2 FGD2

beam

TPC3 DsECAL

•TPC-ECAL matching eff

• tracking eff•hit efficiency•PID•EM energy

resol and scale

Systematic sources•In ND280 we distinguish between:

• Reconstructed observables

• Resolution and scale for Momentum, calorimetric energy, ToF, dE/dx

• Efficiencies

• track finding and track-track/shower matching

• hit/cluster finding and track-cluster/hit matching

• michel electron tagging efficiency

• charge-ID

• Normalization

• flux weight, target mass, Pile-up, etc

5

Strategy for propagation•Once systematic sources have been identified, for each of

them we apply the following procedure

6

# STEP DESCRIPTION

1 Identify systematic source parameters In general mean and sigma for the systematic source. Should decide dependencies and binning

2 Identify a control sample (statistically independent of the analysis sample) or external measurement (i.e. Field map)

Sand muons, interactions outside FV, cosmic muons, etc

3 Implement control sample selection

4 Run control sample selection on DATA and MC

5 Compute systematic source parameters using step 4 output

6 Build text file with systematic source parameters for DATA and MC

7 Implement algorithm for propagation using that text file Will depend on the systematic source type (observable, efficiency or normalization)

8 Run signal selection and systematic propagation for MC

9 Compute systematic covariance matrix for a given variable and binning

For example as a function of muon candidate momentum

•This is an example of systematic source parameter computation: TPC tracking efficiency

# STEP DESCRIPTION

1 Identify systematic source parameters Mean and sigma of the efficiency as a function momentum and angle for each TPC

2 Identify a control sample (statistically independent of the analysis sample) Through-going muons from the beam and cosmic muons

5 Compute systematic source parameters

6 Build text file with systematic source parameters for DATA and MC

Since the efficiency is very large and differs very little on momentum and angle a single bin for each TPC is finally used

Systematic source params7

Propagation mathematics•Systematics are propagated numerically using toy-

experiments (pseudo-experiments or virtual analyses)

•Each toy-experiment is defined by a set of random throws (one for each systematic parameter)

•The covariance of the number of events selected in a given bin is computed in the usual way:

8

Cij =1

Ntoys

Ntoys�

t=1

(N ti −N i)(N

tj −N j)

N ti =

Nevents�

e=1

W te,i N i =

1

Ntoys

Ntoys�

t=1

N ti

average over toys# events in bin i for toy t

Propagation algorithms• Variations: The event is modified taken into account the set of

systematic parameters for a particular toy experiment. Then the entire analysis proceeds on the modified event. For example:

• Momentum scale (smear the momentum of all tracks in MC around the nominal)

• The event weight is not modified

• Weights: a weight (which is 1 by default) is assigned to each event. This weight is computed using event truth/reco info and the systematic parameters for the current toy. This is done in two cases:

• when the variation method is not possible

• Imagine for example the track finding efficiency in one of the TPCs. If the efficiency is larger in data than in MC we can’t easily add a new track into the MC (see slide 7)

• for global normalization parameters (flux, target mass, etc.)

9

Variation systematics•These are associated to reconstructed observables as

momentum, dE/dx,ToF, calorimetric energy, etc

•This is the most straight forward way of propagating the systematic since the selection just proceeds on the modified event. They are selection independent

•For example for the momentum scale, for a given toy t the momentum of each track is varied according to:

10

momentum scale uncertainty

Gaussian random number for toy tG(0,1)

original momentum

varied momentumfor toy t

p(t) = p · (1 + αt · δp)

Weight systematics•In T2K-ND280 we use this method to propagate efficiency-

like and normalization systematics

•Weight systematics are computed only on the events that pass the selection

•By construction they are selection dependent

• Although the propagation method can be kept independent the objects that enter in the propagation are selection-dependent

11

Efficiency-like•As an example of eff-like let’s take the TPC tracking eff

• loop over all relevant true particles for a given selection: with a true segment in a given TPC and a minimum length

• Check association to a reconstructed track in that TPC

• If found apply EFFICIENCY weight

• If not apply INEFFICIENCY weight

• Where ε’data is the varied efficiency for a given toy

• The final weight for each event is the product of all true parts

12

ε�(t)data = (rcs + αt · δrcs) · εMC

uncertainty on CS eff ratio

Gaussian random number for toy tG(0,1)

original Control Sample data/MC eff ratiovaried data efficiency

MC efficiency

Normalization•As an example let’s consider the FGD mass uncertainty

•The weight for each event is computed as:

• For events with true interaction vertex outside FGD: w=1

• “ “ “ “ “ “ inside “ :

13

W (t)e = 1 + αt · δM

FGD mass uncertainty

Gaussian random number for toy tG(0,1)

original weight

varied momentumfor toy t

How is done in practice ?•In practice we use HighLAND, the T2K-ND280 analysis

framework which has tools for systematic error propagation

•The user should provide:

• A text file containing the systematic source uncertainties

• A class inheriting from SystematicVariationBase or SystematicsWeightBase to do the propagation (all machinery available. Th user should provide <100 lines of code)

• Vary a given observable for variation systematics

• Compute a weight for weight systematics

• In the next slides I give a very short summary of HighLAND, but I could give a dedicated talk in another meeting

14

What HighLAND is• HighLAND, High Level Analysis at the Near Detector, is the

T2K ND analysis framework

• Has decreased considerably the learning curve and speed up analysis development in T2K

•Highly optimized, thread safe, compiled c++ code and run on the shell command line (not as root macro)

• It has been proposed for DUNE and it was well received. We are implementing a prototype for DUNE now

•Detailed talks at previous DUNE meetings:• FD sim/reco 23/11/215: https://indico.fnal.gov/conferenceDisplay.py?confId=10882

• LBL 24/11/2015: https://indico.fnal.gov/conferenceDisplay.py?confId=10861

• S&C 15/12/2015: https://indico.fnal.gov/conferenceDisplay.py?confId=11030

• CM 13/01/2016: https://indico.fnal.gov/conferenceOtherViews.py?confId=10276

15

•Event loop, tools for event selection and systematics propagation

•Tools for drawing the analysis results

•Data Reduction functionality

•Tools for incorporating specific analyses into the framework

• Extensible event data model

• Hierarchy of analyses depending on each other

What HighLAND provides16

Analysis flow17

Apply corrections

Apply systematic variations

Apply event selection cuts

Compute systematic weights

event looptoy exp.

loop

Drawing Tools

Event

Output File

Final Plots

Use HighLAND drawing functionality in ROOT macro or ROOT command lineHighLAND

Modify the event (only nominal values) to account for well known residual data-MC differences.

Modify the event to account for detector/flux/physics uncertainties that affect the selection

Can be input for oscillation, x-section, ... fitters

Input File

Covert input format into HighLAND event model with InputConverter’s

HighLAND for DUNE• Ideally same framework for all components: FD,ND, prototypes

• This is possible because:

• HighLAND can accept any input format

• The basic event model can be extended by the user to match the requirements of its particular analysis

• Benefits of common framework

• Moving from one group to another should be easier

• Correlated systematics between near and far detector

• People from different groups would speak the same language when talking about selections, systematics and their associated technicalities

• Time scale

• My guess is that we could have something working and committed to the git repository before the next collaboration meeting

18

Input Data•The Input data for HighLAND can have any format (in

T2K we use root files)

•For DUNE either Art event or AnalysisTree or ...

•The input file information is dumped into the HighLAND data classes (event model) by InputConverter’s, one for each input file type

•Once the information is propagated to those data classes, all analyses are independent of the input format

• Input files should be as small as possible to gain in speed and portability

•HighLAND provides a new level in data reduction as we will see later

19

T2K-HighLAND event model20

Particle

Vertex

Cluster

Hit

Event

TrueParticle

TrueVertex

DetCrossings

Trackor

Shower

In T2K we have something like thisbut it could be different for DUNE

Particle3D point of

entrance/exit of each subdetector

Base Event Model

Extended Event Model

A reconstructed particle can be a collection of

several segments

For some analyses that need hits or clusters

BeamB

EventInfoB

DataQualityB

Run, Subrun, Event numbers, etc

POT, spills, etc

Beam, detector quality flags

List of systematics in T2K•This is the list of 29 detector systematics propagation

methods implemented in HighLAND for T2K

•Not all selections use the same systematics but most of them are common to many selections

•Many of them could be reused in DUNE ND

21

Plots with systematic errors22

draw systematicerror bars error style for MC

using Experiment class

variable binning

Covariance Matrix•binning: 3 theta x 5 momentum x 6 samples = 90 bins

•Cov matrix is computed at plotting time (all info in the tree). Thus the user can change cuts, binning, etc

23

1000throws

official T2K νµCC-0π x-section

Plans• We are currently implementing a prototype for DUNE:

• First step is to decouple detector dependent and independent code in the T2K version. Almost done

• Next step is to create an Analysis Event Model (AEM) for DUNE near detector

• Need to understand recon event model and analysis requirements

• Finally we need to convert the DUNE reconstruction output file into the HighLAND event model

• We already have some files to play with

• We plan to have a working version committed to the git repository for the may CM

24

backup

Event Selection I•It’s a collection of “steps” (cuts and actions)

• Each step inherits from the base class StepBase

• It has a single method Apply, which returns true or false (only relevant for cuts)

•Each selection inherits from SelectionBase, which has a main mandatory method DefineSteps

26

Event Selection II•Example of action (fills the box ...)

•Example of cut (uses the filled box ...)

27

The box is used to pass info

from one step to another

DrawingTools•This is one of the framework classes which can be accessed

from a ROOT macro or command line

• It is initialized with a micro-tree file (HighLAND output)

•When opening a root session the HighLAND classes are already visible so you just do

•Now you can start doing plots

28

Distributions•This plot shows the muon PID likelihood before the muon

PID cut, broken down in “particle” categories

29

variable to plot binning color category events passing cut 4tree name

Data/MC comparisons•We initialize a DataSample class with a micro-tree file

30

Efficiencies & purities•Efficiency and purity after each cut in the selection

31

•Efficiency as a function of true muon momentum

32

Using Experiment class33

Create DataSamplesfor data and MC

Create SampleGroupsone per period

Add SampleGroupsto the Experiment

Create Experiment