a general assistant tool for the checking results from monte carlo simulations koi, tatsumi...
Post on 20-Dec-2015
216 views
TRANSCRIPT
![Page 1: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/1.jpg)
A general assistant tool A general assistant tool for the checking results for the checking results
from Monte Carlo simulationsfrom Monte Carlo simulations
A general assistant tool A general assistant tool for the checking results for the checking results
from Monte Carlo simulationsfrom Monte Carlo simulationsKoi, TatsumiKoi, TatsumiSLAC/SCCSSLAC/SCCS
![Page 2: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/2.jpg)
Contents• Motivation• Precision and Accuracy• Central Limit Theorem• Testing Method• Current Status of Development• Summary
![Page 3: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/3.jpg)
Motivation• After a Monte Carlo simulation, we get an answer. Ho
wever how to estimate quality of the answer.
What we must remember is• Large number of history does not valid result of simul
ation.• Small Relative Error does not valid result of simulatio
n
![Page 4: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/4.jpg)
Motivation (Cont.)• To provide “statistical information to as
sist establishing valid confidence intervals for Monte Carlo results” for users, something like MCNPs did.
![Page 5: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/5.jpg)
Subject of this study• Precision of the Monte Carlo
simulation• Accuracy of the result is NOT a
subject of this study
At first we have to define Precision and Accuracy of simulations
![Page 6: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/6.jpg)
True Value Mote Carlo Results
AccuracyPrecision
Precision and Accuracy• Precision: Uncertainty caused by statistical
fluctuation• Accuracy: Difference between expected value
and true physical quantity.
![Page 7: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/7.jpg)
Subject of this study (Cont.)
• Precision of the Monte Carlo simulation is subject of this study.
• To state accuracy of simulations, we should consider details of simulation, i.e., uncertainties of physical data, modeling of physical processes, variance reduction techniques and so on.
• To make a generalized tool, we have to limit subjects only for precision.
Accuracy is a subject for most of presentations in this workshop.
![Page 8: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/8.jpg)
Principal of this study is
Central Limit Theorem
![Page 9: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/9.jpg)
Central Limit Theorem• Every data which are influenced by
many small and unrelated random effects has normally distribution.
• The estimated mean will appear to be sampled from normal distribution with a KNOWN standard deviation when N approaches infinity.
N
![Page 10: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/10.jpg)
Central Limit Theorem (Cont.)
• Therefore, We have to check that N have approached infinity in the sense of the CLT, or not.
• This corresponds to the checking the complete sampling of interested phase space has occurred, or not.
![Page 11: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/11.jpg)
This is not a simple static test
butcheck of results from nature of Monte Carlo
simulations
![Page 12: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/12.jpg)
Checking Values• Mean
• Variance and Standard Deviation
• Relative error
• Variance of Variance
11
2
2
N
xxS
N
ii
N
iixN
x1
1
N
SSwherex
SR xx
22
2
,
4
2
x
x
S
SSVOV
![Page 13: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/13.jpg)
Checking Values (Cont.)
• Figure of Merit
• Scoring EfficiencyRintrinsic and Refficiency
• Shift
• SLOPEFit to the Largest history scores
TRFOM
2
1
NSxxSHIFT i23
2
N
historiesZERONONofnumberq
![Page 14: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/14.jpg)
What we check?• Behavior of MEAN• Values of R• Time profile of R• Values of VOV• Time profile of VOV• Time profile of FOM• Behavior of FOM• Value of SLOPE• Value of SHIFT
• Effect of the largest history score occurs on the next history.– MEAN– R (Rintrinsic and Refficiency)
– VOV– FOM– SHIFT
Boolean Answer
Numeric Answer
![Page 15: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/15.jpg)
Of cause, Boolean check is carried out
mathematically (statistically)
valuebehavior
time profile
![Page 16: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/16.jpg)
For behaviors and time profiles check
• Derive Pearson’s r from data (results and theoretical values)– r=1(-1), perfect positive (negative)
correlation– r=0, uncorrelated
• null hypothesis is set to uncorrelated• Then, follows student t
distribution of degree of freedom • Checking significance of r with null hypothesis.• Rejection level of null hypothesis is 68.28% (1σ)
212 rNrt 2N
![Page 17: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/17.jpg)
Example• Checking value: Observable Energy of Sampling Calorimeter.• Material
– Pb (Lead)-Scinitillator• Thickens
– Pb: 8.0 mm/layer, Sci: 2.0 mm/layer• Layers
– 120 layers– 1 m x 1 m – interaction surface
• Beam– Electon 4 GeV
• Range Cuts– 1 mm
Pb
8mm
2mm
Sci.
・・・・・・・・e-
![Page 18: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/18.jpg)
SD
9.2
9.3
9.4
9.5
9.6
9.7
9.8
9.9
10
0 20 40 60 80 100 120
SD
Example 100 histories
mean
77
77.5
78
78.5
79
79.5
80
0 20 40 60 80 100 120
mean
R
00.0020.0040.0060.0080.01
0.0120.0140.0160.0180.02
0 20 40 60 80 100 120
R
VOV
0
0.005
0.01
0.015
0.02
0.025
0.03
0.035
0.04
0 20 40 60 80 100 120
vov
SD
VOV
MEAN
R
Does not pass most of Boolean tests
![Page 19: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/19.jpg)
SD
99.29.49.69.810
10.210.410.610.8
11
0 200 400 600 800 1000 1200
SD
vov
00.00050.001
0.00150.002
0.00250.003
0.00350.004
0.00450.005
0 200 400 600 800 1000 1200
vov
R
0
0.001
0.002
0.003
0.004
0.005
0.006
0.007
0 200 400 600 800 1000 1200
R
mean
77
77.5
78
78.5
79
79.5
80
0 200 400 600 800 1000 1200
mean
Example 1,000 histories
SD
VOV
MEAN
R
Does not pass some of Boolean tests
![Page 20: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/20.jpg)
vov
00.000050.0001
0.000150.0002
0.000250.0003
0.000350.0004
0.000450.0005
0 2000 4000 6000 8000 10000 12000
vov
R
00.00020.00040.00060.00080.001
0.00120.00140.00160.00180.002
0 2000 4000 6000 8000 10000 12000
R
SD
99.29.49.69.810
10.210.410.610.8
11
0 2000 4000 6000 8000 10000 12000
SD
mean
77
77.5
78
78.5
79
79.5
80
0 2000 4000 6000 8000 10000 12000
mean
Example 10,000 histories
SD
VOV
MEAN
R
Does not pass one of Boolean tests (SLOPE check)
![Page 21: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/21.jpg)
How to apply Energy Spectrum estimation
etc.• Checking each
confidence level of P1, P2, P3, P4,,,,
• Of course, scoring efficiency becomes low.
P1
P2
P3
P4
E
V/E
![Page 22: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/22.jpg)
Unfortunately, this tool does not work well with
some deterministic variance reduction
techniques.This is come from
limitation of CLT (means some variance are
required for distribution), so that we can not over
come.
![Page 23: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/23.jpg)
And some simulations becomes deterministic
without awaking of users.Please check your
simulation carefully.
![Page 24: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/24.jpg)
Current Status of Development
• Most part of developments has been done.
• Following items are remained under development.– Output of testing result– Class or function for minimization of
multi dimensional functions
![Page 25: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/25.jpg)
We want to include this tool in Geant4
butwhat category is suite for this
tool?
Run, SD, Hits and its collections,
Tally??
![Page 26: A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d4e5503460f94a2d2cc/html5/thumbnails/26.jpg)
Summary• We have successfully developed a gene
ral assistant tool for the checking the results from Monte Carlo simulations like MCNPs.
• Through this tool, users easily know the confidence intervals for Monte Carlo results.