method for monte carlo simulation of experiments

12
METHOD FOR MONTE CARLO SIMULATION OF EXPERIMENTS (David Jackson, Brown University, 060421 / Gaitskell / Comments from Richard Schnee) All calculations to date are based on standard halo parameters, v0 = 220 km/s, vSun = 12 km/s, and escape velocity 600 km/s. The local density is assumed to be 0.3 GeV/cm^3. We repeatedly simulate the results from a dark matter detector, of a given material and exposure in kg*days (see choices below), and a WIMP of a given mass and interaction c-s. In each simulation we Poisson fluctuate the number of dark matter events (based on the calculated expected number of total events) and determine the energy of each event using a cumulative probability distribution based on the calculated underlying WIMP spectra. Every event above a detector specific energy threshold is considered a candidate event and is used in the data analysis. (For Si and Ge detectors, to date, we use thresholds of 10 keV and for Xe we use 16 keV.) We analyze the simulated data with the Extended Maximum Likelihood (EML) Method for an array of possible masses and cross sections. The evaluation grid typically has 600 logrithmically space steps for 3 orders of magnitude variation in cross section (chosen to span likely value) and a similar number for the mass range 10 to 1000 GeV. The grid appeared to be fine enough to avoid artifacts due to course sampling. For each experiment we find the maximum value of the Log Likelihood, ln(Lmax) where L is the Extended Maximum Likelihood estimator. We also find all points on the search grid with Log Likelihood values that are within the contour defined such that ln(L) >= ln(Lmax) - Δln(L). The value for Δln(L) is taken from particle data group’s statistics review, p. 20 (Table 32.2), available at http://pdg.lbl.gov/2005/reviews/statrpp.pdf . E.g. For m=2 (mass and cs) the 90% CL gives a 2 Δln(L) = 4.61. (Comment from Richard Schnee: For the smaller data samples (i.e. only 10 events), the approximation that the log of the likelihood function is chi-squared distributed becomes not so good. However, getting a more accurate result would require significant additional simulations and so may not be worthwhile. Answer: We are looking into this to see if we can quantify the error. Obviously, the size of the 90% CL region becomes large when only a small number of events are detected, so any error on the precise size of the region does not make a significant alteration to the conclusion concerning our ability to determine the underlying WIMP mass.) We keep track of the results from each experiment in a mass vs. cross-section counter matrix where we increment each counter (in the matrix) by +1 if the ln(L) for that mass-cs pair was within Δln(L) of the maximum value of ln(L) for that experiment. We usually track the behavior of 68%, 90% and 99% CL this way. The counter matrices effectively tallies for a given mass-cs how many times that combination of values falls inside the allowed region. e.g. We simulate 1000 experiments, and we find that

Upload: others

Post on 09-Feb-2022

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: METHOD FOR MONTE CARLO SIMULATION OF EXPERIMENTS

METHOD FOR MONTE CARLO SIMULATION OF EXPERIMENTS (David Jackson, Brown University, 060421 / Gaitskell / Comments from Richard Schnee) All calculations to date are based on standard halo parameters, v0 = 220 km/s, vSun = 12 km/s, and escape velocity 600 km/s. The local density is assumed to be 0.3 GeV/cm^3. We repeatedly simulate the results from a dark matter detector, of a given material and exposure in kg*days (see choices below), and a WIMP of a given mass and interaction c-s. In each simulation we Poisson fluctuate the number of dark matter events (based on the calculated expected number of total events) and determine the energy of each event using a cumulative probability distribution based on the calculated underlying WIMP spectra. Every event above a detector specific energy threshold is considered a candidate event and is used in the data analysis. (For Si and Ge detectors, to date, we use thresholds of 10 keV and for Xe we use 16 keV.) We analyze the simulated data with the Extended Maximum Likelihood (EML) Method for an array of possible masses and cross sections. The evaluation grid typically has 600 logrithmically space steps for 3 orders of magnitude variation in cross section (chosen to span likely value) and a similar number for the mass range 10 to 1000 GeV. The grid appeared to be fine enough to avoid artifacts due to course sampling. For each experiment we find the maximum value of the Log Likelihood, ln(Lmax) where L is the Extended Maximum Likelihood estimator. We also find all points on the search grid with Log Likelihood values that are within the contour defined such that ln(L) >= ln(Lmax) - Δln(L). The value for Δln(L) is taken from particle data group’s statistics review, p. 20 (Table 32.2), available at http://pdg.lbl.gov/2005/reviews/statrpp.pdf. E.g. For m=2 (mass and cs) the 90% CL gives a 2 Δln(L) = 4.61. (Comment from Richard Schnee: For the smaller data samples (i.e. only 10 events), the approximation that the log of the likelihood function is chi-squared distributed becomes not so good. However, getting a more accurate result would require significant additional simulations and so may not be worthwhile. Answer: We are looking into this to see if we can quantify the error. Obviously, the size of the 90% CL region becomes large when only a small number of events are detected, so any error on the precise size of the region does not make a significant alteration to the conclusion concerning our ability to determine the underlying WIMP mass.) We keep track of the results from each experiment in a mass vs. cross-section counter matrix where we increment each counter (in the matrix) by +1 if the ln(L) for that mass-cs pair was within Δln(L) of the maximum value of ln(L) for that experiment. We usually track the behavior of 68%, 90% and 99% CL this way. The counter matrices effectively tallies for a given mass-cs how many times that combination of values falls inside the allowed region. e.g. We simulate 1000 experiments, and we find that

Page 2: METHOD FOR MONTE CARLO SIMULATION OF EXPERIMENTS

(mass,cs) of (100 GeV,1e-42 cm^2) is within the 90% CL contour for simulated experiments 55 times (out of 1000). When all experiments are simulated we plot the contour, using the counter matrix data of the median expected allowed region, by finding a contour in the counter matrix, at a height of half number of simulated experiments. i.e. The mass-cs contour within which 50% of experiments would have set a 90% CL (say) or better for that mass-cs combination given some underlying “actual” value of mass-cs. We found it useful to also extract the max and min values of the mass associated with the above contour and then plot the value of the “actual” masses vs the max and min mass values. (Comment from Richard Schnee: A useful alternative to median expected allowed regions is to calculate the Bayesian likelihood of a given point in parameter space given the true point and measurement uncertainties. To calculate that, you average the (correctly normalized) likelihood at each point for each simulation. Then for the upper and lower 90% mass limits, you would take the points that had the largest likelihoods, until you had totaled 90%. Answer: We are working on doing this.) ACTION ITEMS Currently the resolution of the detectors used for the high statistics Monte Carlo runs listed below is assumed to be perfect. Preliminary studies showed that degrading the resolution by ~30% FWHM does not have a significant effect on the conclusions, however, this needs to be added to the full simulations to better quantify the effect. The improvement in the favored WIMP mass range determination, by combining the results from two (or more) detectors was looked at using an earlier version of the Monte Carlo. Unfortunately, we found that the improvement, to leading order, was effectively little different from that obtained if we simply have the total statistics (i.e. event number) from a single experiment. This is because the likelihood contours for the different target materials are pretty similar when simulating the detection of the same number of events. There are modest differences for lighter WIMPs with lighter targets, vs heavier targets, but it was not nearly as dramatic as we originally hoped. We will still have the leverage that theory predicts different event rates (per target nucleus) in different targets, so this can be used to study the underlying theoretical models, and check for possible systematic disagreements. Again, we are now in a position to more fully quantify precisely what improvement there is, and will be running this shortly. We need to talk further with Richard Schnee to implement the Bayesian likelihood method, discussed in test above. It may be interesting to see how varying the underlying astrophysical assumption vary the mass and cs determinations. However, to leading order the WIMP mass is simply dependent on the (v_actual/v_0)^-2 when extracting it from the observed slope in the event spectrum. Similarly, the extracted cs is inversely proportional to the local density.

Page 3: METHOD FOR MONTE CARLO SIMULATION OF EXPERIMENTS

Part I: Comparison of the results of our simulation to similar work to check our method. Figure 1.1 is Richard Schnee’s plot from his 2005 APS presentation. Figure 1.1 To test our simulation we used a 70 GeV WIMP with a cross section of 7.0 e-44 cm^2 and 405 kg-days of exposure for a Germanium detector. We expected 8.036 Candidate Events per detector and ran 1000 simulated experiments. From these simulations we created Figure 1.2 showing the median error contours. The two plots Figs 1.1 and 1.2 look very similar. Figure 1.2

Page 4: METHOD FOR MONTE CARLO SIMULATION OF EXPERIMENTS

Part II: The second part of the analysis is to examine what the median contours look like for determining WIMP properties in three different situations, WIMPs with masses of 60, 120, and 250 GeV. Because experimental accuracy depends on the number of candidate events we examined cases where we had 10, 100, and 1000 potential events for each WIMP mass. To generate 10, 100, and 1000 candidate events we use simulate detectors of 500, 5000, and 50000 kg*Days respectively, by changing the cross section of the heavier WIMPS we hold exposure constant for a given number of events. The 68, 90 and 99 % Confidence limits are plotted in red, green, and blue respectively. Figure 2.1 Blue is 68% Confidence Green is 90% Confidence Red is 99% Confidence Figure 2.2 Blue is 68% Confidence Green is 90% Confidence Red is 99% Confidence

Page 5: METHOD FOR MONTE CARLO SIMULATION OF EXPERIMENTS

Figure 2.3 Blue is 68% Confidence Green is 90% Confidence Red is 99% Confidence Below are the graphs of the median contours for 100 events above threshold in each situation. Again the 68, 90 and 99 % Confidence limits are plotted in red, green, and blue respectively. Figure 2.4 Blue is 68% Confidence Green is 90% Confidence Red is 99% Confidence

Page 6: METHOD FOR MONTE CARLO SIMULATION OF EXPERIMENTS

Figure 2.5 Blue is 68% Confidence Green is 90% Confidence Red is 99% Confidence Figure 2.6 Blue is 68% Confidence Green is 90% Confidence Red is 99% Confidence

Page 7: METHOD FOR MONTE CARLO SIMULATION OF EXPERIMENTS

Finally, we simulate experiments for WIMP of mass 60, 120, and 250 GeV where each experiment detects, on average, 1000 candidate events. Because the confidence regions are small we the results fit on a single graph to aid comparison. Again the 68, 90 and 99 % Confidence limits are plotted in red, green, and blue respectively. Figure 2.7 Blue is 68% Confidence Green is 90% Confidence Red is 99% Confidence

Page 8: METHOD FOR MONTE CARLO SIMULATION OF EXPERIMENTS

Part III: In Part II we saw that for low WIMP masses the 90% (middle) contour lies within a smaller mass range than the 90% contour for heavier WIMP. Increasing the number of candidate events increases our ability to place an upper limit on WIMP mass. Obviously, potential mass values are better constrained with 1000 events than with 10 events. Higher WIMP mass will increase difficulty in characterization as the upper mass limits for higher mass WIMPS are difficult to determine. Additionally the local number density results in fewer interactions for higher mass WIMPs, verse lower mass WIMPS, for a given cross section and exposure. The third part of our work is simulating WIMP mass characterization for a given detector exposure and WIMP cross section, varying WIMP mass. We selected cross section and exposure levels that would give 10, 100, and 1000 Candidate Events for a 100 GeV WIMP in a Germanium detector. The interaction cross section was set to 7.0*10-44 cm^2 and the detector exposures where 562, 5620, and 56200 kg*days. Then we created a list of 75 different WIMP masses running from 19.5 to 1000 GeV. We also set up a list of different detector atomic masses so we could compare Silicon, Germanium, and Xenon detectors. For each WIMP mass and detector type we ran 200 simulated experiments and found the upper and lower mass limit on the 90% confidence contour. We plot the actual mass of the WIMP and the upper and lower the mass limits from each simulation.

Page 9: METHOD FOR MONTE CARLO SIMULATION OF EXPERIMENTS

Figure 3.1 shows upper and lower mass limits for a level of exposure where we expect 10 candidate events, for a 100 GeV WIMP, in our Germanium detector. Plotted are the 90% Confidence limits on WIMP mass. Figure 3.1

Page 10: METHOD FOR MONTE CARLO SIMULATION OF EXPERIMENTS

(Richard Schnee Question: I do not understand the upper limit for Si for fig. 3.1. What is going on there? Answer: In Fig. 3.1 we are trying to place and upper and lower 90% confidence bounds on WIMP masses. For this plot we assume the same cs whatever the mass of the WIMP model considered and then also assume that each detector has an identical exposure in kg-days. The value for kg-days is the one that happens to generate 10 candidate events for a 100 GeV WIMP in a Ge detector given the cs we used. (There are many other ways to “normalize” between different experiments e.g. assume kg-day exposures for each target material that give equal number of events in each target material for 100 GeV WIMPs, or even to assume that the experiments always have the same mean expectation for the number of events at all masses considered.) Given the normalization we did use for the Si and Xe simulations it turns out that we cannot set an upper 90% CL, so the red and blue lines for the upper contours are off-scale. The situation in Fig 3.1 for Xe corresponds to the situation shown in Fig 2.2 where the green region (90% confidence) is ‘open’ on the right hand side, so there is no limit on the maximum mass. )

Page 11: METHOD FOR MONTE CARLO SIMULATION OF EXPERIMENTS

Figure 3.2 shows upper and lower mass limits for a level of exposure where we expect 100 candidate events, for a 100 GeV WIMP, in our Germanium detector. Plotted are the 90% Confidence limits on WIMP mass. Figure 3.2

Page 12: METHOD FOR MONTE CARLO SIMULATION OF EXPERIMENTS

Figure 3.3 shows upper and lower mass limits for a level of exposure where we expect 1000 candidate events, for a 100 GeV WIMP, in our Germanium detector. Plotted are the 90% Confidence limits on WIMP mass. Figure 3.3