dØ monte carlo challenge
DESCRIPTION
DØ Monte Carlo Challenge. A HEP Application. Outline. The DØ experiment The application The NIKHEF DØ farm SAM (aka the DØ grid) Conclusions. The DØ experiment. Fermi National Accelerator Lab Tevatron Collides protons and antiprotons of 980 GeV/c Run II DØ detector DØ collaboration - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/1.jpg)
November 7, 2001 Dutch DatagridSARA
1
DØ Monte Carlo Challenge
A HEP Application
![Page 2: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/2.jpg)
November 7, 2001 Dutch DatagridSARA
2
Outline
• The DØ experiment• The application• The NIKHEF DØ farm• SAM (aka the DØ grid)• Conclusions
![Page 3: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/3.jpg)
November 7, 2001 Dutch DatagridSARA
3
The DØ experiment
• Fermi National Accelerator Lab• Tevatron
– Collides protons and antiprotons of 980 GeV/c– Run II
• DØ detector• DØ collaboration
– 500 physicists, 72 institutions, 19 countries
![Page 4: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/4.jpg)
November 7, 2001 Dutch DatagridSARA
4
The DØ experiment• Detector Data
– 1,000,000 Channels– Event size 250KB– Event rate ~50 Hz– On-line Data Rate 12 MBps– Est. 2 year totals (incl
Processing and analysis):• 1 x 109 events• ~0.5 PB
• Monte Carlo Data– 5 remote processing centers– Estimate ~300 TB in 2 years.
![Page 5: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/5.jpg)
November 7, 2001 Dutch DatagridSARA
5
![Page 6: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/6.jpg)
November 7, 2001 Dutch DatagridSARA
6
The application
• Generate events• Follow particles through detector• Simulate detector response• Reconstruct tracks• Analyse results
![Page 7: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/7.jpg)
November 7, 2001 Dutch DatagridSARA
7
The application
• Starts with the specification of the events• Generates (intermediate) data• Stores data in tape robots• Declares files in database
![Page 8: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/8.jpg)
November 7, 2001 Dutch DatagridSARA
8
The application
• consists of– Monte Carlo programs
• gen, d0gstar, sim, reco, recoanalyze– mc_runjob
• bunch of python scripts
• runs on– SGI Origin (Fermilab, SARA)– Linux farms
![Page 9: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/9.jpg)
November 7, 2001 Dutch DatagridSARA
9
mc_runjob
• Creates directory structure for job• Creates scripts for each jobstep• Creates scripts for submission of metadata• Creates job description file• Submit job to batch system
![Page 10: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/10.jpg)
November 7, 2001 Dutch DatagridSARA
10
The NIKHEF DØ farm
• Batch server (hoeve)– Boot/Software server– Runs mc_runjob
• File server (schuur)– Runs SAM
• 50 – 70 nodes– Run MC jobs
![Page 11: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/11.jpg)
November 7, 2001 Dutch DatagridSARA
11
![Page 12: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/12.jpg)
November 7, 2001 Dutch DatagridSARA
12
node
• At boottime:– Boots via network from batch server– NFS mounts DØ directories on batch server
• At runtime:– Copies input from batch server to local disk– Runs MC job steps– Stores (intermediate) output on local disk
![Page 13: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/13.jpg)
November 7, 2001 Dutch DatagridSARA
13
File server
• Copies output from node to file server• Declares files to SAM• Stores files with SAM in robot
– @ fnal – @ sara
![Page 14: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/14.jpg)
November 7, 2001 Dutch DatagridSARA
14
farm server file server
node
SAM DB
datastore
fbs(rcp,sam)
fbs(mcc)
mcc request
mcc input
mcc output
1.2 TB
40 GB
FNALSARA
control
data
metadata
fbs job:1 mcc2 rcp3 sam
50 +
![Page 15: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/15.jpg)
November 7, 2001 Dutch DatagridSARA
15
SAM @ NIKHEF
• Stores metadata in database at FNAL– sam declare import_<jobstep>.py– scripts prepared by mc_runjob
• Stores files– on tape at fnal via cache on d0mino– on disk of teras.sara.nl and migrated to tape– sam store --descrip=import_<jobstep>.py
[--dest=teras.sara.nl:/sam/samdata/y01/w42]
![Page 16: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/16.jpg)
November 7, 2001 Dutch DatagridSARA
16
SAM @ SARA
• No need to install SAM • Declare teras directories in SAM as
destination• Access protocol
– May 2001 rcp– October 2001 bbftp– ??: gridftp
![Page 17: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/17.jpg)
November 7, 2001 Dutch DatagridSARA
17
SAM on the Global Scale
• Locate files– Monte Carlo data– Raw data from detector– Calibration data– Accelerator data
• Submit (analysis) jobs on local station• Stores results in SAM
![Page 18: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/18.jpg)
November 7, 2001 Dutch DatagridSARA
18
SAM on the Global Scale
CentralAnalysis
Interconnected network of primary cache stationsCommunicating and replicating data where it is needed.
MSS MSS
MSS
WAN
Stations at FNALCurrent active stations •FNAL (several)•Lyon FR (IN2P3), •Amsterdam NL (NIKHEF)•Lancaster UK•Imperial College UK•Others in US
Datalogger
Reco-farm
ClueD0
LAN
(Others)
![Page 19: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/19.jpg)
November 7, 2001 Dutch DatagridSARA
19
Future Plans for SAM• Better specification of remote data storage
locations, especially in MSS.• Universal user registration that allows different
usernames, uid, etc. on various stations.• Integration with additional analysis frameworks,
Root in particular (almost ready).• Event level access to data.• Movement toward Grid components, GridFTP,
GSI…
![Page 20: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/20.jpg)
November 7, 2001 Dutch DatagridSARA
20
Conclusions
• NIKHEF DØ farm is– easy to use (Antares, L3)– easy to clone (KUN)– part of DØ data grid– moving (slowly) to grid standards
![Page 21: DØ Monte Carlo Challenge](https://reader034.vdocuments.site/reader034/viewer/2022042703/56812a44550346895d8d7170/html5/thumbnails/21.jpg)
November 7, 2001 Dutch DatagridSARA
21
0
20
40
60
80
100
120
140
160
1/1/01
3/1/01
5/1/01
7/1/01
9/1/01
11/1/01
0
20
40
60
80
100
120
140
160
1/1/01
3/1/01
5/1/01
7/1/01
9/1/01
11/1/01
antares
L3