bes iii computing at the university of minnesota dr. alexander scott
TRANSCRIPT
BES III Computing at The University of
Minnesota
Dr. Alexander Scott
6/13/2008 BES III software at MN 2
Acknowledgements Cheng Ping Shen (Hawaii)
– Traveled to Minnesota to help set up BESIII environment
Alexey Zhemchugov (Dubna)– Created pacman installation of BOSS,
advised on setup Pete Zweber, Nick Howell (Minnesota)
– Extensive testing of local installation
6/13/2008 BES III software at MN 3
Talk Outline Release installation and testing Monte Carlo farm implementation Comparison of IHEP and MN Monte
Carlo Analysis support Future issues
6/13/2008 BES III software at MN 4
Installing Releases We use Alexey’s pacman packaging of the
BES environment– Instructions for install located at Alexey’s site– We installed both 6.3.1 and 6.3.3 successfully
release 6.3.1 had some version drift had to manually update packages
– Download and installation takes ~5 hours– Directory structure fully built by package
Execute shell scripts to set environment– Release ready for use
6/13/2008 BES III software at MN 5
Testing Releases
We verified that Alexey install alone sufficient to generate MC – Started over with install from scratch on a
workstation isolated from network– From workstation, successfully:
installed release generated Monte Carlo ran sample analysis package
We are developing documentation on doing remote installs
6/13/2008 BES III software at MN 6
Monte CarloExample (6.3.1)
Generated 1M generic J/ψ decays
Used getacAlg analysis package to reconstruct J/ψ→(ηc)
Selected candidates based on good tracks, showers
Compared variable distributions between IHEP and MN
Thanks to Cheng Ping
6/13/2008 BES III software at MN 7
Example 6.3.3 Generated 1M J/ψ
decays Used ρ analysis
package to reconstruct J/ψ→ ρ
Superimposed MN variable distributions on IHEP
Great agreement
Thanks to Nick Howell
6/13/2008 BES III software at MN 8
Monte Carlo Farm Used the
CLEO-c farm implementation for BES III generation
Only cosmetic changes required so far– Both have a “physics” and
“reconstruction” step– Easily modifiable job structure– Generic distributed job infrastructure
6/13/2008 BES III software at MN 9
Physics Generation File
#include "$ENV{KKMCROOT}/share/jobOptions_KKMC.txt"
KKMC.CMSEnergy = 3.770;
KKMC.CMSEnergySpread=0.0014;
KKMC.InitializedSeed={400081,1,0};
KKMC.NumberOfEventPrinted=5333;
KKMC.GeneratePsi3770=true;
EvtDecay.userDecayTableName = “psipp.dec";
#include "$ENV{BESEVTGENROOT}/share/BesEvtGen.txt"
BesRndmGenSvc.RndmSeed = 297181;
#include "$ENV{BESSIMROOT}/share/G4Svc_BesSim.txt"
G4Svc.RunID = 217804;
#include "$ENV{ROOTIOROOT}/share/jobOptions_Digi2Root.txt"
RootCnvSvc.digiRootOutputFile = Test_633_BES3_PsiPP_1_217804_0.rtraw";
MessageSvc.OutputLevel = 5;
ApplicationMgr.EvtMax = 5333;
6/13/2008 BES III software at MN 10
FarmInfrastructure
3 servers + 40 worker nodes– hyper-threaded dual Xeon 2.66 GHz processors– handle 4 jobs apiece
2 dual quad-core machines– 2.33 GHz processors – 3x as powerful as other nodes– handle 8 jobs apiece
Gigabit connections to all machines 10 TB of storage space Condor handles job distribution
6/13/2008 BES III software at MN 11
Monte Carlo Samples
MN farm samples generated using releases 6.3.1 and 6.3.3 – Statistics
ψ’: 3.5 s, 50 kB (5M/day) ψ’’: 3.5 s, 50 kB (5M/day) J/ψ: 2.7 s, 30 kB (6.5M/day)
Monte Carlo generated – Generic samples:
2M J/ψ, 4.5M ψ’, 1.5M ψ’’
– IHEP comparison samples 1M each of J/ψ, ψ’, and ψ’’
6/13/2008 BES III software at MN 12
MN vs. IHEP Comparison
Attempt to generate identical Monte Carlo to IHEP sample– Use the same scripts, random seeds, and
parameter files as IHEP– Generate 1M each of J/ψ, ψ’, ψ’’– Comparisons made at the track/shower level– Distributions match extremely well
IHEP believes discrepancies due to multiple random number engines
6/13/2008 BES III software at MN 13
Sample Distributions
6/13/2008 BES III software at MN 14
Analysis Support Documentation on analysis support for
users Users can create analysis areas and
compile packages– Set up environment and own workarea– Use cmt to create packages within workarea– Check out TestRelease package to run your own
packages Developing our own MC testing package
6/13/2008 BES III software at MN 15
Issues for the Future
6/13/2008 BES III software at MN 16
Network Transfers Transfer speeds differ between IHEP→MN
and MN→IHEP– 30 vs. 300 Mbps using iperf w/ 5 MB window
Require TCP/IP windows to be opened or transfer limited by latency– Investigating dynamic window sizing for scp
operations– Would like better transfer method
6/13/2008 BES III software at MN 17
Actual Run Simulations
Some questions remain before simulations of data– Based on CLEO-c, some components missing
Random triggers? Constants? Run by run info?
– Triggers and runinfo can be handled by existing infrastructure
– Manual installation requires constants database setup
Not a part of pacman installation Already present or not yet integrated?