simulation project overview gabriele cosmo, cern/ph-sft simulation project leader lhcc...

Simulation Project Overview

Gabriele Cosmo, CERN/PH-SFT

Simulation Project Leader

http://lcgapp.cern.ch/project/simu

LHCC Comprehensive Review of LCG

November 23, 2004

LHCC Review , Nov 2004 Slide 2 Gabriele Cosmo, CERN/PH-SFT

Simulation Project

Simulation framework Interface to multiple simulation engines (Geant4, Fluka) and

geometry models exchange Geant4 team participating

Aligned with and responding to needs from LHC experiments, physics validation, simulation framework

Fluka team participating Framework integration, physics validation

Simulation physics validation subproject Assess adequacy of simulation and physics environment for LHC

and provide the feedback to drive needed improvements Generator services subproject

Generator librarian; common event files; validation/test suite; development when needed (HEPMC, etc.)

WitoldPokorski

JohnApostolakis

AlfredoFerrari

AlbertoRibon

PaoloBartalini


Project Organization

Simulation Project Leader

Framework

WPWP

WP

Geant4

WPWP

WP

Flukaintegration

WPWP

PhysicsValidation

WPWP

WP

ShowerParam

GeneratorServices

WPWP

Subprojects

Work packages

Geant4Project

FlukaProject

ExperimentValidation MC4LHC


Simulation Framework – General goals

Support the physics validation activities and test beam simulations Provide flexible infrastructure for efficient development and for usage of detector

simulation applications Provide means for interchanging geometry descriptions between different

applications Provide flexible, LCG-oriented interfaces to simulation toolkits

Agreed approach: set up a simulation infrastructure based on Geant4 and Fluka via Flugg (which uses the Geant4 geometry)

Existing Geant4 benchmarks/simulation-validation setups can be run with Fluka with minimum effort

Idea of creating a generic framework for Geant4 and Fluka for full detector simulation setups abandoned due to lack of strong interest from the experiments (except for ALICE which has its own solution based on VMC)

Interest in having a common tool to be applied for specific physics validation studies

Geant4/Fluka physics validation will be treated on a case by case basis


Simulation Framework – Status & plans

Complete Geant4+Fluka+Flugg setups to meet physics validation needs First example of usage for test-beam validation studies provided (pixel detector

simulation) New use-case examples to be provided, together with proper documentation and

guidelines. Expected removal of PEMF pre-processing stage in Fluka Extensions to Flugg to meet the best achievable level of automation

Establish a persistent exchange format for geometry descriptions: GDML Development motivated by several use-cases:

Debugging, visualization, XML-based, readability, low-overhead geometry exchange, … Status after recent developments:

Schema complete enough to describe realistic geometries like the LHCb detector The Geant4 reader/writer has been developed and tested on realistic (LHCb) geometries

Immediate plan: Further extend the schema to support more complex geometries (ATLAS and CMS)

Short/Medium term: Implement reader/writer for Root

Developed concrete milestones reflecting the program above The program described makes maximum use of existing work Manpower currently assigned to the project is just 0.5 FTE !


Simulation Framework – Longer Term

Increasing interest from different experiments in Python-based frameworks (because of large flexibility)

LCG tools (LCGDict/PyLCGDict) provide excellent starting point for development/study of ‘pythonized’ simulation applications

First tests (creating LCGDict for Geant4 and running Geant4 via PyLCGDict) look very promising

Geant4 expressed interest in the Python technology Require revision of public interfaces and classes exposure safety LCG tools (Pool/Root) could also be applied as persistency mechanism

Simulation environment Provide a working example of Python based (Geant4) simulation

application using LCGDict and PyLCGDict Investigate the possibility of using LCG persistency framework also for

geometry descriptions


Geant4 in production

Three LHC experiments (ATLAS, CMS, LHCb) now using it successfully in production OSCAR (CMS), Gauss (LHCb) and ATLAS’s Geant4-based simulation programs are the

production tools OSCAR and Gauss have replaced G3 based simulation

Substantial productions (numbers from Oct 20th Application Area Meeting presentations) ATLAS DC2 (summer 2004) produced 12M events Oscar (CMS) : 35 M pp interaction events, and first 100 Pb-Pb events Gauss (LHCb): Over 200 M events simulated

In production use demonstrated low crash rate (and decreasing with new releases) Rate decreasing from 1/10K events (5.2, CMS) to 1 per Million events (6.1, LHCb)

G4 team addressed issues found in test productions

The Geant4 LCG/SI sub-project and the Geant4 Collaboration LCG/SI/G4 responsible for CERN/LHC participation in Geant4 Work plan integrated with overall Geant4 plan

Geometry and tracking in field, Physics: hadronic and electromagnetic, testing and release, coordination

Collaborating closely on validation, infrastructure and robustness With Physics Valid. on validation, and with Framework on geometry exchange Worked on shared infrastructure (testing, portal) with SPI: bonsai, Savannah With experiment simulation and physics teams on robustness, integration and validation


Geant4: 2004 goals

Feb 2004 – Savannah/SPI prototype portal for problem reporting system for Geant4 Prototype and assessment delivered; under evaluation in the Geant4 Collaboration

Mar 2004 - Release 6.1 (Contributions in several areas) focused on improving production usage in LCG experiments

Jun 2004 – Scheduled release 6.2 (Contributions) focused on better use of computing resources, including performance and memory use,

and refinements to specific physics models, persistency and windows support Sep 2004 - Development release

included additional geometry volume registration, refinements to physics models supported CLHEP 1.9, and still compatible with CLHEP 1.8

Oct 2004 - First consolidated acceptance suite for LHC applications Suite of simplified test-beam setups created, and being deployed

Dec 2004 - Release 7.0 (Contributions) Release 7.0 contributions focus on improvement of physics models and additional

geometry functionality Dec 2004 - Prototypes & Process Improvements

prototype 3D string fragmentation; ensure maintenance and improve examples, system tests and physics lists

Dec 2004 - Geant4 validation in LHC production (added May 2004) documenting results, response to feedback, status of production use of Geant4


Geant4: some highlights

Geant4 reliability in production: crash rate low and decreasing CMS (Geant4 5.2, 1 crash per 10,000 events) ATLAS (Geant4 6.0 patch1, ~1 crash per 1 Million events) LHCb (Geant4 6.1, ~1-2 crashes per 1 Million events)

Support and maintenance Addressing issues found in LHC experiment production

providing high job ‘robustness’ (less than ~1 per mille job failures in 6.1, 6.2) Code improvements to help identify problem conditions

In hadronics and geometry (Geant4, releases 6.0 & 6.1) Creation of ‘statistical testing’ suite

Automated physics comparisons in simple test-beam-like setups Being deployed for validation of release 7.0 (December 2004)

Requires significant computer resources New and improved physics models

And improvements in EM & hadronics (Geant4, releases 6.1 & 6.2) Refinements & more functionality in kernel

E.g. enabling experiments to easily construct detectors Reflections, divisions, …

New fast shower capability (a-la GFLASH) Integrating efforts in LHC experiments into Geant4 toolkit

New addition, just scheduled for Geant4, release 7.0


Geant4: potential future goals

Provide continued support and maintenance Address queries on physics, geometry, tracking issues

Solve problems and respond to other issues / reports Sustain existing physics use cases and enable emerging LHC-related uses

Enhance physics Address requirements for sub-1% EM stability and precision Address precision needs for combined calorimetry in full detector simul. / LHC exp. Address issues of pre-calibration using Monte Carlo (ATLAS) Further enable use of radiation studies, addressing needs for data and physics lists

Improve and extend ‘automated’ release validation Extend and refine comparisons

Within available computing resources Address requirements for geometry and improved persistency

Improvements to Boolean solids, to address robustness issues Enable exchange format (GDML) and ‘direct’ persistency (POOL / ROOT) for Geant4

Address new requirements for radiation applications Improve ‘standard’ tallying and extend with additional observables

Continue to improve CPU performance Address hot spots identified in 2004 and bring new tools into use


Geant4: manpower in 2004

Effort currently 9.0 FTE from Staff/ Associates

/Fellows ~1.0 visitors

Expertise and effort are required to address ‘expert’ inquiries

Need to maintain expertise Balance between support/maintenance

and development Increased effort on support and

maintenance New requirements necessitate

refinements & developments. Continuity of effort depends

On the expected continued availability of associates & fellows.

Work Area Staff/Assoc/Fell

Coordination 1.00

Hadronics 3.00

Geometry / Biasing 2.75

EM Physics 0.50

Testing and Soft Man 1.75

Total 9.00

Move of 0.25 FTEs to simulation project leadership pushed some geometry deliverables to 2005


Fluka Integration

Fluka is an official joint CERN-INFN project since January 2004 Fluka development is not an LCG activity

Participation involves Integration of Fluka as a simulation engine in the simulation

framework with FLUGG Physics validation

Working with the physics validation subproject – simple benchmarks, test beam (manpower issues !)

Activity is led by Alfredo Ferrari Fluka source code public at CERN by end of 2004


Fluka Integration

Current developments Physics

Heavy ions transport and interactions DPMJET-III interface done Electromagnetic Dissociation done Special effects in heavy ion Coulomb interactions in progress Development of QMD models in progress

Fragmentation Code structure and user interface (pre-condition for the full release) in progress

Integration of PEMF functionalities in the run time code Name-oriented advanced input interface General clean-up Publication of code documentation as CERN yellow report

Man power issues Physics developments: covered by A.Ferrari and non-CERN collaborators OK “Complex benchmarks” (test-beams): expertise support from FLUKA team, manpower

must come from LCG ? Code structure and user interface (release oriented) : CERN-AB manpower absorbed by

urgent LHC tasks. CERN-PH ending soon critical

Struggling to keep the December-Januarydeadline for the public β release of the full code


Physics Validation

Validation based mainly on Comparisons with LHC detector test beam data “Simple benchmarks”: thin targets, simple geometries

Coordinates a lot of work being done in the experiments, Geant4, Fluka Foster cooperation, coherence, completeness

Output of the project Certification that simulation packages are OK for LHC physics

Understanding strengths/weaknesses/uncertainties of Geant4, Fluka Contributions to systematic errors of measurements

Recommended optimized physics lists Simulation benchmark suite for acceptance and performance

monitoring Final report summarizing the work


Physics Validation – Status

Geant4 electromagnetic physics validated at percent level Simulation physics requirements revisited First round of hadronic physics validation has been completed, with good results

For the observables, in the case of the simple benchmarks (pixels, neutron double differential, pion absorption) there is a reasonable agreement between data and both Geant4 and Fluka

For the calorimeter test-beams, Geant4 describes well the pion energy resolution, σ/E, and the ratio e/

The shape of hadronic showers needs some improvement LCG notes: 1. F.Gianotti et al., CERN-LCG-APP-2004-02

2. A.Ribon, CERN-LCG-APP-2004-09

3. F.Gianotti et al., CERN-LCG-APP-2004-10

4. W.Pokorski, to be released very soon Monthly meetings presenting and coordinating experiment and project work Information, results gathering on web page


Physics Validation – Ongoing and Future Work

Validate precision of Geant4 electromagnetic physics at the permil level More detailed studies of hadronic shower profiles, both at the simulation and

experimental (test-beam data) level. Evaluate the possibility to study another simple benchmark, relevant to LHC,

to validate both Geant4 and Fluka. Man power could be a problem here!

Complete validation for Fluka in the calorimeter test-beam validations (as it has been done for Geant4)

The adoption of FLUGG and strong interaction with the Simulation Framework project is required here

Geant4 studies of background radiation in the LHC caverns in progress Will be soon compared with Fluka.

Man-power: People from the experiments are busy with the new test-beam data From LCG: M. Gallas, W. Pokorski, A. Ribon

all of them involved in other activities! First version of simulation test and benchmark suite delayed to end 2004 Physics validation document delayed to June 2005


Physics Validation Manpower

Witek Pokorski ~ 0.4 FTE simple benchmarks and generic framework (general infrastructure

supporting test beam physics validation) Alberto Ribon ~ 0.5 FTE

project leader, test beam calorimeters with Fluka, simple benchmarks Giuseppe Daquino (Geant4 team) ~ 0.75 FTE

radiation background simulation with Geant4 (biasing), background studies in the LHCb detector environment

Manuel Gallas Terreira ~ 0.75 FTE ATLAS combined test-beam simulation

Total ~ 2.4 FTEs ~ 1.5 FTEs currently shifted to experiments-specific validation activities ~ 0.9 FTEs dedicated to simple benchmarks, test beam calorimeters and

coordination !


Hadronic interactions in ATLAS pixel test-beam

A.Ribon LCG-APP-2004-09


Energy resolution of pions

beamtestEsimulationE


e/π ratio

beamtest

esimulation

e


CMS longitudinal shower profile in HCAL for 100 GeV pions


Generator Services

Goal: guarantee the generator support for LHC WP1 - Generator services library (GENSER) WP2 - Tuning and validation of event generators WP3 - Common generator event files, event database (MCDB) WP4 - Event format, interfaces and particle services

Oversight of MC4LHC Project Leader: P.Bartalini (Florida). Most of the resources from LCG Russia.

Generator library – GENSER – progressing well First “production-quality” version 1.0.0 expected for December ATLAS uses GENSER in production, LHCb and CMS currently validating Addresses LHC experimentalists and theorists both at CERN and in external laboratories Supports many MC4LHC-mandated generators, including all first-priority ones

List as of mid-September - GENSER 0_2_0 release: HERWIG (P.Richardson), PYTHIA (T.Sjöstrand), HIJING (X.N. Wang), ISAJET (F.E. Paige), LHAPDF (M.Whalley), ALPGEN (M.Mangano), COMPHEP (A.Sherstnev), EVTGENLHC (P. Robbe), GLAUBER (V.Uzhinsky), FROZEN (PHOTOS, PDFLIB), 10 generators, different versions

In future, to come: CASCADE (H.Jung), DPMJET, MC@NLO, GRACE, MADGRAPH, NEXUS, PHOJET (Z.Was), SFM, TAUOLA, HERWIG++, PYTHIA 7, SHERPA (F.Krauss)

Internal MC4LHC review in March 2004 Definition of a release policy and roles for the GENSER librarian (A. Pfeiffer, PH/SFT) Involvement of A.Ribon (LCG) to help the project leader as ‘liaison’ person between MC

authors and experiments.


Generator Services - 2

Tuning and validation of event generators Advanced proposal for an event generator validation framework

Basic sanity check in standalone way in GENSER sub packages Physics validation through JetWeb (UK collaboration)

Validation of ALPGEN and HIJING in collaboration with groups in Perugia and Dubna

Common event file production / event database Event file database MCDB α-version deployed Core software (MySQL, POOL, CASTOR (RFIO), …) supported by LCG

Software Project Infrastructure Web Interface with dedicated web server: http://mcdb.cern.ch

α version deployed, being extended to support most common browsers Configuration and book-keeping tested at Fermilab and CMS Identified production centers (Santander, Oviedo)

Leveraging existing CERN production infrastructure Event formats and interfaces

HEPML as proposed meta-data format, based on XML for interfacing matrix element generators and showering/hadronization generators

HEPMC as interface between generators and MC simulators Support the new OO MCs (ThePEG, PYTHIA 7, HERWIG++, SHERPA)

http://mcdb.cern.ch/


Generator Services Oversight/Review

Review of the subproject conducted in March (M. Mangano chair and MC4LHC chair) with participation from experiment experts and many leading generator authors

Review report end of April with clear elements Project is delivering what the experiments need

GENSER in production in ATLAS, under validation in CMS and LHCb Manpower issues to be addressed

Stability in project leader: P.Bartalini (now University of Florida) Librarian positions: A.Pfeiffer LCG librarian for GENSER and defined

release policy Communication with the generator providers

A.Ribon as ‘liaison’ person between MC authors and experiments Primary role of Russian collaborators in the “assembly” process of

features to be included in sub-packages


Generator Services 2004 (and beyond) Milestones

Jan 2004 – Proposal for MCDB deployment in the LCG environment Feb 2004 – LHAPDF generator included in generator library Mar 2004 – Agreed format for event-level generator files Apr 2004 – COMPHEP, ALPGEN and EVTGEN in GENSER Jun 2004 – Proposal for generator event production environment Jul 2004 – Beta version of MCDB in production Jul 2004 – Proposal for an event generator validation framework Sep 2004 – Agreement on parton-level event generator file format Dec 2004 – Generator production framework final release

Plan extends through 2005-2006 to reach complete, production versions of GENSER, MCDB, validation framework, etc. (in blue those not yet verified)

Mar 2005 - First C++ Monte Carlo fully integrated in GENSER Jun 2005 - Generator level production framework "beta version 0_1_0" Jun 2005 - First test of ThePEG and Evtgenlhc integration in Herwig++ Sep 2005 - Production centre integrated in the grid-middleware Sep 2005 - Integration of GENSER in JetWeb Dec 2005 - Generator level production framework "release version 1_0_0" Mar 2006 - MCDB Integration, experiment specific APIs and management of

large files Jun 2006 - Generator level validation framework beta version


Generator Services – perspectives on required resources

Critical situation of resources for future developments in GENSER Particularly concerning the new MC generators

Little resources available for the validation activity Requiring MC experts !

Long-term resources for user-support required User-support expected to grow ! IT-expert for maintenance, MC-expert for development E.g. - FAQ for GENSER did not start for lack of resources …

See manpower tables for 2004 and estimation for 2005 in appendix


Simulation Project Milestones

2003/12: Geant4 release 6.0 2003/12: Simulation physics requirements revisited 2003/12: Simulation framework prototype supporting G4 and FLUKA via FLUGG 2003/12: Proposal for MCDB deployment in the LCG environment 2004/2: LHAPDF generator included in generator library 2004/2: First cycle of hadronic physics validation complete 2004/2: SPI-G4 collaborative infrastructure pilot 2004/3: Agreement on formats for event generator common samples 2004/3: Detector description (GDML) proposal to PEB 2004/03: Geant4 6.1 release - production improvements 2004/4: COMPHEP, ALPGEN and EVTGEN generators included in GENSER 2004/5: Review/prioritization of simple benchmarks for simulation physics validation 2004/6: Proposal for generator event production environment 2004/6: Geant4 6.2 release - resource usage refinements 2004/7: Beta version of MCDB in production in the LCG environment 2004/7: Proposal for an event generator validation framework 2004/9: Agreement on parton-level event generator file format 2004/9: Comparison of LHC calorimeters for EM shower development 2004/10: Geant4 geometry volume registration customization 2004/10:First consolidated G4 acceptance suite for LHC applications 2004/12: Generator production framework, final release 2004/12: Geant4 physics model prototype concluded 2004/12: Second iteration of hadronic physics validation complete 2004/12: Simulation test and benchmark suite available 2004/12: Geant4 7.0 release - physics models and geometry 2004/12: Geant4 validation in LHC production 2005/06: Final physics validation document complete


Simulation Project Major Milestones

2003/12: Simulation physics requirements revisited 2003/12: Simulation framework prototype supporting G4 and FLUKA via FLUGG 2004/2: First cycle of hadronic physics validation complete 2004/2: SPI-G4 collaborative infrastructure pilot 2004/3: Agreement on formats for event generator common samples 2004/5: Review/prioritization of simple benchmarks for simulation physics validation 2004/6: Geant4 6.2 release - resource usage refinements 2004/7: Beta version of MCDB in production in the LCG environment 2004/9: Agreement on parton-level event generator file format 2004/9: Comparison of LHC calorimeters for EM shower development 2004/10:First consolidated G4 acceptance suite for LHC applications 2004/12: Generator production framework, final release 2004/12: Second iteration of hadronic physics validation complete 2004/12: Simulation test and benchmark suite available 2004/12: Geant4 7.0 release - physics models and geometry 2004/12: Geant4 validation in LHC production 2005/06: Final physics validation document complete


Current simulation manpower

Simulation framework 0.50

Geant4 8.65

Infrastructure, performance, management (approx) 2.75

EM / Hadronic physics effort (approx) 3.55

Tracking, biasing, geometry (approx) 2.35

CERN staff (4.15), CERN fell./assoc. (4.25), LCG (0.25)

Physics validation 2.40

Validation specific to experiments 1.50

Generator services 2.35

mgmt. (Florida Univ.), documentation/release (LCG/CERN) 0.60

GENSER dev., Validation, MCDB (LCG Russia) 1.50

Production Framework (LCG Spain) 0.25

FLUKA integration 0.00

Management 0.25

Total 14.05

As of Oct 1


Concluding Remarks

Striking success of Geant4 in the LHC experiments ATLAS, CMS and LHCb now using it in full production

Crash-rate close to zero ! Strict collaboration with Geant4 team for support and new requirements

Very active program in physics validation delivering results and conclusions Shift of manpower to experiment-specific validation activities (ATLAS, LHCb) First cycle of hadronic physics validation completed New program of work for test-beam validation and simple benchmarks to be

defined, also according to the available manpower in the project Outstanding progress for generator services

First production release of GENSER in December Already used in production in ATLAS, under validation in CMS and LHCb Impressive collection of validated generator packages with common event DB

Manpower issues partially addressed Simulation framework program scoped to make maximal use of existing software and

meet the reality of minimal available manpower Prototype setup based on Flugg for physics validation Extension of GDML and implementation of interfaces for Geant4/Root to import

and export geometry descriptions Investigations and prototyping on Python-based interfaces

Fluka integration/validation feeling effects of slow progress due to low manpower (physics validation, Flugg framework)

Expected public source-code release of Fluka soon

Simulation in the LHC experiments


ALICE Courtesy of A.Morsch


ALICE – AliRoot Framework

TGeant3 Currently used in production

TFluka Implementation completed and interfaced to TGeo Testing ongoing

TGeant4 Implementation completed Interfacing to TGeo planned

User Code VMC

Virtual Geometrical

Modeller

G3 G3 transport

G4 transportG4FLUKA transport

FLUKA

Geometrical Modeller (TGeo)

Reconstruction

Visualisation

Using VMC (Virtual Monte-Carlo)

External Generators integrated into AliRoot HIJING for underlying event simulation PYTHIA: jets, heavy flavor

Plans for 2005 Validation of the Geometrical Modeller

(GM) with Geant3 and Fluka Extension of the interface to Geant4 Validation of the Geant4 interface Validation of new AliRoot simulation

with GM, including physics Phasing out Geant3.


ALICE – Physics Validation

Emphasis on simple (thin target) benchmark tests Thin target benchmark tests represent the only direct way of

physics validation Calorimeter tests lead in the best case to a multi-parameter tuning

of MC parameters, but not to a better understanding of the underlying physics

Benchmark tests started by ALICE and continued within the LCG simulation project

Test-beam simulation activities ongoing for ITS, TPC, HMPID Problem of manpower Support from LCG requested, but not obtained


ALICE - Status of Simulation ProductionPhysics Data Challenge ‘04

Test and validate the ALICE Offline computing model: Produce and analyse ~10% of the data sample collected in a

standard data-taking year Use the entire (complicated) system: AliEn, AliROOT, LCG,

Proof… Dual purpose: test of the software and physics analysis of the

produced data for the Alice PPR Structure:

Logically divided in three phases: Phase 1 - Production of underlying Pb+Pb events with different

centralities (impact parameters) + production of p+p events Phase 2 - Mixing of signal events with different physics content

into the underlying Pb+Pb events (underlying events are reused several times)

Phase 3 – Distributed analysis


ATLAS Courtesy of A.Rimoldi


ATLAS - Status

A Geant4-based simulation suite for the ATLAS experiment is in place (and available to developers) since mid-’03 Data Challenge phase-2 (summer 2004)

Geant4 -based, ~12M events (leap of faith) Tests of the ATLAS computing model, distributed production Running right now

Characteristics Based on home-grown simulation infrastructure (FADS) Interfaced to the ATLAS common framework (ATHENA) Persistency based on POOL Unified detector description scheme as in ATLAS (GeoModel)

Hand-coded geometries being phased out ~5 millions of positioned volumes (~300K volume types) All detectors described to a very high level of detail Detector configuration chosen at run time Detailed field map covering the whole detector

Primary numbers from NOVA/MySQL (now from Oracle) UI currently provided by Geant4 (moving to python) Several Physics Lists available at any time (QGSP_GN by default)

Very low tracking cuts for precise physics (20-30m in the calorimeters) About 400Mbytes at run time

~660secs for Ze+e-, ~770secs for SUSY events (PIV, 2.4Ghz) over the whole rapidity range ||6 Heavy ions trial successfully done (few full events generated in |eta|<6 or 3.2


Physics validation programme helped debugging the sub-detectors in an independent way

200 talks in a time span of 3 years Sub-detector integration took several months (2003)

Make sure there are no overlaps and check detector layout Move to new detector description scheme Implement missing bits &pieces (services, dead material etc.) Optimize performance

Running long tests since September 03 Continuous monitoring helps maintaining the program functional

Set up production environment in the meantime Currently running Geant4 6.2 for development work and CTB

simulation

ATLAS - Production setup


ATLAS - DC2 production

Slowed down by instability of the production tools GRID middleware?

Simulation part completed 12M events

Exceptional Geant4 performance and robustness Only two jobs crashed b/c of Geant4 problems !

log file examination pending … NorduGrid sample (3.5M events) completed with no job failure !

35K jobs ! 1M Ze+e- events without any problem !

Continuing now with further tests (Tier-0 production…) which are not relevant to Geant4


ATLAS - Future plans

Production for the ATLAS CTB (ongoing) Continuous production (starting in December) for the physics

community Physics Workshop in Rome, May ’05

Commissioning DC-3

ATLAS “initial” layout “GeoModel-ization” of the LAr calorimeters Implementation of missing bits (shielding, support structures…) Python General refurbishing


CMSCourtesy of A. De Roeck


CMS - OSCAR

In CMS, OSCAR, the OO simulation program based on the Geant4 toolkit, has successfully replaced its Fortran/Geant3 predecessor. It has been validated and adopted by all CMS detector and physics groups. It has proven robust and performant, easily extensible and configurable

Complete CMS geometry: 1M geometrical volumes 35M pp interaction events produced Time ~ 200 sec/event at 2.8 GHz CPU (High ET QCD events) First 100 PbPb heavy ion events

CMS has now entered sustained-mode production: 10M physics events/month through the full chain (simulation, digitization, …, DSTs)

OSCAR 3.6.0 released now with G4.6.2 p1 new validation phase (new field map, new forward detectors, new G4 version, many

improvements…) Deploy for mass production next month Continuing validation of G4 physics (EM+HAD) and performance optimization

important. CMS participates in this common effort


OSCAR/G4: Performance

CPU time of OSCAR 2.4.5 (with Geant4 5.2 p2) is 2xGEANT3 time, but much more sophisticated cut/region scheme more conservative cuts than for CMSIM more detailed physics in G4 improvements implemented by G4 team, and by CMS through more

optimized access to/use of the (new) magnetic field. we have now a 20-30% better performance of OSCAR 3.6.0 (with G4.6.2

p1) compared to the version 2.4.5. So now we are at 1.5 x GEANT3; effort is continuing

Memory usage in OSCAR 2.4.5: 220 MB OSCAR vs 100 MB GEANT3 simulation.

further optimization led to ~110 Mb/event for pp (>500 Mb/event for HI) Crashes occurred with 2.4.5: about 1/10000 for pp events. Mostly hadronic

physics (baryon decays, -nuclear intertactions) latest stress test (800K single particles, 300K full QCD events) showed no

single crash ! Many thanks to G4 for help!


CMS – LCG projects

Other LCG projects GENSER being validated, plan to deploy early 2005 Use FLUKA via FLUGG if PRS groups in CMS

require it Validation: participate with new 2004 HCAL and

ECAL testbeam data (beam energies down to 2 GeV, longitudinal readout)

Future Production of additional O(100K) events with OSCAR

for physics TDR


Validation of the Physics in OSCAR

CERN-LCG-2004-10

Validation of G4 physics in the context of the LCG study group So far: Comparisons of hadronic test beam data with models in G4Also: Comparison of EM physics with test beam data

Studies of energy resolutions, e/ ratios, and shower profiles

GenerallyQGSP modeladequate


Physics Events with OSCAR

View of 180 Higgs ZZ event simulated in CMS Tracker detector

SUSY events (LM4 point: leptons, missing ET)

Samples of “standard sets” of events nowautomatically produced for each new release


Ultimate test: Heavy-Ion Collisions CMSIM: chop event in slices of 100 tracks, run them separately

Needed due to limitations in CMSIM OSCAR/Geant4 can run full events.

Timing is good/Memory > 500 Mbyte (2GB memory machines used) Have now run 100 events without problems

Program CPU (2.8GHz) (min)

CMSIM 230OSCAR 2_4_5

320

OSCAR 3_4_0

180

~ Timing for the first event with55K generator tracks

The first CMS PbPb event with OSCAR/G4


LHCbCourtesy of G.Corti


LHCb - Simulation status

Gauss, the “new” Gaudi/Geant4 based simulation application has been put in production this year main simulation engine has replaced Geant3 based simulation for overall simulation to mimic what will

happen in the spectrometer and understand experimental conditions and performance

integrates two independent phases:1. generation of proton-proton collisions and decays of b hadrons

2. tracking of particles in the detector and interactions with the material

production of “hits" when particles cross sensitive detectors makes use of simulation external engines via dedicated interfaces and services

Pythia, EvtGen, HepMC, Geant4 based on LHCb Gaudi framework, EventModel and Detector Description

Digitization performed by Boole a different application Dedicated background studies also performed with FLUKA


LHCb - Gauss status

Gauss is fully operational complete

all detectors simulated all information needed in later processing provided

stable low crash rate, reasonable CPU time good collaboration with Geant4 team to achieve this ( various iteration before

frozen version for DC04 ) validated as replacement of Geant3 simulation with detailed comparisons under continuous validation with test beam data to tune physics settings

work in different sub-detectors at different paces based on their needs and data (RICH, ECAL,…)

collaboration with LCG physics validation project also to use Geant4 for radiation studies


LHCb - Gauss in production (May – August)

LCG inaction

1.8 106/day

LCG paused

Phase 1 Completed

3-5 106/day

LCG restarted

186 M Produced Events

Still producing data

Data produced in ~ 60 different sites

In each job 3 or 4 sets of 500 events produced with Gauss

Geant4 version 6.1


LHCb - Simulation developments & plans - 1

Data produced with Gauss in DC04 will be intensively scrutinized in the coming months by many physicists

allow to understand, improve the simulation, fix bugs, identify new needs Ongoing developments

Validating GENSER as provider of generator libraries need to be “stable” for use in production collaborating for LHC version of EvtGen

Adopting and validating new versions of Geant4 as they are available provide feed back and new requirements to Geant4 try out new relevant features

Physics validation (tuning) with test beam analysis RICH, ECAL test beams just completed, analysis in progress


LHCb - Simulation developments & plans - 2

Future developments ( in random order ) Introduce more realism and details in Gauss

understand better both the simulation and test beam data physics tuning (… or validation) Further investigation of use of Geant4 for radiation studies Introduce accelerator contributions to background (halo, beam gas, etc.)

Investigate alternative, new paths to those adopted currently in Gauss delta rays production production/tracking cuts per region investigation of tracking in magnetic field (parameters, regions)

Investigate use of other generators for proton-proton collisions Follow LCG simulation projects developments

Appendix


Geant4 in LCG/SI, and collaborations

The Geant4 LCG/SI sub-project and the Geant4 Collaboration LCG/SI/G4 responsible for CERN/LHC participation in Geant4

Maintaining the focus of CERN/LHC effort on LHC priorities Leading role in management, development and infrastructure within the

Geant4 Collaboration Work plan integrated with overall Geant4 plan

Physics: hadronic and electromagnetic Geometry and tracking in field Infrastructure, management, coordination

Collaborating with sub-projects & other LCG project, with experiments teams on validation, robustness and infrastructure

Collaboration with Physics Validation, experiment detector groups Some improvements, updates are direct consequence of this feedback

Collaborated with experiment simulation teams, e.g. on robustness Substantial decrease in job crash rate in past year

Work with the framework team on geometry exchange Worked on shared infrastructure (testing, portal) with SPI


Geant4-based Simulations in ATLAS, CMS, LHCb

Three LHC experiments (ATLAS, CMS, LHCb) are now using Geant4 successfully in production

ATLAS G4 simulation is now the ‘standard’ in ATLAS DC2 (summer 2004) produced 12M events

OSCAR (G4) has successfully replaced CMSIM (G3) in CMS 35 M pp interaction events, and first 100 Pb-Pb events

GAUSS (G4) has replaced G3 based simulation for 2004 production

Over 200 M events simulated Reliable in production, with low crash rate (and decreasing with

each release) Rate decreasing from 1/10K events (5.2, CMS) to 1 per Million

events (6.1, LHCb) G4 team addressed issues found in test productions


Geant4: sample details of highlights

Establishment of ‘statistical testing’ suite Automating comparisons of physics quantities

Simple setups for ‘regression testing’ Simplified, typical HEP detectors without digitization Complementing INFN and SLAC efforts (comparing to NIST

data) and full LHC ‘test-beam’ comparisons Identifying problem conditions

In hadronics by recording initial reaction conditions and printing them out in case of ‘soft’ error or program crash problems are reproduced easily, identified and fixed quickly

In geometry by providing a new ‘check’ mode Giving users information on difficulties with their setup


Geant4: future goals (more detail)

Additional geometry/biasing goals: Enable use of dual geometries for radiation studies Improve detection of problems in ‘user’ geometry models

Better identifying issues in large scale productions Improve field tracking, by taking account (inside a ‘step’) of effect of energy loss on

momentumAdditional potential EM goals: Enable the refining of multiple scattering for pions and electrons Improve modeling of transition radiation detectors

and provide specific customization for ATLAS and ALICE XTR detector setups Enable the use of shower parameterization in sampling calorimetersAdditional potential hadronic goals: Establish a 'data service' for radiation studies Establish the use of binary cascade in conjunction with QGS model Bring CHIPS ‘string’ prototype to fruition Validate ion reactions for use in HEPContribute and enable the Geant4 physics teams to: Establish service for physics lists in emerging/upcoming LHC use-cases Establish physics for exotics (ex.: R-hadrons) in collaboration with experiments Provide additional modeling options for most-sensitive use cases


In-flight Pion Absorption

pi+/-

W.Pokorski - LCGAPP note almost ready


LHC hadronic calorimeter test-beams

extended barrelmodule

extended barrelmodule

η=0.25η=0.65

Crystal 25

F. Gianotti et. al LCG-APP-2004-10

CMS HCAL & ECAL test beam setup

ATLAS HEC test beam

ATLAS Tilecal test beam setup


Generator Services – manpower 2004

Coordination: 0.25 (0.50) WP1. GENSER – 0.70 (1.00)

Development: 0.50 (0.50) LCG Russia Maintenance: 0.10 (0.25) LCG Russia User Support: 0.10 (0.25) LCG Russia

WP2. ThePEG – 0.10 (0.25) Development: 0.10 (0.25) * Alberto * (documentation) Maintenance: 0.00 (0.00) Not yet an issue

WP3. MCDB & Simple Production Framework – 1.25 (1.25) Development: 0.75 (0.75) LCG Russia Maintenance: 0.00 (0.00) Not yet an issue User Support: 0.00 (0.00) Not yet an issue Framework Development: 0.50 (0.50) LCG Spain / CMS Framework Maintenance: 0.00 (0.00) Not yet an issue Framework User Support: 0.00 (0.00) Not yet an issue Production 0.00 (0.00) Spanish CENTRE (to be started 2005)

WP4. Validation – 0.30 (1.00) Validation (Standalone) 0.30 (0.50) LCG Russia Validation (Framework) 0.00 (0.50) Collab with JetWeb (NOT STARTED)


Generator Services – manpower 2005

Coordination: 0.25 (0.50) WP1. GENSER – 1.00 (1.50)

Development: 0.50 (0.50) LCG Russia Maintenance: 0.25 (0.50) * Andreas * User Support: 0.25 (0.50) LCG Russia

WP2. ThePEG – 0.25 (0.25) Development: 0.00 (0.00) Frozen Maintenance: 0.25 (0.25) * Alberto *

WP3. MCDB & Simple Production Framework – 1.75 (2.25) Development: 0.50 (0.50) LCG Russia Maintenance: 0.25 (0.25) LCG Russia User Support: 0.25 (0.25) LCG Russia Framework Development: 0.50 (0.50) LCG Spain Framework Maintenance: 0.25 (0.25) LCG Spain Framework User Support: 0.00 (0.25) Not sure LCG Spain can guarantee Production: 0.00 (0.25) LCG Spain (first tests)

WP4. Validation – 0.25 (1.00) Standalone: 0.25 (0.50) LCG Russia Framework: 0.00 (0.50) Cannot Start if no resources...


ALICE


Example TFluka Testing:RICH 5 GeV Pions

Geant3 FLUKA


PDC’04 Phase I

Centrality name Impact parameter value [fm] Produced events

Cent1 0 - 5 20K

Per1 5 - 8.6 “

Per2 8.6 - 11.2 “

Per3 11.2 - 13.2 “

Per4 13.2 - 15 “

Per5 > 15 “


PDC’04 Phase II

Completed Ongoing


ALICE Physics Data Challenges

Period(milestone)

Fraction of the final capacity (%)

Physics Objective

06/01-12/01 1%pp studies, reconstruction of TPC and ITS

06/02-12/02 5%

• First test of the complete chain from simulation to reconstruction for the PPR

• Simple analysis tools• Digits in ROOT format

01/04-06/04 10%

• Complete chain used for trigger studies• Prototype of the analysis tools• Comparison with parameterised MonteCarlo• Simulated raw data

05/05-07/05 TBD

• Refinement of jet studies• Test of new infrastructure and MW• TBD

01/06-06/06 20%

• Test of the final system for reconstruction and analysis


CMS


View of the CMS detector

Sliced view of CMS barrel detectors

OSCAR

Complete CMS detector geometry includedMore than 1M geometrical volumes

muon detectors


LHCb


Crash analysis

A production job is made of several steps: 3 (signal + 2 minbias) Gauss (detector simulation) – 500 events

or 4 (b-incl + 3 minbias) Gauss – 500 and 3*300 events

Two periods analyzed: 62635 jobs run between 5th May and 28th of June

1111 jobs crashed in one of the Gauss runs for any reasons 29 stopped in the generator step 667 stopped in the Geant4 step which includes GiGa interface

97718 jobs run between 16th August and 19th October 529 jobs crashed in one of the Gauss runs for any reasons 85 stopped in the generator step 278 stopped in the Geant4 step which includes GiGa interface

Bug fixes and protections introduced between the 2 periods for ex. against some looping particles


Gauss in production - DC04

Gauss used in LHCb data challenge (DC04) started 3rd of May 2004 in production environment for the first time (after tests) check its robustness Pythia 6.205 for pp-collision generation EvtGen v5r0 ( LHCb modification from – 00 – 11 – 07 ) for B

decays

Geant4 6.1 for detector simulation three versions of Gauss - fix bugs identified in production

in conversion of MCtruth particle getting stuck during tracking


Debugging Gauss in production

More than 50% of the Gauss crashes are independent of Geant4 For the ~ 50% occuring in the Geant4 step

Detailed information available when a problem occurs for foreseen reason is very useful

As it is done in Hadronic processes Detailed printout for normal situation

very important when developing or adopting a new Geant4 version should be under the control of the user to not clutter log files during

production


Gauss benchmarking

Memory usage ~220 MB and stable (no significant memory leaks)

Virtual Memory for a job

Event #

kB

CPU time performance ( 2.4GHz PIV, gcc 3.2 –O2 )

• minimum bias ~ 22 s/event• BBbar inclusive ~ 65 s/event

simulation project overview gabriele cosmo, cern/ph-sft simulation project leader lhcc...

Documents

physics validation activities

simulation infrastructure

detector simulation

physics environment

case basisgabriele cosmo

testbeam validation

generic framework

lowoverhead geometry