doe/u.s. atlas computing sept. 9, 1999 u.s. atlas computing overview status of atlas computing ...

DOE/U.S. ATLAS Computing Sept. 9, 1999

U.S. ATLAS Computing

Overview Status of ATLAS computing U.S. ATLAS

Project Management Organization Status of efforts

Core software Subsystems Facilities

Schedule FY 00 funding request

Summary


Scale of Computing Effort

Rough scaling of factors of 5 to 1E+3 in relevant parameters from Tevatron Experiments Manpower x5 CPU/event x1E+3 (event complexity) Data volume x10 to x1E+2 (channel count) Distribution of data x10

U.S. effort comparable to scale of Tevatron experiment.

Effort $15M/year


Scales from experience

[email protected] : ATLAS Control : 01sep99 : ht tp://arc.nersc.gov/control/

HENP Computing Challenges

Ex perime nt Da ta ComputeE895 (AGS) 10 TB/yr 600 SPECint95BaBar (SLAC) 400 TB/yr 5,000 SPECint95STAR (RHIC) 266 TB/yr 10,100 SPECint95PHENIX (RHIC) 700 TB/yr 8,500 SPECint95D0 Run II (FNAL) 280 TB/yr 4,075 SPECint95CDF Run II (FNAL) 464 TB/yr 3,650 SPECint95ATLAS (LHC) 1100 TB/yr 2,000,000 SPECint95

Experiment CountriesInstitutes Collaborators Time FrameE895 (AGS) 3 12 49 2000BaBar (SLAC) 9 85 600 2010STAR (RHIC) 7 34 400 2010PHENIX (RHIC) 10 41 400 2010D0 Run II (FNAL) 11 77 500 2005CDF Run II (FNAL) 8 41 490 2005ATLAS (LHC) 34 144 1700 2015


Goals for the next year

Project organization Management Identify areas of responsibility

Integration of efforts into ATLAS Inception/development of software U.S. support facilities

Planning/development of infrastructure

Prepare for reviews


International ATLAS

New Computing Coordinator Norman McCubbin (RAL)

Available full time November Approval vote - ATLAS CB June 10th Responsibility: Core software

New Physics Coordinator Fabiola Gianotti (CERN)

Approval vote - ATLAS CB June 10th

Detector specific sim/reconstruction Organized within subsystems


Architecture Taskforce

Software partitioned into work packages

Katsuya Amako, KEK Laurent Chevalier, CEA Andrea Dell’Acqua, CERN Fabiola Gianotti, CERN Steve Haywood, RAL (Chair) Jurgen Knobloch, CERN Norman McCubbin, RAL David Quarrie, LBL R.D. Schaffer, LAL Marjorie Shapiro, LBNL Valerio Vercesi, Pavia


Architecture T.F. Status

Three meetings so far Directions:

Language: C++ (allow for migration to other e.g. JAVA) Examine GAUDI (LHCb) architecture Adoption of “use cases”

Goals for October Outline of architecture design Appointment of Chief Architect Commission work on prototyping of parts of design Create use-cases, requirement document Define packages and relations (package diagram)


Stages in Software Management

Atlas Software Week 1999.09.01 K.Amako 20

Software Development Process: USDP

Workflows vs. Development Phase - Iterative and incremental [USDP p.11]

ManagementEnvironment

Business Modeling

Implementation

Test

Analysis & Design

Preliminary Iteration(s)

I ter.#1

PhasesProcess Workflows

Iterations

Supporting Workflows

I ter.#2

I ter.#n

I ter.#n+1

I ter.#n+2

I ter.#m

Iter.#m+1

Deployment

Configuration Mgmt

Requirements

Elaboration TransitionInception Construction


Quality Control

Recommend software performance specifications, review process

Makoto Asai, Hiroshima Dario Barberis, Genoa Martine Bosman, Barcelona Bob Jones, CERN Jean-Francois LaPorte, CEA Helge Meinhard, CERN Maya Stavrianakou, CERN


Action on other Groups

National Board Supported platforms Regional centers

Training Network of national contacts for training C++, OO programming GEANT 4 ATLAS Specific


ATLAS/CERN Schedule ‘00

Sept. ‘99 Start of Cashmore/Hoffman review

Oct. ‘99 Report of architecture T.F. Commissioning of prototyping code

Jan ‘00 Start preparations for bilateral agreements

Fall ‘00 Report of Cashmore/Hoffman review MOU preparations


U.S. Participation Frank Paige - Co- convenor of SUSY working group David Malon - Co-leader of database group Craig Tull - Architecture Group Ian Hinchliffe - Leader of Event Generator group David Quarrie, Marjorie Shapiro - Architecture Task Force John Parsons - Co-convenor of Top working group Misha Leltchouk - L Ar simulation coordinator Michael Shupe - Convenor of Background working group Fred Luehring - TRT software coordinator Steve Goldfarb - Muon Database Coordinator Tom LeCompte - Tilecal Database Coordinator Krzys Sliwa - Chair of ATLAS World-wide computing group Frank Merritt - Training contact, Tilecal Reconstruction coord. Bruce Gibbard - Regional center contact John Huth- National Board contact


U.S. ATLAS Computing

NSF, DOE: LHC computing activities are now “projectized”

Implications for U.S. ATLAS: Direct reporting lines through Project

Manager (Bill Willis) and BNL Directorate (Tom Kirk)

Appointment of Associate Project Manager for Computing and Physics (John Huth)

Implications for Agencies: Must clarify reporting lines, operations


Proposed Management

William WillisProject Manager

William WillisProject Manager

John HuthAssociate Project

Manager,Computing and

Physics WBS 2

James ShankDeputy

External AdvisoryGroup

Ian HinchliffeManager, Physics

WBS 2.1

TBN1Manager,SoftwareWBS 2.2

Bruce GibbardManager,FacilitiesWBS 2.3

Craig TullControl/

FrameworkWBS 2.2.1.1

David MalonDabase Systems

WBS 2.2.1.2

Srini RajagopalanEvent ModelWBS2.2.1.3

F. Merritt/J.Shank

Detector SpecificWBS2.2.2

F. MerrittTraining

WBS 2.2.4

Laurent VacavantInner DetectorWBS 2.2.2.1

Keith BakerTRT

WBS 2.2.2.2

Srini RajagopalanEM Cal.

WBS 2.2.2.3

Tom LeCompteTilecal

WBS 2.2.2.4

Bing ZhouMuons

WBS 2.2.2.5

Andy LankfordTrigger/DAQWBS 2.2.2.6

SubsystemsCore Software

S. EfstathiadisSoftware Support

WBS 2.3.1.2

AshkanaziHardware/systems

WBS 2.3.1.1

TBNRemote Sites

WBS 2.3.2

TBN(Price)NetworkingWBS 2.3.3

K. SliwaMONARCWBS 2.2.4

Facilities

TBNCollaborative

ToolsWBS 2.2.3


Management Structure

Reflects flow of deliverables to, from ATLAS

Appointments (2 year renewable terms) Physics: Ian Hinchliffe (LBNL) Facilities: Bruce Gibbard (BNL) + deputy

Issues Software manager

Availability within U.S. ATLAS - hire? Flatter structure for the time being? Bring on soon? Most desirable!


Management Plan

Associate Project Manager Member of E.C. Develop and execute project plan Establish and maintain project organization+Tracking Develop annual budget requests Liason to ATLAS Computing Management Appoint L2 managers Review and approve MOU’s to CERN and Institutes Exercise change control authority Establish advisory committees where appropriate Provide reports and organize reviews


Implications for APM

APM is a time consuming job. Actions for John Huth:

Relief from MDT electronics coordinator (Jay Chapman, U. Michigan)

Relief from U.S. ATLAS Inst. Bd. Chair (Jim Siegrist, LBNL)

Teaching relief - spring terms ‘00 and ‘01 Granted by Harvard University


Level 2 Managers

Appointed by APM, concurrance of Exec. Comm. Members of E.C. (+ APM, + deputy) Two year renewable terms Generic responsibilities

Develop definition of milestones and deliverables Define, with APM, organizational substructure of level 2 Develop, with APM, annual budget proposals Identify resource imbalances within subprojects and

recommend adjustments Deliver scope of subproject on time within budget Maintain cost and schedule Provide reports to APM, PM Liason with counterparts at CERN


Specific Responsibilities

Physics Manager Generators, physics objects, benchmark

studies, mock data challenge

Software Core Detector specific sim/recon Training

Facilities Tier 1,2, networking, support


Project Engineer

Same roles as project engineer’s for construction project Tracking Reviews, oversight Reporting Technical input


Proposed Names

Physics: Ian Hinchliffe (LBNL) Contacted, accepted Policy question: physicists on project

Software: search underway Facilities: Bruce Gibbard (+deputy) Deputy: Jim Shank


Facilities Manager

Bruce Gibbard (BNL) Proposal to add U.S. ATLAS Deputy U.S. ATLAS and RHIC Deputies General agreement with Bruce, Tom Kirk

Begin to fill in other areas Networking Tier 1 Remote sites Support


Detector Contacts

L Ar - Srini Rajagopalan (BNL) Tilecal - Frank Merritt (U.Chicago) ID- Laurent Vacavant (LBNL) Muon - Bing Zhou (U. Michigan) TRT - Keith Baker (Hampton) Trigger/DAQ -Andy Lankford (UCI)


Planning Activities

Writing/preparation assignments for review - next week L2 managers where appropriate (ie. Facilities,

physics) Core Sim/recon Training Management (PMP for computing) MRE/IT “team”

Review in October


WBS Structure

Should be flexible while project definition is underway (level 3+beyond)

Level 2’s should be fixed now Commensurate with management

structure Adopt lead number “2”


High Levels of WBS

Draft WBS 2.1 Physics

Generators, benchmarks, mock data challenges, physics objects

2.2 Software 2.2.1 Core

– Control/Framework,database, event model, analysis tools 2.2.2 Detector specific simulation and recon. 2.2.3 Collaborative tools 2.2.3 Training

2.3 Facilities Regional center, remote sites, networking, support


Near Term Activities/Issues

U.S. ATLAS Web-site Weekly video conferences Support role of BNL Gathering FY 00 requests Advisory group appointment Writing assignments for proposal NSF MRE/IT proposal - Tier 2 centers Discussions of deliverables with ATLAS Interactions with agencies

JOG, Computing review


Software

Core Software Control/Framework (Tull) Database, Tilecal Pilot Project (Malon) Event Model (Rajagopalan)

Detector-specific sim/reconstruction Representatives from subsystems chosen

Training (Merritt) Establishment of OO courses (BNL, U.

Chicago)


General Requirements

Software must last over lifetime of experiment, yet track language changes Well defined interface layers

Maintainability, engineering critical Number of users, use of software

professionals

Adaptability to distributed environments

Learn from experiments working on OO (BaBar, D0, CDF, STAR)


Database

David Malon (ANL) Tilecal pilot project

Tilecal testbeam data in object database Testbed for ATLAS technologies and strategies Early feedback to developers Generalized to other subsystems

Database core software Transient and persistent object mapping Definition of database/control interface Specifications Examine alternatives to Objectivity


Tilecal Model

Argonne National Laboratory

Tilecal test beam analysis (M. Nessi)

Data base

DA QGeant4

Geant3

Interface via tile logical description

DataDump Calibration

analysis

Optimalfilteringanalysis

Histogramming

Physics analysis

JAVABrowser

...


Database Schedule


Database Milestones

Jan ‘00 Survey of mapping strategies

Feb. ‘00 Infrastructure for developers deployed

April ‘00 Validation of strategies in testbed

July ‘00 Database management infrastructure defined

Oct ‘00 Infrastructure for distributed access deployed

Jan ‘01 Scalability test

Oct ‘01 Beta release


Control/Framework

Craig Tull (LBNL) Working on user requirements document

(w/ Hinchliffe, Shapiro, Vacavent) Market survey of framework systems

Object component model AC++

Compatibility with ATLAS architecture Resource loaded work plan exists Work with A.T.F. for design requirements Already have tested prototype designs


Framework Milestones

Task NameDraft reqs. complete

Compl. initial framework survey

Req. document complete

Requirements review

Freeze CORE language

Alpha release design review

Alpha release

Freeze CORE architecture

Freeze database interface

Freeze platforms for MDC

Beta release design review

Beta release

Freeze distributed architecture

Vers. 1.0 design review

Vers. 1.0 release

MDC start

Freeze platforms for production

MDC complete

Production release review

Production release

Data taking starts

8/27

2/1

2/1

3/1

3/29

6/29

9/28

3/29

3/29

6/4

7/2

10/1

4/1

7/1

9/30

6/2

7/4

1/2

4/5

7/2

7/1

2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 19 2000 2001 2002 2003 2004 2005 20


Framework Schedule

Task NameProject Management

Developer Support

Define requirements

Survey existing frameworks/arch

Survey technologies

Draft reqs. complete

Evaluate dictionary language

Evaluate core language

Domain decomposition

Development tools

Compl. initial framework survey

Req. document complete

Requirements review

Freeze CORE language

Code generation tools

Flow control

Analysis tools interface

Document Alpha release

Alpha release design review

Test Alpha release

Alpha release

Database interface design/proto

Freeze CORE architecture

Freeze database interface

Tull[10%]

Milford[50%]

Tull[15%],Hinchliffe[10%],Shapiro[10%],Vacavant[10%]

Calafiura[15%],Leggett[15%],Vacavant[15%]

Tull[10%],Calafiura[10%],Milford[10%]

8/27

Tull[15%],Calafiura[15%]

Calafiura[15%],Leggett[15%]

Tull[50%]

Calafiura[33%],Vacavant[33%],Leggett[33%]

2/1

2/1

3/1

3/29

Calafiura[20%],Milford[40%],TBN[20%]

Tull[50%],TBN[50%]

Leggett[33%],Vacavant[33%]

All

6/29

All

9/28

TBN[75%]

3/29

3/29

4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 12000 2001 2002 2003 200


One Framework Model

Software Bus (eg. CORBA)

Component C++ Classes/Objects

Component Class Adatpers

Scripting Interface (eg. Tcl, …)

Command Marshalling (eg. SWIG, ...)

GUI Interface (eg. Tk, …)


Event Model

Type of objects that are stored Client’s view of the event Physical organization of the event Client’s interface to the event Mechanism to navigate between objects Transient to Persistent Object mapping


Event Model Milestones

Nov. 99 Review of models in other exp’s Requirements documents

Dec. 99 Resource loaded schedule

Mar. 00 Baseline design

June 00 Alpha release

Aug. 00 Review experience

FY 01 Beta release


Framework Options

[email protected] : ATLAS Control : 01sep99 : ht tp://arc.nersc.gov/control/

Framework Design Classifications

• Finite State Machine - AC++

• Action on Demand - CARF

• Stream/Record/Frame - CLEO

• Simulated Data Flow - Gaudi

• Mobile Agents - JAS

• Object Network - ONCM

• C++ Interpreter - ROOT

• Software Bus - StAF


Some Detector Activities

TRT/ID Put full TRT

simulation into GEANT4

L-Ar Coil, cryos in

GEANT4 (Nevis) Accordian structure

in GEANT4 (BNL)

Tilecal Pilot project


Some Detector Activities

Muon Study of noise in Higgs-> 4 muon Combined performance of ID+muon system

(A reconstruction) CSC into simulation

Trigger/DAQ Comparison of switching architectures

Background studies Optimization of shielding (100 MeV muon

background)


Training

New paradigm of OO programming Training courses (F. Merritt)

Course offered at BNL (near future) Course offered at Chicago

Successful programs seen at other experiments (CDF, D0, BaBar)

Ongoing need for training throughout course of experiment Documentation ATLAS-specific


Software Development

Asymptotic level - est. 10 software professionals

Peak load (circa 2003) est. 16 S.P.’s Extrapolations based on existing

experiments and proposed areas of responsibility, fractional of U.S. participation

Choice of technology can influence actual needs strongly (e.g. BaBar, STAR in database)


Planned Training

All by Object Mentor (BaBar, others) Organized by Frank Merritt Courses approximately 1 week long Aug. 9 - BNL - OO Design - 13 people Sept. 20 - U.C. - OO Design - 15 people Oct. 18 - ANL or BNL - Advanced OO -

10 people Nov. 8 - FNAL - GEANT 4 - 14 people


Facilities

BNL ramping up support facility Taps into RHIC Computing Facility

Major issue of Tier 1/2 facilities Scale of “Tier 2’s”

Size for support staff, infrastructure Computing model for U.S. (e.g. grids) Being addressed in NSF MRE/IT proposal

In the process of developing policy on usage, support of platforms at institutions


Facilities

Tier 1 (Regional Center) BNL Leverages RCF

ATLAS specific needs, however. Primary support function for U.S.

Code release, support Major processing, event store

Personnel scale estimate: Roughly linear ramp from 2 FTE’s (now) to 22 or

more (depending on computing model)


MONARC

Models of Networked Architecture at Regional Centers (ATLAS+CMS) Alexander Nazarenko, Tufts hire Tasks:

Validate simulation models Perform first simulations of LHC architectures After Dec. ‘99, focus on planning for regional

centers Model validation - end of September

Understanding of U.S. computing facilities


NSF MRE/IT Proposal

Tier 2 centers Approx. 5 total 256 node systems 100 TB tape system Low maintenance

Linked by computing grid

Computing professionals Dual role -

user/developers


Review Process

Send proposals to Advisory Group (Aug. 4th) Charge:

Identify areas of overlap/commonality Suggest simplifications/savings Coherency with ATLAS effort Prioritize

Meet with Agencies to establish scale for FY 00, initial request (Now)

Confer with Advisory group on feedback Prepare, review documentation for Dec.

review (mid-late Sept.)


Priorities

Critical personnel People who would otherwise be lost, fulfilling a

critical role

Core software effort Prerequisite to inclusion of sim/recon software Yet, cannot commit to major ramp (no MOU’s)

Support of U.S. efforts (facilities) Critical studies Transition to OO


Priorities

Coherency in development of plan Matching of facilities scope to usage

E.g. database effort, simulations Contiguous/overlapping areas

E.g. event model, database, control/framework


FY 00 Needs

Starting Oct. 1st - will need more than simple continuation of present level Support functions at BNL Deputy Facilities Manager Core support - database

Estimate roughly 4 FTE increment Remaining needs made part of review

process for Dec. Still give estimates for needs beyond

December -> now


Facilities

Support for U.S. users a necessary precondition for effective U.S. participation (like training)

Use of RCF leverages existing facilities Requesting 3 FTE’s Oct. 1

(deputy+support) $50K of equipment Oct. 1 (500

SpecInt95) $500K (CPU, disk, mass storage) - after

review


Training

Necessary precondition to effective U.S. participation.

Must be done now (trained group of physicists)

Substantial pay-back (experience in industry)

NSF request of $75K to subsidize courses ($1k/student + setup)


Leveraging Funds

Groups that have applied for internal funds BNL: Already support, 2 FTE’s for FY 00

(core software) LBNL: Request for support on

control/framework U.Chicago: Advance support for training,

request for software professional to start


FTE Request

Institute ‘99 Oct. 1 Feb. 1 Sum

ANL 1 0.5 1 2.5

BNL 0 2.7 2 4.7

LBNL 1.4 0.8 1 3.2

U.C. 0 0 1 1

Columbia 0.5 0.5 0 1

Tufts 1 0 0 1

Total 3.9 4.5 5 13.4


Schedule

July Propose management structure to E.C., PM Collaboration meeting Tier 1/2 scoping Plans for FY 00 reviewed MRE “White paper”

August Present FY 00 plans to agencies Outline and writing assignments for proposal

(Dec.)


Schedule

September First drafts of proposal

Management PMP Software: Core and recon/sim Facilities Training, collaborative tools

October Revise proposal, review

November Meeting to prepare for Dec. review


Schedule

December (January?) Agency review

January Revise funding plan for FY 00 Begin work on bilateral agreements Ongoing - and beyond January Prototyping code Progress toward baselining Filling in management slots Bilateral agreements


Summary

Project organization Management Identify areas of responsibility

Integration of efforts into ATLAS Inception/development of software U.S. support facilities

Planning/development of infrastructure

Prepare for reviews


Summary

Major points: Oct. 1st FY 00 needs are more than

ongoing (4 FTE) Hardware to augment RCF Training physicists in OO design Continue to fill in management structure

doe/u.s. atlas computing sept. 9, 1999 u.s. atlas computing overview status of atlas computing ...

Documents

atlas computing

database doeu

atlas architectureresource

atlas control

framework schedule doeu

framework model doeu

framework milestones

database schedule doeu