Scientific Computing at SLAC

Amber Boehnlein

Scientific Computing: SLUO

Amber Boehnlein Head of Scientific Computing (4/25/11)

SLAC History: FNAL D0 collaboration Running experiments Department Head Simulation Department Head DOE LHC Operations SciDAC LHCC referee

What is Scientific Computing?

On-line

ADC FPGA

Computing Detectors

Off-line

World

•  Accommodate diverse user communities

–  Visitors and resident users

mid-range CPU cycles

Collaboration tools

Data Acquisition

Analysis algorithms

Analysis and Data Mining

Data Management, Storage, distribution Data Cataloging

Simulations Calculations Visualization

It’s the elements related to computation and computers that is needed to meet the SLAC scientific mission.

3 Scientific Computing: SLUO

Recent SLAC Scientific Computing Highlights

§  Excellent Performance of the Fermi Processing Pipeline

§  Performance of the SLAC ATLAS T2

§  Deployment of the Long Term Data Analysis platform for BaBar

§  Scaling Tests for LSST databases and a successful preliminary design review

§  Deployment of 4 pb of storage for LCLS

4 Scientific Computing: SLUO

Scientific Computing Workshop

High Level of Interest 2 day kick-off workshop

100+ attended 50 talks

A comprehensive view of SLAC computing

6 Scientific Computing: SLUO

Types of Computing at SLAC §  Business systems »  Payroll, accounting, procurement and other financial systems

§  Enterprise »  General services that support the entire institution (email, web

hosting, document management) »  Desktops

§  In-house special purpose applications §  Scientific »  The application development and hardware platforms

specifically targeted towards support of the Scientific Mission »  75% of the hardware at SLAC supports Particle and Particle

Astrophysics

7 Scientific Computing: SLUO

Building 50: A Busy Place §  Computing Division Director search in progress §  Developing 5 year plans »  Computing as whole »  Cyber-security »  Scientific computing

§  IT project office in place »  Upcoming projects »  Business side »  Network re-architecture »  Account management

§  Safeguards for cyber-security •  Moving towards a risk based approach •  Nov. 7 Workshop

8 Scientific Computing: SLUO

Hosting Scientific Computing Equipment

§  CD provides the system administration and other services »  As is typical in a multi-purpose lab, those costs are covered by

a funding model. §  Improving the hosting capability in BLD 50 »  3.6MW capable (current peak is 1.7 MW)

•  Distributing power and cooling to the racks »  UPS and generator-0.5MW

§  Stanford Research Computing Center »  2.5 MW facility-125 racks »  0.5 MW for SLAC »  Phase 1-FY14

9 Scientific Computing: SLUO

Computing Systems

10 Scientific Computing: SLUO

Storage  Cluster  Summary  

Values  Row  Labels   Sum  of  #  Servers   Sum  of  #  Drives  Lustre  Storage   20   1351  

KIPAC/Lustre/Orange   10   140  LCLS/Lustre/ana01   6   611  LCLS/Lustre/ana02   4   600  

NFS  Storage   113   3322  ACD/NFS   2   23  BaBar/NFS   39   935  CDMS/NFS   1   46  CTA/NFS   1   46  EPP/NFS   2   64  EXO/NFS   2   92  FGST/NFS   14   376  KIPAC/NFS   12   508  LCD/NFS   5   169  LCLS/NFS   1   22  MarFnez/NFS   2   92  MCC/NFS   2   106  NLCTA/NFS   1   9  SDC/NFS   6   276  SIMES/NFS   1   46  SLAC  ATLAS/NFS   2   78  SLAC  BaBar/NFS   10   263  SUNCAT/NFS   1   12  Theory/NFS   2   19  University  ATLAS/NFS   3   102  University  BaBar/NFS   4   38  

Xrootd  Storage   82   4014  ATLAS/Xrootd   27   1550  BaBar/Xrootd   19   874  FGST/Xrootd   36   1590  

Grand  Total   215   8687  

Row  Labels   Sum  of  #  Servers  Compute/Private   1014  

ATLAS/Private/Batch   8  BaBar/Private/Batch   255  FEL/Batch   80  KIPAC/Batch/Coma   21  KIPAC/Batch/Orange   96  Klystron/Batch   22  LCLS/Batch   60  SDC/Batch   40  SIMES/Batch   69  SUNCAT/Batch   260  SuperB/Batch   103  

Compute/Unmanaged   155  MarFnez/Fire   155  

Grand  Total   1169  

Growing Scientific Computing at SLAC

§  Scientific Computing should sustain the full scope of SLAC scientific objectives

§  Developing 5 year strategic plan for Scientific Computing for SLAC to ensure success

§  Focus on areas »  Where leadership in computing will advance the state of the

science »  Where SLAC has skills and heritage »  That foster collaboration with Stanford Campus and other labs/

universities »  That develop common tools and infrastructure »  That gain efficiencies/reduce complexity

11 Scientific Computing: SLUO

Strategic Planning

§  Governance model—Scientific Computing Steering Committee »  SLAC wide representation collaborating with the Head of

Scientific Computing, making connections across the Lab to meet broader mission objectives and priorities

§  Work with the SCSC and SLAC management to set a vision for 2017 »  With the vision in mind, develop milestones and projects

required to achieve vision. »  Cost, schedule, budget »  Extend funding model

§  Support elements go beyond Scientific Computing »  Work with CD and other service providers.

§  Preliminary feedback has been positive 12 Scientific Computing: SLUO

Scientific Computing Applications Department §  SCA in Particle Physics and Astrophysics Directorate

consolidates application development expertise: »  Broad input and representation from project computing

coordinators »  Common efforts: small experiments and community tools »  Innovation for future programs (DES, LSST, Computational

Cosmology Initiative) »  Looking for input and collaborators!

§  Portfolio »  General Community Software Projects that benefit the global

HEP community and beyond »  Core development of applications that support multiple

experiments. »  Provisioning for the hardware needs to support the HEP

community working at SLAC 13 Scientific Computing: SLUO

Strategic Elements

§  Existing and future community software tools »  G4, xrootd, SPIRES, LCSim, Blackhat, ACE3P,… »  Usually involve several laboratories engaged in development

and user support »  Can be proposal based

§  Core development of Framework and Data Management »  Pipeline processing, data monitoring, visualization,

collaborative tools etc, being applied to LSST, SCDMS, EXO, Fermi, CTA

»  Existing experiments benefit from managed migration of software expertise, which remains available at the laboratory

»  Can work towards community based toolkits on the Open Science model

14 Scientific Computing: SLUO

Vision for 2017: Highlights

§  Data management: »  LSST data center »  Center for cosmological simulation data »  A national facility for the repository for light source experimental

data and associated simulations §  Simulation: »  Large scale theoretical simulations in computational

cosmology, materials, catalysis, »  End to end simulation for light source experiments in

production use. §  Computing hardware systems: »  Cost effective computing hardware facilities: appropriate mix of

dedicated resources, local "cloud" and outsourcing

15 Scientific Computing: SLUO