computing at belle ii - desy · computing at belle ii thomas kuhr kit desy ... altmannshofer,...

Computing at Belle II

Thomas KuhrKIT

DESYComputing Seminar

14.06.2010

KEKB: 1 ab-1

SuperKEKB: 50 ab-1

DESY Computing Seminar 14.06.2010Thomas Kuhr Page 2

World Cup

Japan : Cameroon

0 : 0(at 16:00)

(20:30 Italy : Paraguay)

Outline

● Physics at a (Super) B Factory

● SuperKEKB and Belle II

● Belle II Computing Model

● Cloud Computing

● Job and Data Management

● Offline Software

● Resources and Summary

Motivation for B Factories

● 1964, Cronin, Fitch: Discovery of CP violation in K0 system,small effect O(10-3)

➢ 1972, Kobayashi, Maskawa:

➔ CP violation possible, if there are 6 quark flavors

✔ 1974, Burton, Richter: Discovery of charm quark

✔ 1977, E288: Discovery of bottom quark

✔ 1995, CDF, D0: Discovery of top quark

➔ CP violation?

(C)KM-Matrix

● Complex elements Vij

➔ 18 parameters

● Unitarity (VV† = 1) +quark phases

➔ 4 free par.:3 angles + 1 phase (CP)

VCKM =

➔ Flavor changing coupling of W to quark pairs:

➔ CP violation determined by just one parameter➔ Prediction of large CP violation in B0 system

➢ Experimental verification needed B factories→

Physics Objective of Belle

535 x 106 BB

PRL 98,031802 (2007)

✔ Confirmation of KM mechanism of CP in the Standard Model

Baryon Asymmetry in the Universe

● Big bang: Equal amount of matter and antimatter● Today: Only matter left

● Requirements (1967, Sakharov):

➢ Baryon number violation➢ Departure from thermal equilibrium➢ C and CP violation

✗ CP violation in the SM by many orders of magnitudetoo small to generate observed asymmetry

➢ Need sources of CP violation beyond the SM➔ Super B factory

Measurement of CP Violation

● Measurement of phase requires interference betweentwo diagrams

➢ Time dependent asymmetry measurement:

Time Dependent CP Measurement

e- e+Y(4S)

● e+e- Y(4S) B0B0 (50%) B+B- (50%)

● Continuum background

● Asymmetric beam energies ⇒ t = z / c

● Entanglement ⇒ af(t) af(t)

KEK Site

Tsukuba

Mt. FujiNarita

KEKB Performance

➢ World record luminosity: 2.1 x 1034 cm-2s-1

Twice design→➢ 1 ab-1 of integrated

luminosity

Design

Places to Look for New Physics

Time dependent CP violationin b qqs →decays

More Places to Look for New Physics

➢ Charged Higgsin B →

➢ Precision CKM angle measurements➢ Forward-backward asym. in B K→ *l+l- ➢ Direct CP violation (Kπ puzzle)➢ CP violation in D0 mixing➢ Lepton flavor violation

Signal = zero energy

X(3872)

➢ The unexpected

Flavor Physics in the LHC Era

Altmannshofer, Buras, Gori, Paradis and StraubNucl.Phys.B830, 17-94, 2010

● Complementary to direct search for NP at LHC

➔ “DNA test” of NP

Outline

● Cloud Computing

Aim For 50 ab-1

8 x 1035

cm-2 s-1SuperKEKB

SuperKEKB Upgrade: Nano Beam Scheme

e- 2.6 A

e+ 3.6 A

Replace long TRISTAN dipoles with shorter ones (HER).

New damping ring

New IR

TiN coated beam pipe with antechambers

New low emmitanceelectron source

Larger crossing angle 2φ = 22 mrad → 83 mrad

Smaller asymmetry 3.5 / 8 GeV → 4 / 7 Gev

Belle IIBelle IINew Superconducting /permanent final focusingquads near the IP

Projection of Luminosity

50 ab-1

in 2020

Shutdownfor Upgrade

m-2s-1

50 timesBelledata

Belle II Detector

➔ Higher background, higher event rate

Belle II

Nano beam:Synchrotron radiation low,

Touschek background much higher

Belle II Detector: Tracking and Vertexing

Belle II

4 layer siliconstrip detector

Drift chamber

2 layer DEPFET pixel detector

→ much higher event size

Belle II Detector: Particle Identification

Belle IIECL

“Focusing”RICH

Time of propagation

Belle II Collaboration

➢ June 2004: Letter of Intent

➢ March 2008: First proto collaboration meeting

➢ December 2008: New collaboration founded

➔ ~300 members47 institutes from 13 countries

International Collaboration

● Item

You are welcome to join!

Project Status

✔ Preliminary approval by Japanese government in January

● Final decision expected soon

✔ TDR is written (~480 pages)

➔ Reviewed three weeks ago➔ To be published in July

● Belle data taking ends in June

➔ Then upgrade work can start

➢ Well on track to start data taking in 2014

Outline

● Cloud Computing

Belle II Data Acquisition / High Level Trigger

➔ Designed for input rate of 30 kHz

● One challenge:PXD event datareduction1 MB ~200 kB→(all other Belle II detectors together ~100 kB)

Estimated Data Rates

➔ High data rate is a challenge!

Experiment Rate [Hz] Event Size [kB] Rate [MB/s]

High rate scenario for Belle II DAQ

Belle II 6,000 300 1,800

LCG TDR (2005)

ALICE (HI) 100 12,500 1,250

ALICE (pp) 100 1,000 100

ATLAS 200 1,600 320

CMS 150 1,500 225

LHCb 2,000 25 50

Belle Computing: Centralized at KEK

Considerations for Belle II Computing

● 50 times more data in ~10 years

➔ O(50) times more computing resources

Can not expect KEK to provide all resources

Have to enable all Belle II member institutes to contribute

➔ Distributed computing system based on grid services (gLite)

➢ Can benefit from existing LCG infrastructure➢ Profit from experience of LHC experiments and their

well-established and matured solutions

● Tasks: Data processing, MC production, physics analysis

Raw Data Processing

● Huge data size (raw data rate up to 1.8 GB/s) comparable to LHC experiments→

● Write once, read only few times in managed way➔ Tape as storage medium

➢ Most cost and power efficient, but high maintenance effort

● Source of raw data is Belle II detector➔ Store and process at KEK, no replication to remote sites

➢ Would require huge network bandwidth, high operation efficiency, maintenance of tape system

➢ Simpler than LCG model

Raw Data Processing

MC Production

● Run-dependent, generic MC: B0B0, B+B-, charm, and uds events

● Corresponding to N times the real data size

● Produced in managed way

● No input data needed(except generator and background data files)

➔ Well suited for a distributed environment

➢ Including cloud computing resources

➔ Output stored at grid sites where it is produced

MC Production

Physics Analysis

● Need real / MC data in mDST format as input➔ Copy mDST real data to grid sites➔ Replicate MC datasets in a managed way if needed

➢ Produce ntuples on the grid

● Random, uncoordinated access➔ Store mDST real / MC data on disk

● Need fast turn-around time for ntuple analysis➔ Copy them to local resources

➢ Do ntuple analysis on local resources

Physics Analysis

Belle II Computing Model

Raw Data Storageand Processing

MC Productionand Ntuple Production

MC Production(optional)

NtupleAnalysis

Tasks of Computing Facilities

Outline

● Cloud Computing

(Commercial) Cloud Computing

● Resource demands vary with time

● Fair-share can solve this issue only to some extent

➔ Cloud computing allows to buy resources on demand

➢ Well suited to absorb peaks in varying resource demand

Cloud Computing in Belle II

● Risk: vendor lock-in➔ No permanent data storage on the cloud➔ Much less critical for CPU resources

● Large data transfer / storage not cost efficient (now)➔ Use cloud primarily for MC production➔ No data processing➔ Maybe physics analysis

● Accounting issues

➢ Baseline of computing resources provided by the grid➢ Cloud computing is option for peak demands

DIRAC for VM management

➢ Task: Belle MC production

Belle MC Production with DIRAC

Modularity of DIRAC

allows easy integration of

different cloud providers

DIRAC Test Phase I: Only Cloud

250 VMs with 8 cores

Data transferto grid sites

● 120M events (2.7 TB) produced in 10 days

● 0.46 USD / 10k events

DIRAC Test Phase II: Cloud + Local

Cloud (60%)

● 170M events (3.6 TB) produced in 6 days● Amazon Spot Instances 0.20 USD / 10k events→

Local (40%)

DIRAC Test Phase III: Cloud + Grid + Local

Cloud (8 cores)

Local (8 cores)

Grid (1 core)

Outline

● Cloud Computing

Metadata Catalog

● Task: Identify files matching physics selection criteria (dataset)

➔ AMGA (Arda Metadata Grid Application)

● Official gLite metadata service● Support for several

DB backends● Grid-based authentication● Replication

➢ File level metadata: O(60 GB)

➢ Event level metadata investigated: O(20 TB)

Test of AMGAscalability

Analysis Model

● Task: Process selected set of files with given analysis code

➔ Project

➢ Have to make sure each file is processed exactly once

● No solution on project level provided by grid services

● Start with implementation of simple solution Static job – data assignment Job push model

● Then gradually add more complexity Dynamic job – data assignment (a la SAM, CDF/D0) Job pull model (pilot agents)

Project Client

Project Server: Dynamic Data Assignment

✔ Easier job submission (identical jobs)✔ More fine grained monitoring (processed files)✔ Only running jobs at working sites get data assigned✔ Faster nodes get more input files

Automatic load balancing, reduced overall time to finish→

Additionalcentral

component

Same Strategy for MC Production

Input data file is replaced by run number, #events

Outline

● Cloud Computing

Code Management

● Developer with different level of experience,distributed around the world

➔ Need reliable, user-friendly, well-maintainable code

Central subversion code repository with access control Code split in packages, responsible librarians

Nightly and regular integration builds Feature driven releases passing quality tests Compile on different OS (with different compilers)

Documentation: repository browser, doxygen, twiki Language: only C++, python for scripts Coding conventions, code reviews, style formatting tool

Coding Conventions @ TWiki

Framework

● Inspired by frameworks of Belle (basf) + other experiments

✔ Used for simulation, reconstruction, analysis, and DAQ✔ ROOT I/O as data format✔ Software bus

with dynamically loaded modules

✔ Python steering✔ Parallel processing✔ (Backward compatibility with Belle)

Parallel Processing

● roobasf

● new basf2

Python Steering Example

import osfrom basf2 import *

#Register modulesgearbox = fw.register_module("Gearbox")

#Set parametersgearbox.param("InputFileXML",os.path.join(basf2dir,"Belle2.xml"))gearbox.param("SaveToROOT", True)gearbox.param("OutputFileROOT", "Belle2.root")

#Create pathsmain = fw.create_path()

#Add modules to pathsmain.add_module(gearbox)

#Process eventsfw.process(main,1)

Geometry and Conditions Database

● (Time-dep.) parameters stored in XML files and DB

➔ Unique interface to obtain parameters Gearbox→

● TGeo geometry created from parameters Geant4 simulation using G4root Same geometry and parameters used for reconstruction

and analysis

Generic Event Viewer

Outline

● Cloud Computing

Human Resources

Computing group:● ~10 in core

software team● ~15 in core

distributed computing team(including computing scientists)

Hardware Resources

● Preliminary estimates depend on many unknown parameters(accelerator performance, data reduction, performance of simulation/reconstruction, analysis requirements, ...)

Summary

➢ SuperKEKB is planned to deliver O(50) times more data than current B factories

➔ Allows Belle II to discover, identify or exclude New Physics

➔ Huge data volume is challenge for computing Distributed computing system based on grid Cloud computing option for peak demands,

already works for Belle MC production Development of user interface to distributed

resources and offline software in progress

✔ 6th Open Belle II Meeting July 5-7

➔ You are welcome to attend http://belle2.kek.jp

World Cup

Japan : Cameroon

1 : 0(my guess at half time)

computing at belle ii - desy · computing at belle ii thomas kuhr kit desy ... altmannshofer,...

Documents

engineering data management - desy · engineering data...

desy - atlas.kek.jp

das deutsche elektronen-synchrotron desy€¦ · das...

summer studentsª - desy · summer students will work in...

grids@desy investigating grid middleware to be used at desy

desy lks baru

cp violation and rare b decays at belle desy seminar 11...

antivirus @ desy

desy - arxiv.org

desy - psi

desy - overview

zaven akopov (desy -l-) for the inspire collaboration desy

superconductivity - desy

makalah desy

diraction - desy

desy purnamasari

desy superconducting sase fel’s jean-paul carneiro desy...

desy overvie · desy overview welcome summer students 2010...

ecfa-desy, amsterdam v. korbel, desy and calice1 the...

dr desy mikroteknik