computing for lhcb-italy domenico galli, umberto marconi and vincenzo vagnoni genève, january 17,...

Computing for LHCb-Italy

Domenico Galli, Umberto Marconi and Vincenzo Vagnoni

Genève, January 17, 2001

Computing for LHCb-ItalyDomenico Galli, Umberto Marconi and Vincenzo Vagnoni

2001 Farm for LHCb-Italy

CPU

DRAMChipsetGC

NIPCI

AGP

CPU

DRAM Chipset GC

NIPCI

AGP

IDE

swap

CPU

DRAM Chipset GC

NIPCI

AGP

IDE

swap

CPU

DRAMChipsetGC

NIPCI

AGP

CPU

DRAMChipsetGC

NIPCI

AGP

CPU

DRAMChipsetGC

NIPCI

AGP

CPU

DRAMChipsetGC

NIPCI

AGP

CPU

DRAMChipsetGC

NIPCI

AGPCPU

CPU

DRAM

Chipset

Switched Node Backplane

NI

NI

PCI

hostadapter

Switched Node

Disk IF

Switched Node

Disk IF

Switched Node

Disk IF

Switched Node

Disk IF

Switched Node

Disk IF

Switched Node

Disk IF

Ultra ATA Ultra ATA Ultra ATA Ultra ATA Ultra ATA Ultra ATA

Network Attached Storage

Login Nodes Job Execution Nodes

Ethernet Switch


Components (Motherboards, Racks, NAS, Switches)


2001 Funded Investments

Funded

racks (with cooling and power supply) 1 6 ML

Asus FX monoprocessor motherboards: on-board network IF Intel Pentium III 800 MHz CPU 256 MB RAM 133 MHz bus

15 30 ML

NAS RAID Raidzone OpenNAS RS15-R1200 Dual processor Pentium III 800 MHz 15 IDE 80 GB Ultra ATA/100 disks Dual 100 Mbps network IF Dual redundant 300W hot swap power supply

1 60 ML

56 ports ethernet switch 1 6.5 ML

Total 102.5 ML


2001 Investments Availability of more funds, later in 2001, is

conditional to: The production of a believable computing model for

2001 (given the present WAN bandwidth).

The evidence of a high load of the installed CPUs.

The use of produced MC by Italian institutes for their own analyses.

All LHCb Italian groups are therefore invited to estimate their CPU need for MC production in 2001, in order to ask more funds.


Preliminary Farm Tests Performed in Bologna Network boot

Boot-prom required (PXE?) Swap-less operation

Memory requirements for job execution nodes is stable RAM errors

Memtest86 and badram NFS functionalities

Kernel 2.2.18 required in order to allow broken setuid on NFS root Network boot for Asus FX boards Wake on network

In order to switch on and off boards from remote. Electromagnetic interferences between neighbouring

boards


Finale Computing Organization MONARC hierarchy

Tier-0: CERN Tier-1: Regional Centre (1 for each country). Tier-2: Satellites of Tier-1 (mainly devoted to analysis). Tier-3: Department Server (1 for each institute). Tier-4: Desktop

Data formats RAW, RAWmc ESD, ESDmc: Event Summary Data; reconstruction output. AOD: Analysis Object Data, public analysis format. DPD: Derived Physics Data, private analysis format,

ntuples.


LHCb Differences With Respect to Other LHC Experiments The first stage in the analysis (ESD AOD, final

state reconstruction) is performed in common for all the analyses that subsequently follow (more than one algorithm could have to be run on any single event, if that event satisfies more than one tag criterion).

The first stage in the analysis is therefore performed in production (Production Analysis) soon after data taking at CERN (real data) and in MC production centres (MC).

AOD (20 TB/a real data) are systematically distributed to all Tier-1 centres. ESD (100 TB/a real data) are kept in production centres.

Other LHC experiments export ESD.


Final Tier-n Organization in Italy ATLAS plan

Tier-1 stores ESD. Performs reprocessing and first analysis stage (AOD production). No MC production.

Tier-2s store AOD and perform MC production + analysis. Tier-3s execute analysis

CMS plan Tier-1 stores ESD and performs collaboration scheduled

MC production + reprocessing Tier-2++ (Legnaro) performs local requested MC production

+ reprocessing + analysis Tier-2s/Tier-3s produce local requested MC production +

analysis Tier-2/Tier-3 difference is not functional


LHCb Computing Model Tasks

MC production ( RAWmc, ESDmc), production analysis (ESDmc AOD) and reprocessing, performed in Tier-1 (but also in Tier-3 dead time).

User analysis (AOD DPD) performed in a collaboration between Tier-1 (selection) and Tier-3 (computation).

Interactive analysis of DPD performed in Tier-3 and Tier-4. Storage

RAW and ESD stored only in the Tier-0 (CERN) (not distributed).

RAWmc and ESDmc stored only in the Tier-1 centre which produce them (not distributed).

AOD (from real data and all Tier-1 MC) stored in every Tier 1 (distributed).

DPD (ntuples) stored in Tier-3.


LHCb Tier-n Tasks Tier-1

MC production ( RAWmc, ESDmc) + production analysis ( AOD) + reprocessing.

User analysis ( DPD) in collaboration with Tier-3s. Storage of RAWmc and ESDmc produced in the centre

itself. Storage of all the AOD (real data produced at CERN, MC

data produced in all Tier-1 centres). Tier-3s

User analysis ( DPD) in collaboration with Tier-1. DPD storage. Interactive analysis of DPD. In dead time, MC production .

Tier-2s (not in the MONARC sense) are large Tier-3, functionally equivalent to Tier-3.


2007 LHCb Tier-n Organization in Italy

1 Tier-1 centre: 140 kSI95, 110 TB disk.

9 Tier-3 centres (Bologna, Cagliari, Ferrara, Firenze, Frascati, Genova, Milano, Roma1, Roma2): average 1 kSI95, 1 TB disk.

Tier-3 dimension can vary on the basis of group size and analysis activity. Large Tier-3 centres could be named Tier-2,

but not in the MONARC meaning.

computing for lhcb-italy domenico galli, umberto marconi and vincenzo vagnoni genève, january 17,...

Documents