01-oct-00h. f. hoffmann, cern-dg/di1 collaborations in particle physics example cern the mission of...

01-Oct-00 H. F. Hoffmann, CERN-DG/DI 3

CERN, evolution of resources

0

200

400

600

800

1000

1200

1400

52-54 1956 1958 1960 1962 1964 1966 1968 1970 1972 1974 1976 1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000

Years

MCHF, 2000 prices

Personnel

300 GeV

Exploitation

EnergieLHC

LEP 2

LEP 1

ppbar

ISR

2M&BEBC

PSSC

1400

1200

1000

800

600

400

200

0

Expenditure at 2000 prices

CERN paid (2750 staff, 800 fellows, paid associates)

Unpaid visiting scientists - Users

Contributions

0

100

200

300

400

500

600

700

800

900

20 25 30 35 40 45 50 55 60 65 70 75 80

Staff

Users

Age distribution>5500 visitors, 2750 staff,>1200/year turnover (3.99)Excellent education

CONTRIBUTIONS TO THE CERN PROGRAMMES, (MCHF, 2000 PRICES)

0

200

400

600

800

1000

1200

1400

0

1000

2000

3000

4000

5000

6000

7000

Year1956195819601962196419661968197019721974197619781980198219841986198819901992199419961998

CERN Paid Total

Users


aéro

port

GenèveAtlas

CMS

Alice

LHCb

PS

19542000


Evolution of CERN Particle Accelerators

• PS, 1959, ~1 GeV (cm)• ISR, 1971, ~9 GeV (cm)• SPS, 1976,~4.5 GeV (cm)

• SppbarS 1981 ~90 GeV (cm)

• LEP, 1989, 80-209(?) GeV (cm) • LHC, 2005, ~2'000 GeV (cm)

("cm" center of mass of partons)

s.c.cavity


"Facilities"

81 cm Bubble Chamber PS 1962-68Constructed in France

Big European Bubble Chamber PS and SPS 1974-1985External muon identifier

SFM facility (magnet, vacuum chamber,SFMDetector) at the ISR, external detectors as specific triggers by external laboratories with help of CERN

MWPC, 1968,SFMD: 300K wires 1974DAQ, Trigger,large datarates


Experimental Apparatus

BEBC photo, v-beam, ~30 tracks, semi-automatic scanning,very sophisticated tracking and analysis codes---->computer literacy>computer literacyresolution ~ 100 µ, < 1 event/sec, 20m3 liquid superheated hydrogen, 1.5 Tesla S.C. magnet


"Counter" Experiments

M&G Fidecaro, 1964, spark chambers, -->

N* experiment at ISR, 1972

UA1 at SppbarS 1981-1989 ALEPH at LEP 1989-2000

Charm Search at ISR, 1975

ATLAS at LHC, 2005-2020150*106 sensors;


Experimental Apparatus, continued

Pixel detector50*100µ/pixel, 140million

channels

LHC collisions, 109 events/s Complexity of data: ~250SPECint95*sec/event 1 / 1013 selectivity

Basic building block in full pixel readout chip : 8 pixels/12 000 transistors in 400 by 425 mm2

General Purpose detector,Multiple, simultaneous detection modes, OO programming,millions of lines of code


ATLAS Collaboration

0

50

100

150

200

250

300

Austria

Czech Republic

DenmarkFinlandFranceGermany

Greece

Italy

Netherlands

NorwayPolandPortugal

Slovac Republic

SpainSweden

Switzerland

United Kingdom

CERNArmeniaAustralia

Azerbaijan RepublicRepublic of Belarus

BrazilCanada

China PR

Republic of Georgia

IsraelJapanMoroccoRomania

Russia

JINR Dubna

SloveniaTaiwanTurkey

United States

1800 physicists,150 institutes; 35 countriesR&D, proposal, design, reviews, approval: 1988-1996Construction, installation 1996-2005, operation 2005-2020Material Cost 300 M Euro, CERN part:20%


CERN's Network in the World

267 institutes in Europe, 4603 users208 institutes elsewhere, 1632 userssome points = several institutes


Virtual Room Videoconferencing System• 3267 machines

reg.• 1994 people in• 52 countries• 182 institutes• http://vrvs.cern.ch/

Bandwidth >256 Kbps--> >10 frames/sec

9

Virtual Rooms Concept

u Enter a Virtual Room Through Your NearestReflector


– Open, global collaboration of critical mass, able to deal with all problems posed, together with CERN and collaborating institutes

– "Lean, bottom-up" self-organisationself-organisation; success based on experienced collaborators, eager young people, common goals and competition

– MoU, best intentions but not legally bindingMoU, best intentions but not legally binding– Free choice of collaborating institutes to participate -

or not– Clear common long-term mission, clear objectives,– Free exchange of ideas, technologies, R&D results – Often best people in the field of interest– External peer reviews; elaborate internal reviews and

QA– Good record of achievements in terms of delivery to

specs, schedules, budgets

Some typical features of such collaborations


Basic Organisation of any Physics- , Scientific- Experiment

• Organigram– Hierarchy - Heterarchy

. . . .


ATLAS ORGANIZATION

(Chair : J.D. DowellDeputy : M. Cavalli-Sforza)

Plenary Meeting

Spokesperson(P. Jenni)

CB ChairAdvisory Group

Technical Co-ordinator

(M. Nessi)

Resource Co-ordinator

(P. Schmid)

Inner Detector(M. A. Parker

M. Tyndel)L. Rossi

LAr Calorimeter(D. FournierD. LissauerH. Oberlack)

Tile Calorimeter(M. Nessi)

(G. CiapettiC. Fabjan)

MuonSpectrometer

Electronics(H. Williams)

(N. Ellis)

Trigger

DAQ(L. Mapelli)

Software,Computing(J. Knobloch)

Physics andDetectorSim.

(D. Froidevaux)

Collaboration Board

Magnet(H.TenKate)

Executive Board

Resource ReviewBoard

(Deputy: T. Akesson)

Gen. Members(P. Le DuA. Zaitsev)


CMS COLLABORATION RRB CMS-D 98-31

Memorandum of Understandingfor Collaboration in the Construction of the CMS Detector

betweenThe EUROPEAN ORGANISATION FOR NUCLEAR RESEARCH,

andan Institution/Funding Agency of the CMS Collaboration

Preamble(a) A group of Institutes from CERN Member and non-Member States, and CERN,has agreed to collaborate to form the CMS Collaboration (Annex 1). This Collaboration has proposed to CERN an experiment to study particle interactions at the highest possible energies and luminosities to be reached with the Large Hadron Collider (LHC). These Institutes have secured the support of their Funding Agencies to enable them to participate in the CMS Collaboration.(b) Agreement to this Collaboration is effected through identical Memoranda of Understanding (hereafter referred to as MoU) between each Funding Agency or Institute, as appropriate, in the Collaboration and CERN, as the Host Laboratory. These MoUs define the Collaboration and its objectives, and the rights and obligations of the collaborating Institutes.(c) On the basis of a Technical Proposal submitted in December 1994 (CERN/ LHCC/94-38) and a detailed review of the scientific merits, the technological feasibility and estimates of the needed resources, the LHC Committee (LHCC) recommended approval of the experiment to the CERN Research Board, subject to a set of milestones to be met by the experiment in its initial phase (CERN/LHCC 95-76).(d) Based on the recommendation by the LHCC and in agreement with the list of milestones, the Research Board recommended to the Director General of CERN to approve the project, together with plans, including milestones, leading to the sub-detector Technical Design Reports.


(e) The Director General accepted the Research Board recommendation and approved the project to build the detector for the CMS experiment within a cost ceiling not exceeding 475 MCHF (in 1995 prices).

(f) Before proceeding to the final construction phase, each sub-detector (c.f.. Article 4.1) will be subjected to a technical, financial, and manpower review (CERN/DG/RB 95-234) by the LHCC based on the Technical Design Reports. This process will be completed during 1997 and 1998 for most of the sub-systems.

(g) A Resources Review Board (RRB) has been constituted which comprises the representatives of all CMS Funding Agencies and the managements of CERN and the CMS Collaboration. It is chaired by the CERN Director of Research.The role of the RRB includes :· reaching agreement on the Memorandum of Understanding· monitoring the Common Projects and the use of the Common Funds· monitoring the general financial and manpower support· reaching agreement on a maintenance and operation procedure andmonitoring its functioning· endorsing the annual construction and maintenance and operation budgets of the detector.The management of the Collaboration reports regularly to the RRB on technical, managerial, financial and administrative matters, and on the composition of the Collaboration.

(h) Interim MoUs become obsolete

(i) This MoU is not legally binding, but the Institutes and Funding Agencies recognize that the success of the Collaboration depends on all its members adhering to its provisions. Any default will be dealt with, in the first instance,by the Collaboration and if necessary then by the RRB.


ATLAS Organisation (as example)• Principles:

– Democracy– Separation of policy-making and executive powers– Minimal formal organisation– Limited terms of office

• Plenary Meeting:– Forum of the all-hands discussions, – all major decisions concerning physics objectives and results,

hardware and software design, organisational matters must be discussed there

• Collaboration Board:– Policy- and decision-making body with typical tasks:

• Decisions on global detector design• Policy matters wrt official bodies• Financial and human resources• Elections• Organisation and membership


Project Organisation (example ATLAS)

• ATLAS project=organised sum of sub-projects and parts, conceived, designed and fabricated to a variety of habits, standards, cultures around the world

• Engineering organisation -- "top cultural layer", common language and common "rules of the game" to permit "engineering communication" throughout the project– Project breakdown structure (product, assembly breakdowns)– Work packages and WP-descriptions– Schedules, milestones, reporting for project follow up– Quality assurance

• Reviews at the various project stages like design, construction, assembly

• Configuration management

– Integration, mechanical, services, "environmental", accelerator

– Safety (CERN-LHCC/99-01; ATLAS TDR 13, "Tech. Co-ordination"; 31-01-1999, 598

pages)


From Particles to Petabytes:Challenges in High Throughput Computing

Example:(Data-) Grid,(EU-Project, NSF Project,..)

a global Particle Physics Projectto make world-wide LHC computing

possible


On-line System• Multi-level trigger• Filter out less interesting• Reduce data volume• 24 x 7 operation

Level 1 - Special Hardware

Level 2 - Embedded Processors

40 MHz40 MHz (1000 TB/sec) equivalent)

(1000 TB/sec) equivalent)

Level 3 – Farm of commodity CPU

75 KHz75 KHz (75 GB/sec)fully digitised

(75 GB/sec)fully digitised5 KHz5 KHz (5 GB/sec)

(5 GB/sec)100 Hz100 Hz (100 MB/sec)

(100 MB/sec)

Data Recording &

Data Recording &

Offline Analysis

Offline Analysis

Digital telephone1-2 KB/sec


1 billion people 1 billion people surfing the Websurfing the Web

1 billion people 1 billion people surfing the Websurfing the Web

How Much Data is Involved?

105

104104

103103

102102

Level 1 Rate (Hz)

Level 1 Rate (Hz)

High Level-1 Trigger(1 MHz)High Level-1 Trigger(1 MHz)

High No. ChannelsHigh Bandwidth(500 Gbit/s)

High No. ChannelsHigh Bandwidth(500 Gbit/s)

High Data Archive(PetaByte)High Data Archive(PetaByte)

LHCBLHCB

KLOE

HERA-BHERA-B

CDF IICDF II

CDFCDF

H1ZEUS

H1ZEUS

UA1UA1

LEPLEP

NA49NA49

ALICEALICE

Event Size (bytes)Event Size (bytes)

104104 105105 106106

ATLASCMSATLASCMS

106106

107107


Complex Queries = More CPU Per Byte

Evolution of Computing CapacityEvolution of Computing Capacity

Thousands ofThousands ofSPECint 95SPECint 95

00100100200200300300400400500500600600700700800800900900

1'0001'000

1997

1997

1998

1998

1999

1999

2000

2000

2001

2001

2002

2002

2003

2003

2004

2004

2005

2005

YearYear

OthersOthers

COMPASSCOMPASS

LHCLHCCERN 1999:CERN 1999:

3.5K SI953.5K SI95900 CPUs900 CPUs

CERN 1999:CERN 1999:3.5K SI953.5K SI95900 CPUs900 CPUs

?


CERN Computer Center Today…--> Commodity


• No longer aligned with supercomputing philosophies

CERN Computer Center Today…

Therefore, our natural affinity has shifted from Therefore, our natural affinity has shifted from supercomputerssupercomputers

towards ISPs, e-commerce and data marketerstowards ISPs, e-commerce and data marketers

• Require many small independent problem solutions

– “High Throughput Computing” processing “click-like” interactions in parallel

• A marriage of supercomputer storage systems with supermarket commodity CPU

– Disk access layer (hw+sw) sandwiched between

– “Middle-ware” on network layer important


On-demand creation of powerfulOn-demand creation of powerfulvirtual computing and data systemsvirtual computing and data systems

Grid: Flexible, high-performance access to all significant resources

Sensor nets

http://

http://

Web: Uniform access to HTML documents

Data Stores

Computers

Softwarecatalogs

Colleagues

Grids: Next Generation Web

Web-sites

0

50

100

150

200

250

300

Austria

Czech Republic

DenmarkFinlandFranceGermany

Greece

Italy

Netherlands

NorwayPolandPortugal

Slovac Republic

SpainSweden

Switzerland

United Kingdom

CERNArmeniaAustralia

Azerbaijan RepublicRepublic of Belarus

BrazilCanada

China PR

Republic of Georgia

IsraelJapanMoroccoRomania

Russia

JINR Dubna

SloveniaTaiwanTurkey

United States


LHC Vision: Data Grid Hierarchy

Tier 1

Tier2 Center

Online System

Offline Farm,CERN Computer

Ctr > 20 TIPS

German Centre

FNAL Center Italy Center UK Center

InstituteInstituteInstituteInstitute ~0.25TIPS

Workstations

~100 MBytes/sec

~2.5 Gbits/sec

100 - 1000

Mbits/sec

Bunch crossing per 25 nsecs; 100 triggers per second. Event is ~1 MByte in size

Physicists work on analysis “channels”

Each institute has ~10 physicists working on one or more channels

Physics data cache

~PByte/sec

~0.6-2.5 Gbits/sec

Tier2 CenterTier2 CenterTier2 Center

~622 Mbits/sec

Tier 0 +1

Tier 3

Tier 4

Tier2 Center Tier 2

Experiment


The Grid Middleware Services Concept• Standard services that

– Provide uniform, high-level access to a wide range of resources (including networks)

– Address interdomain issues: security, policy

– Permit application-level management and monitoring of end-to-end performance

• Broadly deployed, like Internet Protocols

• Enabler of application-specific tools as well as applications themselves


GEANT, necessary infrastructure

Minimum bandwidth of 2.5 Gbps between core nodes, possibility of starting with some 10Gbps (STM-64/OC-768c) circuits is not excluded.

Connection to other World Regions in principle via core nodes only, They will, together, form a European Distributed Access (EDA) “point” conceptually similar to the STAR TAP.


The Web, a historical case study• Invented at CERN in 1989 as application layer

on top of the internet infrastructure• Development started in Europe (small) and US

(big, >50 computer scientists initially for MOSAIC)

• 80% of the most visited sites: US, <10% Europe

Web Site Servers


Questions and AnswersQ1:Flexibility of institutional structure to allow researchers to change field

Q2: Exchange of knowledge across disciplines and institutions

Q3: Main obstacles to international co-operation

Q4:Information revolution

A1:Beginning discussion with astro-, space- physics; technology transfer-> funding for interdisciplinary activities? However,very clear mission, "mono-culture"A2: More fellowships in technological and interdisciplinary fields with specific funding: domain competence and add-on competence(KI, DataGrid)A3: Funding agencies, scientists, politicians still think "national" Employment conditions, spouses, schools--> keep national employment in Europe "in exchange" plus adjustment allowance, help for spouses to find appropriate work, international schoolsA4: Promote e-science, grids, . . .


International Collaboration Objective: top scientific excellence in your particular field

Add resources("Resources": Talent; particular knowledge, experience, methods; specific scientific apparatus, technologies; funds)

Create a complete, competitive, technological infrastructure in your particular field beyond local, regional, national meansCreate a network of competence to solve detailed problems quickly and to prepare new meansBase your collaboration to a large degree on Universities (talent)

Reach, sustain excellence by attracting the best people

"Nobody is perfect" - but a team can be

01-oct-00h. f. hoffmann, cern-dg/di1 collaborations in particle physics example cern the mission of...

Documents

cern programmes

cern particle acceleratorsps

gev cmsppbars

gev cmlep

gev cmisr

gev cmsps

gev cmcm center of mass

nuclear research