cern r. jacobsson between lhc and the grid - aspects of operating the lhc experiments – t....
DESCRIPTION
CERN R. Jacobsson l At all levels we need tools to predict, prevent, and limit “non-bb” exposure Framework to combine information and understand correlations between background, beam characteristics and machine settings Dedicated archiving and analysis tools We need to look at the entire LHC machine 3 Instantaneous damage Beam Interlock Background ………….…………Trigger rates…………………… ………...………..Poor data quality………………… ……………………..….Single event upsets………. …………..……Accelerated aging……….………… …………………Long-term damage……………..... Online monitoring Accumulated dose and Luminosity Background Beam characteristics Machine settings Halo/beam-gas/………………....…….scraping……………....Beam incidentTRANSCRIPT
No Slide Title- Aspects of Operating the LHC Experiments –
T. Camporesi, C.Clement, C. Garabatos Cuadrado, L. Malgeri, T. Pauly, R. Jacobsson
1
Experiments
ALICE
ATLAS
CMS
LHCb
http://indico.cern.ch/conferenceDisplay.py?confId=87720
R. Jacobsson
The Scene
Experiments have never in the past been so tightly connected to the accelerator
What’s so special about LHC?
Stored energy 2 x 360 MJ and fragile detectors
Experiment protection
High interaction rate and large events size
Fast and reliable readout, storage and transfer to offline processing
Fast feedback from Data Quality checking
High intensity proton collider and sensitive detectors
Monitoring and understanding/analyzing/optimizing experimental conditions
Luminosity determination
Automatic Calibrations
Many years of 24h operation with few people and non-experts
Operating the whole detector from one console
Understandable high-level tools for diagnostics, alarms and data monitoring
Homogeneity in the system
Shifter training
These lectures will describe largely how we have addressed and solved these
2
Protecting Experiment and Data
At all levels we need tools to predict, prevent, and limit “non-bb” exposure
Framework to combine information and understand correlations between background, beam characteristics and machine settings Dedicated archiving and analysis tools
We need to look at the entire LHC machine
3
Subdetectors
The “Orchestra Director”
Pool of RSs for autonomous stand-alone running of any subdetector
R. Jacobsson
7
Histogram collected from all systems
Monitoring Farm spying on event streams at best effort
Also produces histograms from an online reconstruction at best effort
Histogram analysis Automatic checks and alarms
Histogram inspected by Data Manager Shifter
Calib
Histogram
Handling
(ECS)
HISTO
DB
Automatic
Histogram
Analysis
2 GB file(~60 kevts) / 30s
Offline
Typical Physics fill cycle
“Machine Development” (MD) periods
Some scheduled days/weeks the LHC will be operated not to produce physics data (Stable Beam) but to study machine physics and improve performance
LHCb will not take data, but safety should be ensured. A shift crew will be needed still, but with lower activities.
9
Control of data transfer
Load balancing
Physics triggers
Calibration triggers
Luminosity triggers
Luminosity scans (Vernier scan)
Driven and managed by the LHCb Timing and Fast Control System
Responsible for distributing timing, trigger and synchronous and asynchronous information to entire readout system
FPGA based master: Readout Supervisor
Also performs rate control and generates all types of auto-triggers and calibration sequences
Centralized Readout Control
Beam Modes for Physics
In LHCb we synthesize Machine/Beam modes into 9 “Internal LHC States”
INJECTION, RAMP, PHYS_ADJUST, PHYSICS, ADJUST, DUMP
EOF (End-Of-Fill), NO_BEAM, MD (Machine Development)
12
INJECTION, ADJUST and BEAM DUMP
R. Jacobsson
NO_BEAM (Injection Permit = FALSE, Any state of LHCb, Internal clock)
INJECTION (Injection Permit = TRUE, VELO out, External clock)
RAMP (Injection Permit = FALSE)
EOF (Internal Clock, Calibrations)
Handshake for Adjust
Confirm
RAMP
15
=
Be ready to receive, process and analyze 7 million events in the first hour of collisions
LHCb
Simulated events
Replacing detector with injection of 108 “accepted” simulated events real-time in Online system at HLT rate (2 kHz)
Also allows testing new HLT versions with minimum bias events
R. Jacobsson
16
16
Two years of intense work 2006 – 2008 with the aim to:
Operate the detector AND people as a unit with common tools
Bring all components (sub-detectors and service systems) to operational state.
Define, implement and validate the tools and procedures needed to run the detector as a whole
Organise the activities to reach the ready state in time
Understand and calibrate the detector
Test pulses, radioactive sources
Understandable high-level tools for diagnostics, alarms and data monitoring
Homogeneity in the system
Reach operational efficiency
Actually all achieved for pilot run 2009!
Crucial tool:
Readout and processing of sets of consecutive 25ns clock cycles around “detector activity” trigger
Time and space alignment
Optimize signal over spill-over
CERN
Alignment
17
LHCb did use cosmics but for obvious geometrical reasons not sufficient…
Beam 2 dumps on injection line beam stopper (TED) ideal.. But backwards…!
TED
TI8
LHC
18
Out position: 35mm from beam line
In nominal data taking position: 5mm from beam line
Data taking position determined during Open Tracking by determining luminous region before moving in for every fill
May only leave its “garage position” during STABLE BEAM
R. Jacobsson
Directly connected to LHC Beam Dump System (LBDS)
Also hardware interface for Injection Permit
High-sensitivity and high time resolution monitor
Scintillator based Beam Loss Monitor
25ns integration and fast readout
Used mainly to understand background and discover problems early (collimator settings, aperture, beam-gas, injection etc)
20
BCM
S
a
f
e
B
e
a
m
F
la
g
s
FEElectronicsFEElectronicsFEElectronicsFEElectronicsFEElectronicsFEElectronicsFEElectronics
L0 trigger
L0 Trigger
320 ROBs•24 [email protected] Gb/s•4 outputs@1 Gb/s50 TB with 70 MB/s3000 GbE ports35 GB/s50 subfarms of ~40 nodes5000 optical/analog linksO (4 Tb/s)
Offline
SWITCH
FEElectronicsFEElectronicsFEElectronicsFEElectronicsFEElectronicsFEElectronicsFEElectronics
No Beam X X X X X X X X
Setup X X X X X
Abort X X X X X
Injection Probe Beam X X X X X
Injection Setup Beam X X X X
Injection Physics Beam X X X X X
Prepare Ramp X X X X X
Ramp X X X X X
Flat Top X X X X X
Squeeze X X X X X
Adjust X X X X X
Stable Beams X X X X
Unstable Beams X X X X
Beam Dump X X X X X
Ramp Down X X X X X
Cycling X X X X X
Recovery X X X X X
Inject & Dump X X
Circulate & Dump X X
FEElectronicsFEElectronicsFEElectronicsFEElectronicsFEElectronicsFEElectronicsFEElectronics
1.0
0.8
0.6
0.4
0.2
0.0
Y (
1.0
0.8
0.6
0.4
0.2
0.0
Y (
T. Camporesi, C.Clement, C. Garabatos Cuadrado, L. Malgeri, T. Pauly, R. Jacobsson
1
Experiments
ALICE
ATLAS
CMS
LHCb
http://indico.cern.ch/conferenceDisplay.py?confId=87720
R. Jacobsson
The Scene
Experiments have never in the past been so tightly connected to the accelerator
What’s so special about LHC?
Stored energy 2 x 360 MJ and fragile detectors
Experiment protection
High interaction rate and large events size
Fast and reliable readout, storage and transfer to offline processing
Fast feedback from Data Quality checking
High intensity proton collider and sensitive detectors
Monitoring and understanding/analyzing/optimizing experimental conditions
Luminosity determination
Automatic Calibrations
Many years of 24h operation with few people and non-experts
Operating the whole detector from one console
Understandable high-level tools for diagnostics, alarms and data monitoring
Homogeneity in the system
Shifter training
These lectures will describe largely how we have addressed and solved these
2
Protecting Experiment and Data
At all levels we need tools to predict, prevent, and limit “non-bb” exposure
Framework to combine information and understand correlations between background, beam characteristics and machine settings Dedicated archiving and analysis tools
We need to look at the entire LHC machine
3
Subdetectors
The “Orchestra Director”
Pool of RSs for autonomous stand-alone running of any subdetector
R. Jacobsson
7
Histogram collected from all systems
Monitoring Farm spying on event streams at best effort
Also produces histograms from an online reconstruction at best effort
Histogram analysis Automatic checks and alarms
Histogram inspected by Data Manager Shifter
Calib
Histogram
Handling
(ECS)
HISTO
DB
Automatic
Histogram
Analysis
2 GB file(~60 kevts) / 30s
Offline
Typical Physics fill cycle
“Machine Development” (MD) periods
Some scheduled days/weeks the LHC will be operated not to produce physics data (Stable Beam) but to study machine physics and improve performance
LHCb will not take data, but safety should be ensured. A shift crew will be needed still, but with lower activities.
9
Control of data transfer
Load balancing
Physics triggers
Calibration triggers
Luminosity triggers
Luminosity scans (Vernier scan)
Driven and managed by the LHCb Timing and Fast Control System
Responsible for distributing timing, trigger and synchronous and asynchronous information to entire readout system
FPGA based master: Readout Supervisor
Also performs rate control and generates all types of auto-triggers and calibration sequences
Centralized Readout Control
Beam Modes for Physics
In LHCb we synthesize Machine/Beam modes into 9 “Internal LHC States”
INJECTION, RAMP, PHYS_ADJUST, PHYSICS, ADJUST, DUMP
EOF (End-Of-Fill), NO_BEAM, MD (Machine Development)
12
INJECTION, ADJUST and BEAM DUMP
R. Jacobsson
NO_BEAM (Injection Permit = FALSE, Any state of LHCb, Internal clock)
INJECTION (Injection Permit = TRUE, VELO out, External clock)
RAMP (Injection Permit = FALSE)
EOF (Internal Clock, Calibrations)
Handshake for Adjust
Confirm
RAMP
15
=
Be ready to receive, process and analyze 7 million events in the first hour of collisions
LHCb
Simulated events
Replacing detector with injection of 108 “accepted” simulated events real-time in Online system at HLT rate (2 kHz)
Also allows testing new HLT versions with minimum bias events
R. Jacobsson
16
16
Two years of intense work 2006 – 2008 with the aim to:
Operate the detector AND people as a unit with common tools
Bring all components (sub-detectors and service systems) to operational state.
Define, implement and validate the tools and procedures needed to run the detector as a whole
Organise the activities to reach the ready state in time
Understand and calibrate the detector
Test pulses, radioactive sources
Understandable high-level tools for diagnostics, alarms and data monitoring
Homogeneity in the system
Reach operational efficiency
Actually all achieved for pilot run 2009!
Crucial tool:
Readout and processing of sets of consecutive 25ns clock cycles around “detector activity” trigger
Time and space alignment
Optimize signal over spill-over
CERN
Alignment
17
LHCb did use cosmics but for obvious geometrical reasons not sufficient…
Beam 2 dumps on injection line beam stopper (TED) ideal.. But backwards…!
TED
TI8
LHC
18
Out position: 35mm from beam line
In nominal data taking position: 5mm from beam line
Data taking position determined during Open Tracking by determining luminous region before moving in for every fill
May only leave its “garage position” during STABLE BEAM
R. Jacobsson
Directly connected to LHC Beam Dump System (LBDS)
Also hardware interface for Injection Permit
High-sensitivity and high time resolution monitor
Scintillator based Beam Loss Monitor
25ns integration and fast readout
Used mainly to understand background and discover problems early (collimator settings, aperture, beam-gas, injection etc)
20
BCM
S
a
f
e
B
e
a
m
F
la
g
s
FEElectronicsFEElectronicsFEElectronicsFEElectronicsFEElectronicsFEElectronicsFEElectronics
L0 trigger
L0 Trigger
320 ROBs•24 [email protected] Gb/s•4 outputs@1 Gb/s50 TB with 70 MB/s3000 GbE ports35 GB/s50 subfarms of ~40 nodes5000 optical/analog linksO (4 Tb/s)
Offline
SWITCH
FEElectronicsFEElectronicsFEElectronicsFEElectronicsFEElectronicsFEElectronicsFEElectronics
No Beam X X X X X X X X
Setup X X X X X
Abort X X X X X
Injection Probe Beam X X X X X
Injection Setup Beam X X X X
Injection Physics Beam X X X X X
Prepare Ramp X X X X X
Ramp X X X X X
Flat Top X X X X X
Squeeze X X X X X
Adjust X X X X X
Stable Beams X X X X
Unstable Beams X X X X
Beam Dump X X X X X
Ramp Down X X X X X
Cycling X X X X X
Recovery X X X X X
Inject & Dump X X
Circulate & Dump X X
FEElectronicsFEElectronicsFEElectronicsFEElectronicsFEElectronicsFEElectronicsFEElectronics
1.0
0.8
0.6
0.4
0.2
0.0
Y (
1.0
0.8
0.6
0.4
0.2
0.0
Y (