Download - Summary of first LHC logging DB meeting
SUMMARY OF FIRST LHC LOGGING DB MEETING
Michele FlorisMichele Floris
Logging Database Use Case
Outline• Joint LHC machine experiments meeting:
Interfacing to the LHC logging database for postprocessing (offline analysis).
• Goals of this meeting:– Establish a first inventory of use cases and potential
users
– Discussion on required functionality
– How to continue this?
• Talks:– Users (Machine + Experiments)
– Logging and measurement service
Logging Database Use Case
LHC Beam Instrumentation
• Quite some development already
• Want at least same feature/performance
• Quite some development already
• Want at least same feature/performance
Logging Database Use Case
ALICE
15/03/2010Michele Floris 4
DIPDIP DCSDCS OCDBOCDB
Logging DatabaseLogging
Database
APIAPI
PROCESSINGPROCESSINGDelay of
publication?Delay of
publication?
Data format?Versions?
Data format?Versions?
Post-processing/calibration should be done centrally
Post-processing/calibration should be done centrally
What triggers the migration?What triggers the migration?
raw
v1
Logging Database Use Case
ATLAS
15/03/2010Michele Floris 5
• They also use DIP as primary data source
• They also use DIP as primary data source
Value based queries impossibleValue based queries impossible
Logging Database Use Case
CMS
15/03/2010Michele Floris 6
Logging Database Use Case
- We are archiving everything on our own, including all beam and machine parameters, that we receive via DIP
advantage: we have the same access interface to ONE database disadvantage: we always have to make sure we have everything and we clearly double the effort the tool should be interfaced to ANY database, each parameter
or condition in the right database
- We don’t have direct access to nominal settings and nominal parameters, like collimators settings, golden orbit, etc
- BLM thresholds we have but we archive the whole set ourselves…
- We don’t have access to the corrected and calibrated data
May also allow correlating the whole set of data with shifters’ names in experiment control room and CCC….
Current drawbacks
LHCb
Federico Alessio 7
PL/SQL filtered data
transfer
Current StatusCurrent Status
15-Mar-2010 Forum on Interfacing to the Logging Database for Data Analysis 8
LDBMDB
Equipment – DAQ – PLCEquipment – DAQ – PLCEquipment – DAQ – FEC
fff fff fff ffffff
fff ff
f
ELEC
COMMCV
EAU
TIM
f
ffff
QPSPIC
SU
Coll
CNGS
Exp
Cryo CIET
WIC
VACRad
BLM
BETSBIC
BCT BPM
FGC MSMK
VAC
<Data User>
~20 Years filtered data
7 Days raw data
~ 200’000 Signals~ 50 data loading processes~ 5.1 billion records per day~ 130 GB per day 46 TB per year throughput
~ 800’000 signals~ 300 data loading processes~ 3.8 billion records per day~ 105 GB per day 38 TB per year stored
> 300 extraction clients0.4 2 million extraction requests per day
15-Mar-2010 Forum on Interfacing to the Logging Database for Data Analysis 9
CERN Accelerator Logging Service
Data Extraction – Java APIData Extraction – Java API
Spring HTTP Remoting
READAPI
<Data User>Custom Java Applications (currently > 30)
10g AS
Spring HTTP Remoting
READAPI metadata
JDBC TS Data
JDBC TS Data
JDBC Metadata
<Data User>TIMBER
READAPI
LDB
MDB
They will only provide a JAVA interface (already used by timber and some 30 applications) We will need to implement a wrapper
They will only provide a JAVA interface (already used by timber and some 30 applications) We will need to implement a wrapper
Logging Database Use Case
Misc from discussion
15/03/2010Michele Floris 10
• DIP not guaranteed to be reliable (uptime < 100%)– Access to DB mandatory
• 2 main use cases– Running conditions (few users, lots of data)
Direct access to logging DB
– Offline analysis (many users, less data)Backfill of OCDB?
• Central post-processing: commonly requested – Versions! Format? Same Logging DB?
• Concurrent R/W: will also provide mirror DB?
Logging Database Use Case
Most relevant variables• bunch/beam intensities
• beam losses
• beam positions
• beam sizes (emittances)
• collimator positions
• some vacuum gauges
• but also sporadically-measured quantities such as:– crossing angles
– beta functions.
15/03/2010Michele Floris 11