����� ������� ����� ������������� �������������! "��#� "$&%'�)("*�+�,*�("*.-��/ 0� "�1("0��"�

I.Alexandrov1, A.Amorim2, E.Badescu3, M.Barczyk4, D.Burckhart-Chromek4, M.Caprini3,M.Dobson4, J.Flammer4, R.Hart5, R.Jones4, A.Kazarov4,8, S.Kolos4,8, V.Kotov1, D.Liko4,

L.Lucio2,4,7, L.Mapelli4, M.Mineev1, L.Moneta6, I.Papadopoulos4, M.Nassiakou4,9, N.Parrington7,L.Pedro2, A.Ribeiro2, Yu.Ryabov8, D.Schweiger4, I.Soloviev4,8, H.Wolters2

1) Joint Institute for Nuclear Research, Dubna, Russia2) FCUL (Science University of Lisbon), Lisbon, Portugal3) Institute of Atomic Physics, Bucharest, Romania4) European Organization for Nuclear Research (CERN), Geneva, Switzerland5) National Institute for Nuclear Physics and High Energy Physics (NIKHEF), Amsterdam, Netherlands6) Physics Section, University of Geneva, Geneva, Switzerland7) Sunderland University, Sunderland, England8) Petersburg Nuclear Physics Institute (PNPI), Gatchina, St. Petersburg, Russia9) NTUA, National Technical University of Athens, Athens Greece

2436587:9<;>=<7ATLAS will be one of the four detectors for theLHC (Large Hadron Collider) particleaccelerator currently being built at CERN,Geneva. The project is expected to startproduction in 2006 and during its lifetime (15-20years) to generate roughly one petabyte per yearof particle physics’ data. This vast amount ofinformation will require several meta-datarepositories which will ease the manipulation andunderstanding of physics’ data by the final users(physicists doing analysis). Metadata repositoriesand tools at ATLAS may address such problemsas the logical organization of the physics dataaccording to data taking sessions, errors andfaults during data gathering, data quality orterciary storage meta-information.The OBK (Online Book-Keeper) is a componentof ATLAS' Online Software - the system whichprovides configuration, control and monitoringservices to the DAQ (Data AQquisition system).In this paper we will explain the role of the OBKas one of the main collectors and managers ofmeta-data produced online, how that data isstored and the interfaces that are provided toaccess it - merging the physics data with the

collected metadata will play an essential role forfuture analysis and interpretion of the physicsevents observed at ATLAS. We also provide anhistorical background to the OBK by analysingthe several prototypes implemented in thecontext of our software development process andthe results and experience obtained with thevarious DBMS technologies used.

?�@�A8B6C:D<E>F6G6H<C:IJE>BHEP (High Energy Physics) has traditionally been aheavy user of database techniques and products, usually atsignificantly higher levels than the typical marketdemand. This is due to the enormous amount of datasamples physicists need to collect and store in order to beable to produce statistics which are accurate enough tovalidate their theories.

ATLAS is one of the four detectors being built for theLHC particle accelerator at CERN to be completed in2006. After being accelerated, particles will collide insidethe detector. From observing the results of those collisionsphysicists expect to be able to expand our knowledge onthe basic contituents of matter.

Physics data ready for analysis is composed of whatphysicists call events – each event describes theinteraction of particles and their final state products. Theprocess of producing and storing interesting events ishowever a very complex problem: in the detector the realevents occur at rates that require extremely sophisticatedhardware and software to filter out only a fraction(1/107), which at the ATLAS may still mean 100interesting events of 1 megabyte each, per second. Giventhat ATLAS is expected to run for 15-20 years, the

K�LNMPORQ SPSJQ TVUXW T)YNT Z\[X]^Q W _`Tba`WVcVLdLfe\g ghTbMhZie:MJWhT cjW _kQ SlO^e`W LNMJQ e`gmQ Son`MPe:U`W LdpZ�MJTVq:Q p1LdprW _:e`WsW _1LtYNT ZuQ LdS4e:MPL�U`T1WvOme:p1LwTbMxp\Q SJW MJQ yba`W LNp�cbTbM4pkQ MPLVYNWYVTbO^O^LNMPYbQ e\g�e:p1qde:U\W e:n1LNz<W _1Lv{V|~}^��YNT Z\[\MJQ n1_`W�U\T:W Q YdL�e:U1p�W _1LfW Q W g LfT c"W _1LZ�a1y1g Q YJe\W Q TbUfe:U:p�Q W S�p:e\W LlePZ:Z�Lde1MPz�e:U:pfU\T:W Q YdL6Q S�n~Q qNLdUjW _:e\W�YNT Z\[8Q UbnXQ SoyP[Z�LNMPORQ SPSdQ TbU�T cwW _:L {bLNM [�|8e1M�n`L�}�e`W ew��e1S�L���U:p`Tb]uO^LdU\W ���`T�YVT Z`[T1W _1LNM�]^Q S�LNz\TbMuW T>MPL Z�a`y:g Q S�_1z1MPLd�1a\Q MPLNS�e8cNLNLme:U1pN� TbM<S Z�LVYVQ e\gPZiLNMPORQ SPSdQ TVU8cVMJTbOW _:Lh�iU1p`TV]uO^LdU\W�h�:�k�:�:�1�i� �i�~�^�k�\�:�i�<�h�k  ¡8¢�£�¤¦¥�§¦¨k©iª�«:¬1«:©�­:«:®¯�¨~©i°f±)¨k©i°~®~§¦²i³ ©i´k®Nµh¶~¶Nµ

generated databases may go up to tens of petabytes[SHI98].

However, the data produced is not limited to events.Such large machines (the LHC and ATLAS) arecomposed of many sub-systems which have their role indata-taking. Since all these sub-systems are highlyconfigurable and/or may change over time, all theparameters and running conditions will also have to bestored for later contextualization of the collected events.These additional databases may include for examplesoftware and hardware configurations of the computinginfrastructure, alignment of the parts of the detector orenvironmental conditions like pressure, temperature andother information that need to be taken into considerationto achieve precise physics measurements.

In order to make the massive amounts of datacontained in the above mentioned databases accessibleand understandable by the physicists doing analysis,metadata (data about data) is useful and necessary.Although it is difficult to define the boundary betweenwhat is data and what is metadata, in this paper we takeon the following definition: metadata is “ ·�¸�¹ ¸»º�¼�½ ¾k¼¿ ¸�ÀkÁkÂ�·�¸�¹ ¸ ¿jÃ�Ä Á�¸�¾k¾kÁkÂbÂb½ Å�Æ Á�¹ à ¹ ¼�Á�Ç�ÂbÁ Ä ” [BAR97]. In thissense several systems within ATLAS may be consideredas metadata repositories. The OBK in particular dealswith online cataloging of physics’ information.

How these metadata databases are organized andaccessed, how they will interact with the main databasesor what technologies will they employ is still unclear,given that the subsystems are still being built and willonly be ready for production in 2006. This paper tries toaddress some of these questions from the viewpoint ofthe OBK. References to other metadata repositories inHEP are also provided in order to better put the probleminto perspective.È�É ÈËÊjÌ�ÍJÌ�ÎhÌ�ÏbÐ"ÍJÐkÑkÒhÓhÔ�Õ Ô�Ö�×vÎhÌ�ÑkØuÖ�ÙkÔ�ÚhÓhÛAs mentioned before, database technology has alwaysbeen a hot topic in the HEP experiment domain. Untilrecently however, either because of market solutions’limits of performance and scalability or because ofphysicists’ preference for custom solutions, the HEPworld and the database community often did notcollaborate in finding solutions to their commonproblems.

Commercial databases made their appearance in HEPexperiments during the 1980s for the handling of book-keeping data (metadata). The management of physics’data itself was left to specialized solutions. In the 1990showever, given the appearance of more flexible datamodels (Object Oriented) and the increasing performanceand scalability of commercial DBMSs, the HEPcommunity showed interest in the available technology.

In the context of the future LHC experiments –including ATLAS – the RD45 project at CERN set out totest commercial DBMSs for HEP requirements [MAL97].

The Object Oriented design model was of particularinterest, given that it is a closer match for particle physics’needs in term of data model than previous relationalapproaches. As a result of the RD45 project, theObjectivity/DB DBMS was chosen as a vehicle for studyof LHC long term data. As will be explained in this paper,the OBK also followed this trend in a first prototypeiteration. Note that specific solutions are still very presentin the current ATLAS database panorama, along withmarket solutions other than Objectivity/DB.

Currently, investigations by CERN/IT group[NOW01] point to Object/Relational DBMSs as the futureof databases in HEP. Oracle 9i attracted a lot of attentionfor its rich data model as an implementation of SQL:1999and also for its VLDB oriented features.

Ü>Ý�Þ�ß6à�á�â6ãJäåâ6àwæ^ç>è1ébê�ë>ì<à�è1ì<ç>íîé:ß6à�á�ïñðóò�ôõ äJà<ê�ö6ç>äåâ6é

To understand what kind of metadata the OBK stores andthe technology context the system fits in, this section ofthe paper provides an overview of the Online Software,focusing on the components the OBK interacts with most.

÷�ø ùvújûkükýhþ ÿ��kükÿ��hû������^ÿJý������ þ ��� ����Jÿ ����û��The OBK is a software component of the Online

Software, which in turn is a sub-system of the TDAQ(Trigger-DAQ) – the set of software and hardwaresystems responsible for acquiring, filtering and movingevent data from the detector to mass storage, in order tomake it available for subsequent analysis.

The role of the Online Software is to provide to otherTDAQ systems configuration, control and monitoringservices. It excludes however the processing andtransportation of physics’ data [ATL00]. Morespecifically, the Online Software provides such servicesas uniformly assigning commands to other systems so thatthey can change state coherently, interfaces to databaseswhich hold parameters for the configuration of the wholeTDAQ, inter process communication facilities,monitoring (including sampling and data quality checks)and book-keeping services. A range of ancilliary servicessuch as graphical user interfaces are also included.

As can be understood from the above explanation, theOnline Software is a generic framework designed to beable to supervise many distinct data taking configurations.Since the ATLAS will be a very large machine whosecomponents must work concurrently, the Online Softwareis also reponsible for managing that concurrency in termsof software. A subset of the detector and TDAQhardware/software able to acquire data independently iscalled a ������� � � � ��� .

The architectural configuration of the Online softwarecan be observed in figure 1. The various components arebunched in groups called ���������! �"�#$��"�%���%�& � , whichinclude ')(�*,+.-�*�/ 0�-�1 , 2�3�4�4�5�6�7 8�6 , 9�:�;�< = :�>�< ;�? , @BA�C�D E E D F�GIH

and JLK�M K�N�K�O�P�O . In order to interact with the TDAQsystems the Online Software supervises, a custumizableprogramatical interface called QSR�T�U�VXW.R�Y�Z [�R�V V \�[ isprovided to each of those systems.

]�^ ]`_.a�a�bdc�bde�egf�h i�j`k�l�m�l`n�a�o�p�q�e�nThe OBK is considered to be part of the Databases

super-component together with the ConfigurationDatabases – although the roles of the two components arequite distinct. A high level description of the purpose ofthe OBK inside the Online Software is provided in[ATL00] and goes as follows: the OBK “ r�s�t�u�v w�x�yv z|{~}�s��Lr�� v }�z`r���}������ u�x���r�� r`s�x�t�}�s���x���� }.��x�s��Lr�z�x�z���y�� }�s�r���x����� u�x��$�B��yI�dy�� x��L�g����s�x�t�}�s���y�� u�x`v z|{~}�s��Lr�� v }�z�� }���x`� r�� x�s��y�x��`����s�v z�����r�� r�r�z�r�� �dy�v y�}�z`r.��x�s���s���z���r�y�v ySr�z�����s�}�w�v ��x�yr�z����L��x�s�}|{�v z�� x�s {~r�t�x�y.{~}�s�s�x�� s�v x�w�v z���r�z���������r�� v z���� u�xv z|{~}�s��Lr�� v }�z ” . In particle physicists’ jargon, a ���� corresponds to a data-taking period with a certainparameterization of the systems involved. The OBK canbe defined as a run cataloger.

A significative part of the non-physics informationcirculates inside the TDAQ by means of the OnlineSoftware’s Messaging super-component, which includesthe IS (Information System) and the MRS (MessageReporting System). Both the IS and the MRS use asbackbone the IPC (Inter Process Communication)package, a CORBA implementation [AMO97]. The MRSis aimed at providing a facility which allows all softwarecomponents in the ATLAS DAQ system and relatedprocesses to report error messages to other components ofthe distributed TDAQ systesm. On the other hand, the ISprovides a mean of inter-application informationexchange in the distributed environment. The structure ofthe IS and MRS messages is highly flexible andparameterizable.

Other non-physics data sources which are interestingfrom the OBK’s point of view are the Configuration

Databases. Since this component manages configurationdata for all of the TDAQ, some of that data will berelevant for correctly cataloging runs. The configurationdatabases component relies on OKS (Object KernelSupport) [JON97] for database schema definition an datapersistency. OKS is an Online Software homegrownsolution and acts as an object oriented in-memorypersistent object manager.

IS, MRS and Configuration Databases make dataavailable through C++ APIs which the OBK needs to use.Figure 2 depicts the interaction of the OBK with theseOnline Software components.

¡�¢¤£¦¥�§�¨�©�ª�«¬©�§�­¦®�®�¯�°±¯�§�§�²�§�³

´�µ ¶¸·L¹�º�¹�» ¼�½¿¾¦¹gÀ�Á�Â�½�½�Ã�¼�Â�Ä�ÅWork on the OBK component started in 1996, within thecontext of the DAQ/EF-1 [MOR97] project which aimedat building a prototype for a full vertical slice of the DAQsystem. The implementation of this prototype had also asobjectives the study of technological and architecturaloptions, along with software development methodologiesand environments.

The Online Software adopted a software developmentprocess with the following phases: problem statement;analysis; design; implementation; testing; maintenance.Work on the OBK followed this process loosely, applyingthe described phases to each of the three implementedprototypes.

Generically, the process used to develop the OBK canbe described as prototypical spiral, in which eachimplementation follows the cascade approach whilebuilding on reviewed requirements and previousexperience. This sort of approach seems appropriate to along term project in which the requirements for thecomponent evolve as the full system grows. To be notedthat maintenance plays as very important role in everydaylife at the Online Software, as the software is frequently

LVL1

Detector

DataFlow

SCADA

LVL2

EF

Online Sw.

RunControl

Messaging

Monitoring

Databases

Ancilliary

Æ�Ç ÈÊÉ�ËÍÌÏΖ Simplified UML package diagram of the Online Software

IS

MRS

Conf. DB

OBK

Ð�Ñ ÒÊÓ�ÔÍÕ�Ö– Simplified UML package diagram of the OBK

required to supervise simulated and real data acquisitiontests.

Another output of the development process other thanthe code is the documentation. For the OBK, documentssuch as the Implementation Notes, the User Manual andTest Reports (according to a predefined Test Plan) areproduced at each prototype iteration. Some of them willbe refered to in this paper as they include interestingresults.

The Online Software makes use of a number of toolsto support the development. For example, CVS(www.cvshome.org) is used as the code repository, SRT(http://atddoc.cern.ch/Atlas/DaqSoft/sde) and CMT(http://www.lal.in2p3.fr/SI/CMT/CMT.htm) – bothCERN homegrown solutions – for platform dependencyand release management, Rational Rose and Together forhigh level design, Perl and Unix shells for scripting,Framemaker for documentation generation, etc. This,added to a multiple-language programming environment,makes the software developing environment for the OBKquite heterogeneous and advanced.

×�Ø Ù¸Ú�Û Ü�Ý¦Þ ß�à�ß�Þ�á�ß�â�ã�Û á�ßgä�ß�å�æ�ç�è�å¿é¦Ü�ß�å�ß�á�Û ê�è�á�ê�Ý�Û æ�ß�ê�æ�ã�á�ßIn section 2.2 a very high level description of therequirements for the OBK has already been provided.This paragraph extracted from the technical proposal(latest publication in [ATL00]) was the departure pointfor the work on the OBK.

The starting round of formal requirements collectionwas performed in 1997 [AMO97]. The resultingdocument provided a first refinement of what the OBKshould do. A brief summary of that document is asfollows:

• The OBK shall record information on a per-run basis.This information should include: run configuration;trigger state and configuration details; recordingdevice details; run dates (start/stop date and time);accelerator beam information; data quality; errorstatistics; structured user comments;

• The OBK shall provide access to the archivedinformation for DAQ and detector groups via anumber of interfaces which may include a generaluser interface and APIs;

• The OBK shall allow the data taking periodcoordinator to modify the information stored for aparticular run;

• The OBK will require the retrieval of informationfrom the following DAQ components: MRS,Configuration Databases and Accelerator;

• The OBK shall be accessible during and outsideDAQ data taking sessions.

From 1997 to today, the Online Software (at the timeDAQ/EF-1 Backend) evolved in terms of design,technology, performance and scalability. Being an OnlineSoftware component, the initial requirements posed on theOBK also evolved. A formal review on the OBKrequirements is in progress at the time that this paper isbeing written – this review will make it possible toincorporate requirements that came up during the threeprototype iterations and also include new desired features.

Although the requirements revision is still takingplace, it is already possible to say that the new set ofrequirements is mainly a constrained version of the firstdocument. The main additions can be resumed in thefollowing points:

• Some of the data required to be stored in the firstdocument is now removed. However, the definitionof data quality is enhanced and the OBK is alsorequired to store data samples (histograms) for offlinedata quality checks;

• The fashion in which the OBK acquires data is betterspecified: the tool should be able to gather data eitherby collecting TDAQ circulating messages which areinteresting or by requesting their productionexplicitly;

• External frameworks and tools for which the datastored by the OBK is interesting are named. Thismeans that interfaces with those frameworks or toolsshould be investigated;

• The need for the implementation of a web-basedinterface to the stored data is explicitly mentioned.

From these requirements a simple generic architecture forthe OBK can be devised. A graphical representation ofthis architecture can be found in figure 3.

DBMS

OBK acquisitionsoftware

C++ APIWeb BrowserOther interfaces…

ë�ì)í î ì)ï�ð.ñ)ò�ó ôöõ�÷�ï

ø�ù úÊû�üÍý�þ– Generic OBK architecture

The architecture of the OBK consists then of anapplication which subscribes to various data sources andstores into a DBMS online relevant data for runcataloging. It also makes available several interfaces toretreive that data offline.

ÿ�� ÿ������� ��� ��� ���� ����������� �During the 4 years of existence of the OBK project threedistinct prototypes were implemented. All of them arecoded using C++ but different DBMS technologies:Objectivity/DB (www.objectivity.com) , OKS andMySQL (www.mysql.com). The advantages behind thismulti-technology approach are to to gain technologicalexpertise about different DBMS technologies and ingeneral to be able to make a solid recommendation for atechnological and design solution for a production book-keeper tool.

All the OBK prototypes are available in the OnlineSoftware’s current release (0.0.16) – the user can choosewhich one to use by setting an environment variable.��� ��� � ������� � ! " ! #$��%�&'� (As was mentioned in 1.1, Objectivity/DB was selected bythe LHC experiments study the management of long-termdata. Naturally, the first OBK prototype to beimplemented was based on this DBMS.

Being a pure Object Oriented DBMS, Objectivity/DBprovided the possibility to explore the persistent Object-Oriented (OO) data model. Despite its power andflexibility, this data model has less acceptance than theRelational model, mainly because of the state of maturityand ease of use of the latter. The reason why particlephysicists invested strongly in OO persistency in the1990s resides in the fact that this data model is veryflexible and can describe complex particle physicsinformation better than the relational one. An example ofstrong usage of Objectivity in HEP is the BABARexperiment [GAP01] taking data in the United States.

This first OBK prototype was designed to take fulladvantage of the data model proposed by Objectivity/DB,which is organized into federations, databases, containersand objects. The mapping of the collected data was quitenatural and intuitive.

Some highlights on the experience with Objectivityare the following:

• Objectivity/DB’s schema definition mechanism(ODMG compliant) is very powerful and reduces thedifference between transient and persistent objects.This enables the programmer to set a mindframe on ahigh-level language paradigm – OO – throughout allthe development process;

• Objectivity/DB databases at CERN are concentratedin a small number of machines with mass storage

connections (CASTOR) and running AMS(Advanced Multithreaded Server) for remoterequests. The Objectivity OBK prototype made use ofthese services during it’s deployment;

• Being a commercial tool, Objectivity/DB provides alarge amount of features, including graphicalbrowsing utilities, transaction and recoverymanagement, etc. It has however the disadvantages ofnot running on all the platforms supported by theOnline Software and also requiring a commerciallicence, which raised problems within an OpenSource developing environment such as the OnlineSoftware.

The Objectivity based OBK prototype was succesfullyused for the 2000 test-beam, a full scale test for theTDAQ involving a functional subdetector of the ATLAS.A web-based browser has also been developed as part ofthe prototype.)�* )�* + ,�-/.10�2�3'4 5The second OBK prototype used the OKS DBMS forpersistency. OKS is an in-memory persistent objectmanager implemented to satisfy the specific needs of theATLAS TDAQ in terms of configuration databases. Thestrength of the OKS lies in being lightweight and orientedto clients which pose strong efficiency and real-timerequirements. The data model adopted by the OKS is alsoObject Oriented, although not as sophisticated asObjectivity/DB’s. At the moment, the OKS is the adoptedsolution to manage ATLAS TDAQ configurationdatabases. Data files (containing objects) are stored asXML files in the filesystem (local, AFS or NFS). Accessto the data is done by explicitly reading the data files, notthrough a centralized server.

The motivation to use OKS for OBK persistency camefrom the fact that the commercial licence required byObjectivity/DB posed problems when it was necessary torun the OBK outside CERN (where the licenses are notvalid). Another reason for choosing OKS is thatObjectivity/DB ships by default with too many featuresthat the OBK does not use. All this extra functionalitycreates an overhead for the OBK and for the OnlineSoftware in general. Since the OKS package is already apart of the Online Software, compilation, linking andshipping of the code becomes straightforward.

Concerning the technical aspects of theimplementation, the mapping of the database schemapreviously defined for Objectivity/DB into OKS wasrelatively simple, given that both DBMSs use the OO datamodel. The mapping of the Objectivity/DB concepts ofFederation, Database and Container on which the firstprototype relied posed however some challenges, giventhat the OKS only has two levels in its data model

hierarchy: data file and object. Also, since concurrencycontrol is not implemented for OKS, that responsibilitywas passed onto the OBK’s code.

This second implementation includes somesignificative improvements on the first one: a morefunctional web browser, a C++ API, dealing with dataquality and more object-oriented design.

When used in the 2001 test-beam the componentbehaved well, acquiring several megabytes of data whichwere stored on the machine’s local filesystem and later onAFS. The scattering of data across filesystems washowever a problem that soon arose.6�7 6�7 6 8:9<;�=?>A@�B�C'D EFinally, the most recent OBK prototype uses the MySQLDBMS for data storage. MySQL is an Open Sourceproduct known worldwide for its speed and ease of use.Inside HEP it is also extensively used for metadatamanagement solutions, as will be pointed out in section 5of this paper. MySQL implements the Relational datamodel and access to its data is via a centralized serverwhich processes SQL queries.

The decision to implement a MySQL based prototypewas triggered not only by power of the underlying SQLengine, but also by the desire to try a relational approachwhile dealing with the OBK data.

Technically, the mapping of the predefined OO OBKdatabase schema into the relational model requiredsubstantial redesigning, as the two models are essentiallydifferent. From our point of view the relationalimplementation of the schema is less elegant than theprevious approaches.

The code itself is however less voluminous andcomplicated than previous approaches, given that theC/C++ API provided by MySQL enables the execution ofhigh-level SQL queries instead of forcing the directmanipulation of persistent/transient objects.

The MySQL OBK prototype is work in progress. It issupposed to encompass all the functionality of theprevious prototypes and also to address the newlycollected requirements (see section 3.2). At the momentthat this paper is being written the acquisition and webbrowsing facilities are already available and integratedwith the rest of the Online Software.

In principle the MySQL OBK prototype will be usedfor this year’s (2002) test-beam.F�G F�G HJILK K M N'NPO Q1O R�MSN'O Q�T'M U1U�V�O VThe OBK makes available several data retrieving utilitiesand interfaces: command line utilities, a C++ API and aweb browser.

The command line utilility is an executable binary whichthe user can pass several parameters in order to choosewhat OBK data to retrieve (dump to the screen). Althoughnot very sophisticated, this utility can be used for examplein debug situations or when not many resources areavailable.The C++ API consists of a shared library the user can linkwith his/her C++ applications to have access to the OBKdata as C++ objects. The library makes heavy use of STLto compose the query return structures.The web-based OBK browser makes available a graphicalversion of the data through a regular HTTP client(Netscape, Iexplorer, etc.). The mechanism relies on PHPscripts to access the database along with the ApacheHTTP server to make the information available in theweb. We currently have a machine deployed to providethis service. In figure 4 a snapshot of the browser can beobserved.

W�X Y�Z�[ \'] ^�_�`�_a�]cb�dfe [gb�[g_h�`�h�] i�_�] \'\'j[g\In this section we comment on some of the design andimplementation issues we found to be more relevant whiledeveloping the three prototypes. Although some of theremarks in the following list may seem to ovelap withsection 3.3, we try to pose them in a more generic fashion:

• k�l�m�l�nl�o'p�o'q rpgs�l�t as mentioned before, the OO datamodel was found to be quite flexible for mapping theOBK data. The resulting database shemas (forObjectivity/DB and OKS) are very natural since eachtype of data to be stored is modelled into a class. Thatrelates to other classes in the schema throughinheritance or composition. The Relational datamodel proved to be less easy to use: data types suchas collections had in fact to be treated as XMLstrings, given that the data model does not support the

uwv xzy|{~}<�– Snapshot of the OBK (OKS implementation) web browser

concept. The relational OBK database schema is lessnatural than the OO one, given that classes had to befitted into tables and also that optimizations wereperformed in order to facilitate access to the data;

• ������ � we found a very noticeable tradeof betweenthe elegance of code achieved with the OO datamodel and the reduced amount/complexity of thecode in the relational prototype;

• �/����� ���� �������P����1�� �'� ���� while within the context ofan online system, a strong requirement for the OBKis light weight in terms of processing and primarymemory usage. To fulfill this purpose we tried toevolute the design in terms of minimizing accesses tothe database while keeping in primary memory theleast possible amount of data. In terms of dataretrieval we found that using caches for frequentlyaccessed run parameters reduced drastically readtimes;

• ���'�����������/  in the MySQL prototype, XML wasused to cope with the difficulty of storing collectiontypes, not available in the Relational data modelimplemented by that DBMS. Despite not being tooelegant, this technique seems to simplify storage ofdata during acquisition and could even be used inconjunction with the previous OO prototypes to avoidthe creation of too many simple data objects.

¡�¢ £�¤¦¥ § ¨�©�§gª�«�¬­ ¥S«�¬®�¯'­ «�° «�±² ° ² ³�´1² ¯�¯'µ¥g¯The Objectivity/DB and OKS OBK prototypes were usedin test-beams and Online Software scalability tests, as partof the Online Software. Although some bugs were found,no major performance or scalability problems wereidentified. However, these tests envolved only a smallfraction of the final system the production OBK will haveto handle. To be able to quantify the capabilities of theOBK we also performed tests in a controlled environment.Some of the results are described in the following lines.

Figure 4 shows the storage time of an important MRSmessage for the three prototypes. The message in questionindicates the start of a new run and triggers heavyprocessing within the OBK. As can be seen, theObjectivity/DB prototype is the slowest while the MySQLone is the fastest. For the Objectivity/DB and OKSprototypes a non-desirable growth in storage time can alsobe observed. We attribute these differences mainly to theevolution of the design from prototype to prototype. Wethink that better tuning of the OO OBK solutions wouldplace them closer to the MySQL one in terms ofperformance. More about these tests can be found in[ALE01].

Scalability problems can exist in two different forms forthe OBK: in terms of data storage or in terms of data

sources. In what concerns data storage we are confidentthat no major problems will arise, given that metadatarepositories are by definition orders of magnitures undermain databases in terms of occupied space. In any case, ifsecondary storage is not enough, CERN makes available aterciary storage service (CASTOR).In terms of data sources, the OBK (OKS implementation)was tested to understand what happens if interesting datais produced at a rate that the OBK cannot cope with. Thetests are described in [LUC02] and involved starting alarge number of data sources simultaneously and makingthe OBK subscribe to them. When overflowed, it wasobserved that the OBK may cause delays to other OnlineSoftware components. Although the simulated situationwas much above what may be required by a productionsystem, the results are indicative that the storage time forthe OBK should be kept as low as possible.

All the above mentioned tests were performed usingPentium III client and server machines running Linux.

¶¸·º¹�»¸¼¾½À¿�Á¾Â�Ã�»¸¼¾Â�ĸ¼/Å�ÆÇÁ¾È�Á¾ÉÀÊÌËÍ»¸ÉÀÎ

The OBK is work in progress. From the start of theproject in 1996 until now effort has been put intounderstanding the problem of book-keeping for ATLASOnline Software. Given the long duration of the project,the used software developing process emphasized thestudy of alternative designs and technologies byimplementing three prototypes based on different DBMStechnologies: Objectivity/DB, OKS and MySQL. Thesealternative implementations also enabled us to easilyincorporate new requirements for the component as theyappeared in the context of the evolution of the TDAQ.

At the time that this paper is being written we thinkwe are already in a good position to recommend thetechnology and the design for a production book-keeper.However, we must also take into consideration that until2006 ATLAS’ Online Software may still change and thattechnological advances in computing happen at anextremely fast pace. An implementation of the OBK usingOracle 9i is not excluded.

ÏÐ ÑgÒwÓ�Ô?Õ – Comparative performance of the three prototypes

Ö×�Ø�ÙÛÚ�ܸÖ<ÚLÝJÖ<Ù1ÝPÞ|ßzßgà|áâÞ

0

0,05

0,1

0,15

0,2

0,25

0 50 100 150 200 250ã�äÇåçæ�è'é|êÇë ì�äÇí

îï ðñò óô Object.

OKSMySQL

õ ö ÷'ø'ù'÷ûúýü þ ÿzøú'ÿ����çþ ü ü ���¦þ ø úû÷ýù��ÿ����� �'ú~ü þ �'þ ü �

Also to be mentioned that the most important relevantquality of metadata is that it should make data moreaccessible to the user. Given that the ATLAS TDAQsystems are now starting to move torwards integration, inthe close future we will have to concentrate on how theOBK data will integrate with the main data repositories,i.e., how to make the OBK data useful.

������������������� ��!�"In this section, some examples of what is being currentlydone in the area of metadata repositories in HEP areprovided. The set of mentioned systems is interestingfrom the point of view of the OBK and does not by anymeans try to represent the general state of affairs in thearea.

Inside ATLAS two systems which are very close tothe OBK are the #%$ &('($ )+*-,�.(/(02143(565(798�79:9:<;�:9= [ALB01] andthe >@? A B9C9D(A�E@F(G(G(H9I�H9J9J<K�J9L [SOL01]. Both of them store runcatalog data and use as underlying technologies PHP andMySQL.

Another interesting metadata repository being used inATLAS, this time for cataloging and replication ofdistributed data, is the MON(P(Q(N project [WEN02]. Magda iscurrently being used by the offline ATLAS DataChallenges for keeping track of event data generated in asimulation context. Technologies used include MySQL,Perl, Java and C++.

Finally, although the RTS�U V WXU Y�Z project of the DataGridData Management Work Package [HOS01] is not ametadata repository on its own, it aims at providingmiddleware for the integration of heterogenous SQLbased metadata repositories. Spitfire’s purpose is to act asan intermediary between clients of metadata databases invery large distributed systems. Underlying technologiesinclude (among others) Java, XML and MySQL as thedefault database backend.

[�\�]T\�^�\�_a`�\�b[ALB01] Solveig Albrand and Jerome Fulachier,“Bookkeeping Database Search Interfaces” , URL:http://larbookkeeping.in2p3.fr/

[ALE01] Igor Alexandrov et al., “Experience usingdifferent DBMSs in prototyping a Book-keeper forATLAS’ DAQ software” , ced�f(g9h9h9i(j k(l(mnfporqtsvuecxw(y6y(z ,Beijing, China, 2001, pp 248-251.

[AMO97] Antonio Amorim and Helmut Wolters,“Requirements for the Run Book-keeper system for theATLAS DAQ prototype –1” , URL:

1 A subdetector of the ATLAS, specialized in observingcertain characteristics of the physics events.

http://atddoc.cern.ch/Atlas/DaqSoft/document/URD_27.html

[ATL00] The Atlas Collaboration, “High-Level Triggers,DAQ and DCS – Technical Proposal” , {@|9}9~(�(� }9�(��e���T���(� �(� , CERN, Geneva, 2000, pp 165-168.

[BAR97] Antek Baranski, “Metadata” , URL:http://wwwinfo.cern.ch/asd/cernlib/rd45/SlidesWorkshopJuly97/Metadata/index.htm

[GAP01] Igor Gaponenko et al., “The BABAR Database:Challenges, Trends and Projections” , �e���(�9�9�9�(� �(�(���p��t�v� ���(�(�(� , Beijing, China, 2001, pp 9-14.

[HOS01] Wolfgang Hoschek and Gavin McCance, “GridEnabled Relational Database Middleware” , presented atthe �v� �(�(�(���v��  ¡£¢e�(��¤(¥ , Frascati, Italy, 2001.

[JON97] Robert Jones at al., “The OKS in-memorypersistent object manager” , ¦§e§e§©¨@ª�«(¬(­�«(®9¯ ° ±(¬(­²±(¬³£´ ®9µ ¶9«(ª¸·(®9° ¶9¬(®9¶9¹�º»±(µ ¼ ½6¾(¹¿¬6±(¼ ½(¹tÀ ´(Á6´ ­�¯nÂ(Ã(Ã(Ä , pp 1958-1964

[LUC02] Levi Lucio et al., “Test Report of the OnlineBook-keeper for the Atlas DAQ Online Software” ,Å-ÆÈÇ@Å-É£ÊËÅ-Ì ÍÎ(Ï Ð9Ñ�Î(Ò(ÓÔ£Õ(Ï Ð×Ö�Ø(Ø

, CERN, Geneva, 2002.

[MAL97] David Malon and Eduard May, “CriticalDatabase Technologies for High Energy Physics” ,ÙeÚ�Û(Ü9Ý9Ý9Þ(ß à(á(âãÛpä�å æ(Ýèç(é4ê ëíì�î%ïvðòñtó(ôpõXö9÷�ö9ô(ø9ö

, Athens,Greece, 1997, pp 580-584.

[MOR97] G. Mornacchi et al, "The ATLAS DAQ andevent filter prototype "-1" project", 10th IEEE Real TimeConference, Beaune, France, 1997

[NOW01] Marcin Nowak et al, “Object Persistency forHEP Data using an Object-Relational Database” ,ùeú�û(ü9ý9ý9þ(ÿ ����� û������eù �������

, Beijing, China, 2001, pp272-275.

[OMG02] Object Management Group, “The OMG’sCORBA website” , URL: http://www.corba.org

[SHI98] Jamie Shiers, “Building a Multi-PetabyteDatabase: The RD45 Project at CERN”, �������������� � � � ��� � ��� � �"!#� ��� � ��� , Prentice Hall, 1998, pp 164-176.

[SOL01] Igor Solovianov, “ATLAS Tile CalorimeterRun Information Database” , URL:http://tileinfo.web.cern.ch/tileinfo/runinfo.php

[WEN02] Torre Wenaus, “Magda – Manager for Grid-based data” , URL: http://atlassw1.phy.bnl.gov/magda/info