SEAL Project Overview
Lorenzo Moneta/ CERN-EP on behalf of the SEAL team
ACAT03
IX International Workshop on Advanced Computing and Analysis Techniques in Physics Research
1-5 December, 2003KEK
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN2
Outline
Introduction – Scope and motivation
SEAL Work Packages – Description – Current status
Future plans and summary
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN3
Project motivation What is SEAL ?
– project in LCG Application Area – started last year after BluePrint RTAG recommendations– common project on core tools and services
» SEAL: Shared Environment for Applications at LHC SEAL aims to
– Provide the software infrastructure, basic frameworks, libraries and tools that are common among the LHC experiments
– Select, integrate, develop and support foundation and utility class libraries
– Develop a coherent set of basic framework services to facilitate the integration of LCG and non - LCG software
Do not re-invent the wheel– Try to use as much as possible of existing software– Develop/adapt/generalize to achieve necessary level of
coherency
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN4
Project scope
Foundation Class Libraries– Basic types (STL, Boost,…), utility
libraries, system isolation libraries, domain specific foundation libraries
Mathematical Library– Basic mathematical functions,
linear algebra, fitting and minimization libraries
Basic Framework Services– Component model, reflection, plug-
in management, incident (event) management, distributed computing, grid services, scripting
Basic Framework
Foundation Libraries
Simulation Framework
Reconstruction Framework
Visualization Framework
Applications
. . .
Optional Libraries
OtherFrameworks
Software Structure
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN5
Project Work Packages
FoundationFoundation and Utility Libraries and Plug-in Manager
MathLibs Math Libraries Support and Coordination
Dictionary LCG Object Dictionary
Framework Component Model and Basic Framework services
Scripting Scripting Services
Grid Grid Services
Documentation
Education and Documentation
Not yet active
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN6
Foundation and Utility Libraries
Inventory of existing C++ utilities (http://seal.cern.ch/components.html)
– Recommended classes useful for LCG projects and experiments – Large number from Boost (open source project)
» Support for Boost is provided (in contact with developers) Develop in SEAL what is missing in STL and Boost Have a set of low-level, fairly independent class libraries
200 classes originated mainly from classlib (CMS) – Basic abstraction and foundation utilities classes– Timing utilities – Stream-oriented I/O, compression and decompression
High quality, efficient, well tested and documented library – Recommend best practice to LHC developers
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN7
Plug-in Manager
Plug-in Management– Service in charge of managing, querying, [un]loading plug-ins
Low level plug-in management (no framework constraints)– Ability to create “module” libraries containing “plug-ins”– Ability to instantiate concrete implementations knowing the
plug-in category and the concrete type– Caching information for fast loading
» what modules contain » what kind of plug-ins exist and which categories
Imported from CMS IGuana– Basic concept: advanced object factory
» Interfaces for object instantiation and plug-in provider– Dynamic loading completely orthogonal — and optional!
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN8
Mathematical Libraries GSL (Gnu Scientific Library) evaluation
– Extensive comparison with NagC– Validation and test suite
Re-implementation of MINUIT in C++– Prototype available with Migrad and Minos– Undergoing numerical and performance tests
» Comparison tests with Fortran version (compatibility within errors)
Linear Algebra– Prototype wrapper around GSL matrices. – Comparison of various linear algebra packages (CLHEP, GSL,
[u]CBLAS) Participation in CLHEP project Provide to experiments math and statistics libraries to be
used in analysis, reconstruction and simulation. – support, consultancy, etc..
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN9
Object Dictionary
Adding “introspection/reflection” capabilities to C++ Use for persistency and interactivity (e.g. Python) Modern languages provide reflection inherently
– In C++ reflection is very limited (RTTI) The basic C++ concepts shall be supported
– e.g. inheritance, methods, data members, accessibility, templates
Seal provides : – Reflection Builder and Reflection (API interface) packages– Dictionary generator to produce dict. source from C++ files
» Uses gcc-xml + Python script– Dictionary for some standard packages (STL, CLHEP)
Used by POOL (persistency)
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN10
gcc-xml to parse header-files (extension to gcc ) XML file generated with dictionary information Python script to generate dictionary C++ code
– use an additional XML file to customize class selection Compiled and loaded at run-time to create
dictionary – Provide meta information about the object to the user
Dictionary Generator
.xml
.h
.h lcgdict.h.h
gccxml .xml filter+
exten
gendictparser
_dict.cpp ..Dict.somake
selection file
(Python script)
#include files
ReflectionLibraries Clients
User
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN11
Framework Services
Motivation: – Allow LCG developers and users to write portable and modular
code– Provide abstractions for modelling software systems
Component model definition– Basic Components
» Set of “base classes” and interfaces needed to build more complex components
» Provide standard functionality instantiation, configuration, lifetime management, etc
Users (framework developers) should model their services using these proposed base classes
SEAL provide in addition a set of basic services implemented following the component model
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN12
SEAL Component Model
Hierarchy of bases classes– Components living in a Context– Services are Components providing their local Context
User classes inherit from Component or Services– Plug-in functionality for free
ContextApplication
C1C1 C1C2
Component
Svc2C1C6 C1C7
Svc1C1C3 C1C4
C1C5
Svc1
ServiceC1C7
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN13
Basic Common Framework Services
Application Service– Defines the top level Context– Defines initial set of Components to be loaded in the application
Message Service– Message composition, filtering and reporting
Configuration Service– Management of Component properties and loading configurations– Multiple backends foreseen:
» Gaudi style options, .INI style, CMS style, XML, ... Other services will come later:
– Object Whiteboard (Transient Object repository) – Dictionary Service (Loading Dictionaries in the application)
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN14
Scripting Services Scripting needed for Interactivity, Configuration, Rapid prototyping,
Integration (Component bus), etc. Main scripting language :
– Python
Define guidelines for developing Python bindings : – Various tools available
(Boost.Python, SWIG)– Recommend solutions:
»Boost.Python »LCG Dictionary
GUI
Python
mathmathshell
PyGaudi
DatabaseEDG APIGUI
XML
GaudiFramework
PyROOT
RootClasses
PVSS
JPE
JavaClasses
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN15
Scripting Packages
PyLCGDict– Python extension module to the LCG dictionary– Provide access to C++ libraries from Python, “for
free”– Automatically generates Python proxies for C++
objects using dictionary information»Everyone can create binding effortlessly
PyROOT– Python bindings for ROOT classes
»access to ROOT functionality from Python– Binding using the ROOT dictionary (CINT)– Missing gateway ROOT -> Python
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN16
Education/Documentation
Produce documentation– Reference documentation (generated with doxygen) – HowTo pages for the released elements
» Plugin Manager, Dictionary generation, PyROOT, PyLCGDict, etc.
– Release notes Training material (tutorials)
– Help developers incorporating SEAL into the LCG projects and the experiment frameworks
Python courses– Hands-on introduction course to Python programming– Very positive feedback received
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN17
Software Process and QA
Design– Team design sessions– Prototypes using Pythons
Code Review– No formal code reviews– Coding done very often in pairs (XP style)– More than one developer knowledgeable for each package
Testing– More than 200 SEAL unit tests based on CppUnit– All are driven and run automatically by QmTest
Bug reporting and tracking– Savannah Portal (http://savannah.cern.ch/projects/seal)
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN18
Future plans (1)
Foundation libraries (Plug-in Manager) – Follow Boost developments– Little development planned in SEAL (hash-maps)
MathLibs: – Produce inventory of mathematical functions– Evolve Minuit C++ into a final product– Linear Algebra studies– Generate dictionary library for Python and CINT
Dictionary: – Implementation of new reflection model
» Overcome some existing limitations (typedef,…)– Converge to a common dictionary with CINT ?
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN19
Future plans (2) Framework:
– Develop Whiteboard (object repository) and Dictionary service (loading of dictionary libraries on-demand)
Scripting : – PYLCGDict:
» Migrate much of functional core from C++ to Python » Support more natural Python features (eg iterator protocol)
– PyROOT:» Unify core with PyLCGDict » Develop gateway ROOT -> Python
Grid : – New LCG project (ARDA) is starting– Expect need to interface to some of the ARDA services
Documentation: – Produce tutorials for foundation classes, Boost, etc..
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN20
Summary SEAL has delivered basic foundation, utility libraries
and object dictionary– The main “client” have been POOL (persistency project) and PI
» Plug-in manager is also used in CMS – Very positive feedback received for plug-in manager – Currently being integrated into experiments’ frameworks of
LHCB (Gaudi) and ATLAS (ATHENA) The first version of the Component Model and
Framework services is available– Start integrating in POOL and the LHC experiments– Wait for feedback before developing more services
Scripting based on Python– Boost.Python and PyLCGDict recommended for Python
bindings– PyLCGDict is used to provide bindings for classes of Physics
Interfaces (PI) project
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN21
http://seal.cern.ch/
Acknowledgments:Seal team members: R. Chytracek, J. Generowicz, W. Lavrijsen, M. Marino, P. Mato, S. Roiser, L. Tuura, M. Winkler
Conclusions
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN22
Additional Slides
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN23
LCG Application Domain Coverage
EventGeneration
Core Services
Dictionary
Whiteboard
Foundation and Utility Libraries
DetectorSimulation
Engine
Persistency
StoreMgr
Reconstruction
Algorithms
Geometry Event Model
GridServices
I nteractiveServices
Modeler
GUIAnalysis
EvtGen
Calibration
Scheduler
Fitter
PluginMgr
Monitor
NTuple
Scripting
FileCatalog
ROOT GEANT4 DataGrid Python Qt
Monitor
. . .MySQLFLUKA
EventGeneration
Core Services
Dictionary
Whiteboard
Foundation and Utility Libraries
DetectorSimulation
Engine
Persistency
StoreMgr
Reconstruction
Algorithms
Geometry Event Model
GridServices
I nteractiveServices
Modeler
GUIAnalysis
EvtGen
Calibration
Scheduler
Fitter
PluginMgr
Monitor
NTuple
Scripting
FileCatalog
ROOT GEANT4 DataGrid Python Qt
Monitor
. . .MySQLFLUKA
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN24
Module Library
PluginManager
PluginFactoryBase
PluginFactory
T
PluginInfo
XXXFactory XXXInfo
Module ModuleDef
ModuleBase
MyModuleDef*
*
PluginCache
Client
*
Plug-in Manager Design
Originated from CMS Iguana
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN25
PropertyListItem
MethodField
ItemBuilder
MethodBuilderFieldBuilderClassBuilder
NamespaceBuilder
Reflection
ReflectionBuilder
ClassNamespace
Reflection System
Reflection Builder: the loading interface Reflection : the user API
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN26
LCG Object Dictionary: Usage
ROOT I/O
LCGDictionaryCINT
DictStreamer
.h
in
out
ROOTCINT
CINT generatedcode Dict generating
code
.adl.xml
ADL/GOD
OtherClients:
(python, GUI, etc.)
LCG
to C
INT
Dic
t gate
way
(2)
(1)
Population
Conversion
Reflection
GCC-XML
ACAT03, 1-5 Dec. 2003
SEAL Project OverviewL. Moneta / CERN27
Seal Component Model
Hierarchy of bases classes Each Component lives in a
Context Contexts form their own hierarchy Support for component look-up A Service provides its own local
Context– Components of a Service live in the
scope defined by its own Context User classes inherit from
Component or Service– Plug-in functionality for free
Component
Service
MyComponent
MyService
Context
*
*
1
ComponentFactory
ComponentLoader
1