root: status developments1 aachen 1 february 2005 ren the root project status developments

ROOT: Status & Developments 1

Aachen1 February 2005

René[email protected]

The ROOT ProjectStatus &

Developments


ROOT in a nutshell An efficient data storage and access system

designed to support structured data sets in very large distributed data bases (Petabytes).

A query system to extract information from these distributed data sets.

The query system is able to use transparently parallel systems on the GRID (PROOF).

A scientific visualisation system with 2-D and 3-D graphics.

An advanced Graphical User Interface A C++ interpreter allowing calls to user defined

classes. An Open Source Project


ROOT: An Open Source Project

The project is developed as a collaboration between :

Full time developers: 6 people full time at CERN 1 core developer at FermiLab 1 core developer in Japan (Agilent Technologies) 1 core developer at MIT 1 mathematician at CERN sponsored by a US Finance Company

Many contributors spending a substantial fraction of their time in specific areas (> 50).

Key developers in large experiments using ROOT as a framework.

Several thousand users given feedback and a very long list of small contributions.


Project History Jan 95: Thinking/writing/rewriting/??? November 95: Public seminar, show Root 0.5 Spring 96: decision to use CINT Jan 97: Root version 1.0 Jan 98: Root version 2.0 Mar 99: Root version 2.21/08 (1st Root workshop FNAL) Feb 00: Root version 2.23/12 (2nd Root workshop CERN) Mar 01: Root version 3.00/06 Jun 01: Root version 3.01/05 (3rd Root workshop FNAL) Jan 02: Root version 3.02/07 (LCG project starts: RTAGs) Oct 02: Root version 3.03/09 (4th Root workshop CERN) Feb 04: version 4.00 (5th Root workshop SLAC) Dec 04: version 4.02 (current stable version) Feb 05: Developing 4.04 to be released in June (LCG/ROOT

merge)

10 years !!


The ROOT web pages

http://root.cern.chGeneral Information and News

Download source and binaries

Howto & tutorials

User Guide & Reference Guides

Roottalk Digest & Forum

http://root.cern.ch/

http://root.cern.ch/


Current ROOT structure & libs


CINT: the C++ interpreter


My first session

root [0] 344+76.8(const double)4.20800000000000010e+002root [1] float x=89.7;root [2] float y=567.8;root [3] x+sqrt(y)(double)1.13528550991510710e+002root [4] float z = x+2*sqrt(y/6);root [5] z(float)1.09155929565429690e+002root [6] .q

root

root

root [0] try up and down arrows

See file $HOME/.root_hist


My second session

root [0] .x session2.Cfor N=100000, sum= 45908.6root [1] sum(double)4.59085828512453370e+004Root [2] r.Rndm()(Double_t)8.29029321670533560e-001root [3] .q

root

{ int N = 100000; TRandom r; double sum = 0; for (int i=0;i<N;i++) { sum += sin(r.Rndm()); } printf("for N=%d, sum= %g\n",N,sum);}

session2.C

unnamed macroexecutes in global scope


My third session

root [0] .x session3.Cfor N=100000, sum= 45908.6root [1] sumError: Symbol sum is not defined in current scope*** Interpreter error recovered ***Root [2] .x session3.C(1000)for N=1000, sum= 460.311root [3] .q

root

void session3 (int N=100000) { TRandom r; double sum = 0; for (int i=0;i<N;i++) { sum += sin(r.Rndm()); } printf("for N=%d, sum= %g\n",N,sum);}

session3.C

Named macroNormal C++ scope

rules


My third session with ACLIC

root [0] gROOT->Time();root [1] .x session4.C(10000000)for N=10000000, sum= 4.59765e+006Real time 0:00:06, CP time 6.890root [2] .x session4.C+(10000000)

for N=10000000, sum= 4.59765e+006 Real time 0:00:09, CP time 1.062

root [3] session4(10000000)for N=10000000, sum= 4.59765e+006Real time 0:00:01, CP time 1.052root [4] .q

#include “TRandom.h”void session4 (int N) { TRandom r; double sum = 0; for (int i=0;i<N;i++) { sum += sin(r.Rndm()); } printf("for N=%d, sum= %g\n",N,sum);}

session4.C

File session4.CAutomatically compiled

and linked by thenative compiler.

Must be C++ compliant


Macros with more than one function

root [0] .x session5.C >session5.logroot [1] .q

void session5(int N=100) { session5a(N); session5b(N); gROOT->ProcessLine(“.x session4.C+(1000)”);}void session5a(int N) { for (int i=0;i<N;i++) { printf("sqrt(%d) = %g\n",i,sqrt(i)); }}void session5b(int N) { double sum = 0; for (int i=0;i<N;i++) { sum += i; printf("sum(%d) = %g\n",i,sum); }}

session5.C

.x session5.Cexecutes the function

session5 in session5.C

root [0] .L session5.Croot [1] session5(100); >session5.logroot [2] session5b(3)sum(0) = 0sum(1) = 1sum(2) = 3root [3] .q

use gROOT->ProcessLineto execute a macro from a

macro or from compiled code


Combining UI and GUI

root [0] .x session2.Cfor N=100000, sum= 45908.6root [1] sum(double)4.59085828512453370e+004Root [2] r.Rndm()(Double_t)8.29029321670533560e-001root [3] .q


The Histogram Package


The Histogram Classes Structure

1-Dim

2-Dim

3-Dim


Histograms

class THStack Long list of functions in

TH1 Plenty of drawing options Filling with string

variables Automatic binning TH1::SetBuffer TH1::GetRandom h3 = 3*h1 – h2


Filling with strings

See tutorials

-hlabels1.C

-hlabels2.C

-cernstaff.C


Peak Finder + Deconvolutionsby Miroslav Morach

TSpectrum


Fitting

TVirtualFitter

TFitter

TMinuit

TH1::Fitbin chisquarebin likelihood

TGraph::Fitunbinnedchisquare

User

TTree::Fitunbinnedlikelihood

TFumili


Graphics


Gui/Graphics strategy

TVirtualX

TGWin32 TGX11 TGWin32GDKTGQt

TVirtualPad

TPad

User/Root GUI and Graphics classesApplications see onlyAbstract Interfaces

Low levelscreen graphics

and GUI

High levelpad graphics

TG??


A Data Analysis & Visualisation tool


Graphics : 1,2,3-D functions


Full LateX support

on screen and

postscript

TCurlyArcTCurlyLineTWavyLine

and other building blocks for Feynmann diagrams

Formula or diagrams can be

edited with the mouse

R.Brun 1 Feb 2005 ROOT: Status & Developments

ROOT 3D Graphics

Simple Box

The ROOT basic 3D shapes


Alice 3 million nodes

R.Brun 1 Feb 2005 ROOT: Status & Developments

R. Brun (CERN), O. Couet (CERN), M. Gheata (ISS), A. Gheata (ISS), V. Onoutchine (IFVE), T.Pocheptsov (JINR)

ROOT 3D GraphicsText …………………

Atlas


GUI


Available Widgets Complete set of widgets:

label, icon, button, check button, radio button, picture button, button box, list box, combo box, list view, icon view, number entry, text entry, text view, text edit, tree view, tab view, scrollbar, slider, menubar, popup menu, cascading menu, statusbar, toolbar, message dialogs, file selection dialog, progress bars, tooltips, ...


Basic Widgets


Advanced Widgets Color selector dialog: TGColorDialog


GUI Examples


GUI Examples –Histogram Browser


More GUI Examples –Period System


Object Editors


Histogram Editor


Object Editors (rebinning)


Input/Output


Evolution of ROOT I/O Hand-written Streamers Streamers generated via rootcint Support for Class Versions Support for ByteCount Persistent class Dictionary written to files rootcint modified to generate automatic Streamers Support for Automatic Schema Evolution Can generate code for “DataObjects” classes in a file Support for complex C++ cases Can read files without the classes Persistent Reference pointers Support for foreign classes Full support for STL

3.00

3.01

1995

2005

3.02

3.04

4.02


Object Persistency(in a nutshell)

Two I/O modes supported (Keys and Trees). Key access: simple object streaming mode.

A ROOT file is like a Unix directory tree Very convenient for objects like histograms, geometries,

mag.field, calibrations Trees

A generalization of ntuples to objects Designed for storing events split and no split modes query processor

Chains: Collections of files containing Trees ROOT files are self-describing Interfaces with RDBMS also available Access to remote files (RFIO, DCACHE, GRID)


ROOT I/O : An Example

TFile f(“example.root”,”new”);TH1F h(“h”,”My histogram”,100,-3,3);h.FillRandom(“gaus”,5000);h.Write();

TFile f(“example.root”);TH1F *h = (TH1F*)f.Get(“h”):h->Draw();f.Map();

Program Writing

Program Reading

20010831/171903 At:64 N=90 TFile 20010831/171941 At:154 N=453 TH1F CX = 2.0920010831/171946 At:607 N=2364 StreamerInfo CX = 3.2520010831/171946 At:2971 N=96 KeysList 20010831/171946 At:3067 N=56 FreeSegments 20010831/171946 At:3123 N=1 END

demoh.C

demohr.C


A Root file pippa.rootwith two levels of

directories

Objects in directory/pippa/DM/CJ

eg:/pippa/DM/CJ/h15


Self-describing files Dictionary for persistent classes written to the file when

closing the file. ROOT files can be read by foreign readers (eg JavaRoot

(Tony Johnson) Support for Backward and Forward compatibility Files created in 2003 must be readable in 2015 Classes (data objects) for all objects in a file can be

regenerated via TFile::MakeProject

Root >TFile f(“demo.root”);

Root > f.MakeProject(“dir”,”*”,”new++”);


Automatic Schema Evolution


Auto Schema Evolution (2)


File types & Access in 4.02/xx

LocalFile

X.xml

RFIO Chirp

CastorDcacheLocalFile

X.root

http rootd/xrootd

Oracle

SapDb

PgSQL

MySQL

TFileTKey/TTree

TStreamerInfo

user

TSQLServerTSQLRow

TSQLResult

TTreeSQL


Memory <--> TreeEach Node is a branch in the Tree

0123456789101112131415161718

T.Fill()

T.GetEntry(6)

T

Memory


Tree example Event (write)void demoe(int nevents) { //load shared lib with the Event class gSystem->Load("$ROOTSYS/test/libEvent"); //create a new ROOT file TFile f("demoe.root",”new"); //Create a ROOT Tree with one single top level branch int split = 99; //try also split=1 and split=0 int bufsize = 16000; Event *event = new Event; TTree T("T","Event demo tree"); T.Branch("event","Event",&event,bufsize,split); //Build Event in a loop and fill the Tree for (int i=0;i<nevents;i++) { event->Build(i); T.Fill(); } T.Print(); //Print Tree statistics T.Write(); //Write Tree header to the file}

All the examples can be executedwith CINTor the compiler

root > .x demoe.Croot > .x demoe.C++


Tree example Event (read 1)

void demoer() { //load shared lib with the Event class gSystem->Load("$ROOTSYS/test/libEvent"); //connect ROOT file TFile *f = new TFile("demoe.root"); //Read Tree header and set top branch address Event *event = 0; TTree *T = (TTree*)f->Get("T"); T->SetBranchAddress("event",&event); //Loop on events and fill an histogram TH1F *h = new TH1F("hntrack","Number of tracks",100,580,620); int nevents = (int)T->GetEntries(); for (int i=0;i<nevents;i++) { T->GetEntry(i); h->Fill(event->GetNtrack()); } h->Draw();}

Rebuild the full eventin memory


8 Branches of T

8 leaves of branchElectrons

A double-clickto histogram

the leaf


The Tree Viewer & Analyzer

A very powerful classsupporting

complex cuts,event lists,

1-d,2-d, 3-d viewsparallelism


Tree Friends0123456789101112131415161718

0123456789101112131415161718

0123456789101112131415161718

Public

read

Public

read

User

Write

Entry # 8


Tree Friends

Root > TFile f1(“tree1.root”);

Root > tree.AddFriend(“tree2”,“tree2.root”)

Root > tree.AddFriend(“tree3”,“tree3.root”);

Root > tree.Draw(“x:a”,”k<c”);

Root > tree.Draw(“x:tree2.x”,”sqrt(p)<b”);

x

Processing timeindependent of thenumber of friendsunlike table joins

in RDBMS

Collaboration-widepublic read

Analysis groupprotected

userprivate


PROOF and GRID(s) interface


Batch/Interactive models

BatchProductionSimulation

reconstruction

Interactive

Chaoticanalysis

Interactive

batchmodel

Need experiment framework+widely available tools

Need onlywidely available tools


GRID: Interactive AnalysisCase 1

Data transfer to user’s laptop Optional Run/File catalog Optional GRID software

Optionalrun/FileCatalog

Remotefile servereg rootd

Trees

Trees

Analysis scripts are interpretedor compiled on the local machine



Remote data processing Optional Run/File catalog Optional GRID software

Optionalrun/FileCatalog

Remotedata analyzer

eg proofd

Trees

Trees

Commands, scripts

histograms

Analysis scripts are interpretedor compiled on the remote machine



Remote data processing Run/File catalog Full GRID software

Run/FileCatalog

Remotedata analyzer

eg proofd

Trees

Trees

Commands, scripts

Histograms,trees

TreesTreesTrees

TreesTreesTrees

slave

slave

slave

slave

slave

slave

Analysis scripts are interpretedor compiled on the remote master(s)


Parallel ROOT Facility The PROOF system allows:

Parallel analysis of trees in a set of files Parallel analysis of objects in a set of files Parallel execution of scripts

on clusters of heterogeneous machines Its design goals are:

Transparency, scalability, adaptability Prototype developed in 1997 as proof of concept,

full version nearing completion now


Parallel Script Execution

root

Remote PROOF Cluster

proof

proof

proof

TNetFile

TFile

Local PC

$ root

ana.Cstdout/obj

node1

node2

node3

node4

$ rootroot [0] .x ana.C$ rootroot [0] .x ana.Croot [1] gROOT->Proof(“remote”)

$ rootroot [0] tree->Process(“ana.C”)root [1] gROOT->Proof(“remote”)root [2] chain->Process(“ana.C”)

ana.C

proof

proof = slave server

proof

proof = master server

#proof.confslave node1slave node2slave node3slave node4

*.root

*.root

*.root

*.root

TFile

TFile


Google: a good modelMake it simple

Simple interfaceAvailable

everywhereHidden Parallelism

Distributed DBDon’t know a priorithe data location

Fast


I did not talk about


Currently in ROOT 4.02 SQL interface (MySQL, Oracle, postgres,sapdb) Threads Networking Mass Storage interfaces (Dcache,Castor,Chirp..) Mathlibs, Physics vectors Linear Algebra The geometry package Virtual MonteCarlo Interfaces to event generators The GUI Builder Neural Network Image processing PyRoot (The ROOT/Python interface) Carrot: ROOT web interface


Current developments (4.03/4.04)

(in June release)


Consolidation Robustness Zillions of tiny details User support


OpenGL interface A lot of effort invested in the GL interface Real Time navigation in complex detectors

with millions of physical volumes. Show only what makes sense

Invisible, translucent, variable precision rendering

Dynamic particles High quality output GUI


Mathlib Consolidation of TMath (now a

namespace) Improved algorithms Robust Fitters (from the R package) Outliers, cluster finders Interface with a subset of GSL Collaboration with SEAL/Math


Image processing Collaboration with Sasha Vasko for the

development of libAfterImage Fast graphics (also in batch): an

alternative to X11. Will be used by Carrot for execution of

remote scripts and fast interactive graphics on the web.

Could be used for fast image manipulation (better than large scatter plots)


xrootd Joint development with BaBar Robust and efficient distributed file server

with load balancing. Could be integrated with Castor. Integration with PROOF Requires consolidation of authentication

and security. Implementation of read-ahead for ROOT

Trees.


PROOF Add support for all features currently

available in ROOT Trees (friends, event lists, selectors)

Special (but growing case) of one client, one server with no slaves

Stateless connections with multiple PROOF objects in the same ROOT session (long and short queries).

View/Import results of batch queries Save ROOT/PROOF session with resume

capabilities. Interfaces with GRID services.


GUI Continuous improvements. More & more objects edited via the GUI. GUI objects persistent. Currently the GUI can be saved as a C++

script. We would like to save it as a canvas.root too.

More features in the GUI Builder (+ doc)


SQL Interface RDBMS access via a set of abstract base

classes TSQLServer, TSQLResult and TSQLRow

Concrete implementations for MySQL and Oracle exist TMySQLServer, TMySQLResult and

TMySQLRow TOracleServer, TOracleResult and

TOracleRow


New TTreeSQL classTTreeSQL tree(const char *db,const char* uid,…)

tree.Print(), Browse, Scan, etc

tree.Draw(“var1:var2”,”varx <0”)

TTree query style converted to SQL

Connect to an existing db

TTreeSQL tree(“mysql://localhost/test”,”nobody”,”new”);

Event *event = new Event;

tree.Branch(“top”,”Event”,&event);

tree.Fill();

tree.AutoSave();

A TSQLRow is filled and sent to the server

Columns created using the normal split

algorithm.Blobs created below

split.

Create the data base on server


CINT Masa Goto is now releasing a new version

of CINT with better C++ support, in particular for namespaces and scoping rules.

Only the byte code option supported


ROOT and the LCG projectsApplications

managerArchitects

forum

Applicationsarea

meetingSimulation

projectPI

projectSEAL

projectPOOLproject

SPIproject

decisionsstrategy

consultation

ROOTUser - provider

LCG & ROOT in 2002,2003,2004

But substantial changes this month….


PH/SFT group reorganisation

VMC

G4

GRID

catalogscollections

storage

dict

pgm

POOL

ROOT

SEAL

SIMU

Merged dictand pgm

mathlibmove to

move to

move to

useuse

use

New SFT

generators


Final remarks 1965: CERNLIB 1975: HBOOK and other packages 1985: start of PAW project 1995: start of ROOT project 2005: ROOT & LCG common

developments 2015: ??


ROOT: 2005-20XX Strong support from CERN/FNAL System will be consolidated, continuously

developed. Better and better UI and GUIs, including

web. More and more support for parallel

distributed computing. What about C++ ?

root: status developments1 aachen 1 february 2005 ren the root project status developments

Documents