user analysis environment

Lucas Taylor, Northeastern University

User Analysis Environment

27-28 October 1999, CERN

1st Internal Review of CMS Software and Computing

User Analysis EnvironmentUser Analysis Environment

Lucas Taylor

Northeastern University





IntroductionIntroductionIntroductionIntroduction

“User Analysis Environment” data analysis and presentation histograms, ntuples/tags, and plotting interactive 2D/3D detector and event visualisation graphical user interfaces statistical and numerical analysis

“IGUANA” (Interactive Graphical User Analysis) Recently created CMS project (May 1999) Together with LHC++, addresses CMS needs for “User Analysis

Environment” Follow/collaborate with related activities

(AIDA, HEPVis, Hippodraw, JAS, OpenScientist, ROOT, WIRED,...)





IGUANA PhasesIGUANA PhasesIGUANA PhasesIGUANA Phases

1) Select Baseline technologies (completed) design should allow for reasonable mixing of technologies...but... should concentrate on one specific set of tools early on

trying to be overly general can mean nothing concrete gets done!

2) Prototypes (ongoing) test each choice of technology with a realistic (if simplistic) prototype test the mixing of the various technologies with prototypes

3) Development / deployment of applications (started) should start to evolve naturally from the prototypes early involvement of end-users (don’t make products nobody wants)

Functional prototype milestone: June 2000





OpenGLX11

qxt Qwt SoQt

Xt OpenInventorQt

Objectivity

GEMINI

NAG_C

HEPVis

HepODBMS

HTL QGL

IGUANA / CMS / User libraries

HEPFitting

IGUANA Component Software StrategyIGUANA Component Software StrategyIGUANA Component Software StrategyIGUANA Component Software Strategy

Consistent with LHC++ strategy (and many specific components) Combine (only the required) components to build a given application Develop/acquire, test, deploy, maintain, & replace components individually Leverage non-CMS efforts (commercial/free SW, common projects)

MasterSuite

HEPInventor





IGUANA Baseline Toolkit forIGUANA Baseline Toolkit forGraphical User InterfacesGraphical User Interfaces

IGUANA Baseline Toolkit forIGUANA Baseline Toolkit forGraphical User InterfacesGraphical User Interfaces

Evaluation of widget sets led to choice of: QtQt Extensible OO GUI Toolkit with C++ API Full set of “standard” widgets Support for Unix/X11 (incl. Linux) and Windows 9x / NT Free for all Unix platforms (~$1500/developer for NT) Excellent documentation and examples High-level 2D drawing classes with good off-screen rendering

Already a lot of interest from LHC++, ATLAS, HEPVis,... (since presentations at HEPVis99 and subsequent discussions)





IGUANA PrototypesIGUANA PrototypesIGUANA PrototypesIGUANA Prototypes

Prototypes should demonstrate (or not!): functionality of individual technologies

(component-like) inter-operability of various technologies to make realistic applications

1) Embedding OpenGL and OpenInventor Components in Qt

2) Embedding Xt and Motif Components in Qt

3) Multi-document architecture

4) Interactive 3D Detector and Event Visualisation with ORCA

5) 2D Display for Graphical Online Monitoring and Control

6) Retrieval & Display of OO Histograms/Tags from Objectivity





Prototype 1: Embedding OpenGL / Prototype 1: Embedding OpenGL / OpenInventor Components in QtOpenInventor Components in Qt

Prototype 1: Embedding OpenGL / Prototype 1: Embedding OpenGL / OpenInventor Components in QtOpenInventor Components in Qt

OpenInventorObjects

Onlinehelp

OpenGL renderwindow

Graphics hierarchymanager

VisibilitycontrolRotate

Zoom

Pick Manipulate

Seek

Perspective / Orthographic

“QGL” extension for OpenGL http://www.troll.no/qt/opengl.html

OpenGL windows can be created & managed as any other Qt widget

“SoQt” extension for OpenInventor http://www.sim.no/

Needs OpenGL extension Needed some “tweaking”

Aside remark:LiveConnect extension for plug-ins http://www.troll.no/qt/nsplugin.html

For both Navigator and Explorer Full Qt functionality on Unix,W9x/NT





Prototype 2: Prototype 2: Embedding Xt / Motif Components in QtEmbedding Xt / Motif Components in Qt

Prototype 2: Prototype 2: Embedding Xt / Motif Components in QtEmbedding Xt / Motif Components in Qt

A potentially useful feature for legacy or third-party widgets

IGUANAsheet

manager

IGUANAObjectivity / HTL browser

“SciPlot” Xt widgetwrapped by Qt widget exploits qxt extension

(Troll Tech) http://www.troll.no/qt/xt.html

“SciPlot” Xt widgetwrapped by Qt widget exploits qxt extension

(Troll Tech) http://www.troll.no/qt/xt.html





Prototype 3: Prototype 3: Multi-document ArchitectureMulti-document Architecture

Prototype 3: Prototype 3: Multi-document ArchitectureMulti-document Architecture

Common services

Sheet management

Customisedservices





Prototype 4: Interactive 3D Detector and Prototype 4: Interactive 3D Detector and Event Visualisation with ORCAEvent Visualisation with ORCA

Prototype 4: Interactive 3D Detector and Prototype 4: Interactive 3D Detector and Event Visualisation with ORCAEvent Visualisation with ORCA

Reconstructed (ORCA) object

Generic (GEANT3)detector display





Prototype 5: 2D Display for Graphical Prototype 5: 2D Display for Graphical Online Monitoring and ControlOnline Monitoring and Control

Prototype 5: 2D Display for Graphical Prototype 5: 2D Display for Graphical Online Monitoring and ControlOnline Monitoring and Control

Push Button Check box

Drag&droptoolbars

Pull-downmenu

List views

SliderStatus bar

Multi-linetext editor

Paint window

Radio box

Scrollbar

XML rich text

Tabbedwindow

Postscript (vector)

Demonstrate with a pseudo-realistic application:

the wide-range of Qt widgets

2D drawing capabilities

short development time (~1 week)





Dis

pla

yC

reat

eS

tore

Bro

wse

ORCA Application

Persistent HTL (& HepODBMS/Objectivity)

Transient HTLHBOOK4

HEPInventor

(& OpenInventor)

ZEBRA

file

SciPlot

(Xt in Qt)

HEPExplorer

(& IRIS Explorer)

Objy HTL browser

(Qt component)

Prototype 6: Retrieval & Display of OO Prototype 6: Retrieval & Display of OO Histograms and Tags from ObjectivityHistograms and Tags from Objectivity


Objy HTL browser

(Motif “component”)

Qwt

(pure Qt)

New widget

(pure Qt)

PAW

IGUANA OO prototypes

6(a)6(a)6(a)6(a) 6(b)6(b)6(b)6(b) 6(c)6(c)6(c)6(c) 6(d)6(d)6(d)6(d)







6(a) HepExplorer and IRIS Explorer







6(b) HepInventor and OpenInventor

6(c) SciPlot (Xt / Motif)







6(d) Qwt and Qt

Browsing persistent OO (HTL) histograms

in Objectivity DB

Browsing tag data in Objectivity DB and

displaying histograms interactively







None of the prototypes was a complete solution...but the Qwt (Qt-based) solution was preferred

Fully OO (not even wrapped), hence easily extended Well-integrated with the rest of the GUI Only Qt is required

No need for OpenGL, OpenInventor, Mastersuite, IRIS Explorer,... Good vector hardcopy (Postscript)

Strategy for Histogram display Develop a new Qt-based histogram widget (light-weight and free) Collaboration with others (LHC++, etc...) For “3D views of plots” (e.g. Lego plots) evaluate:

Simple system based on Qt Fully functional 3D system (OpenInventor)





From Prototypes to Project: From Prototypes to Project: Software Configuration ManagementSoftware Configuration Management

From Prototypes to Project: From Prototypes to Project: Software Configuration ManagementSoftware Configuration Management

Software repository CVS-based (similar to ORCA) Minimise external dependencies for each package

(library)Build and release system

SCRAM-based (similar to ORCA)Documentation

doxygen auto-documentation tool examples in progress

HEPVis

qxt

SoQt

qwt

IgQt

Ig_ExtensionsIg_Imports

QGL

IgHTL

IgObjectivity

IGUANA

Ig_Examples Cmscan

IgMotif

IgXt

IgGeant3qextmdi

Ig3DViewer

IgExDBrowser

IgTagBrowser

Examples

IgCmscan





IGUANA between now and June 2000IGUANA between now and June 2000(towards the Functional Prototype Milestone)(towards the Functional Prototype Milestone)

IGUANA between now and June 2000IGUANA between now and June 2000(towards the Functional Prototype Milestone)(towards the Functional Prototype Milestone)

CMSCAN Development (Detector &Event Visualisation Toolkit) Conversion of GUI from Motif to Qt (and feedback to HEPVis) Correlation/picking of information (graphics / GUI underlying data) 2D views and non-cartesian transformations (-, fish-eye,...)

CMSCAN Deployment Full range of reconstructed objects (ORCA3 and ORCA 4) OO detector display tools (GEANT4 / OSCAR and ORCA) Test-beam visualisation systems

Physics Analysis Tools Continue prototypes in close collaboration with LHC++, HEPVis, etc. IGUANA focus is more on graphics and GUI’s (rather than scripting,

statistical analysis tools, histogram/tag storage and retrieval, etc.) Definition of generic IGUANA deliverables (in context of LHC++) Development of GUI’s for test-beam monitoring, control, analysis,...





Manpower Shortfall andManpower Shortfall and Schedule Slippage Schedule Slippage

Manpower Shortfall andManpower Shortfall and Schedule Slippage Schedule Slippage

V = Visualisation SW EngineerHired April 99

Later than hoped and must also cover shortfall on

analysis tools prototyping

U = UI / GUISW Engineer(not yet hired)

(late)

Not finished

Delayed

CMSCAN: CMSCAN: ~ 4 month slippage of CMSCAN schedule so far in 1999 ~ 4 month slippage of CMSCAN schedule so far in 1999Analysis Tools: prototypes defined as high priority (on time for now)Analysis Tools: prototypes defined as high priority (on time for now)

Delayed

(late)





SummarySummarySummarySummary

Technology Evaluations and Choices (completed) Modular software components

Software is robust, maintainable and can evolve Leverage efforts elsewhere (HEP, free software community, commerce)

Baseline choice of a specific set of tools has been made

Prototyping (ongoing) Realistic tests of each choice of technology and mixing of technologies Interactive 3D detector / event visualisation (with ORCA) Browsing and display of histograms and tags in Objectivity DB

Development / deployment of CMS applications (started) ORCA underway, start with OSCAR and test-beams by end 1999

Excellent progress has been made so far but... ...timely continuation is contingent on sufficient SW engineering

Functional prototype milestone: June 2000Functional prototype milestone: June 2000

user analysis environment

Documents