gridcc: real-time instrumentations grids

First EELA Conference, Santiago(Cile), 4th September,2006 ([email protected] - [email protected])

GridCC: Real-time Instrumentations Grids

A real-time interactive GRID to integrate instruments, computational and information

resources widely spread on a fast WAN

Andrea PetrucciIstituto Nazionale di Fisica Nucleare

Laboratori Nazionali di Legnaro, Legnaro Italy

Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 ([email protected] - [email protected]) - 2

Overview The GridCC Project: Introduction

Bringing Instrument into the Grid: the Instrument Element

The GridCC Test-bed: Pilot application

Instrument Instrumentation Fast Instrument Communication Channel Standard Grid Interaction


General on the GridCC ProjectParticipant name Country

Istituto Nazionale di Fisica Nucleare Italy

Institute Of Accelerating Systems and Applications

Greece

Brunel University UK

Consorzio Interuniversitario per Telecomunicazioni

Italy

Sincrotrone Trieste S.C.P.A Italy

IBM (Haifa Research Lab) Israel

Imperial College of Science, Technology & Medicine

UK

Istituto di Metodologie per l’Analisi ambientale – Consiglio Nazionale

delle Ricerche

Italy

Universita degli Studi di Udine Italy

Greek Research and Technology Network S.A.

Greece

It is a 3 years project. Started the 1st September 2004

Funded by EU in the Frame Program 6

10 Partners from 3 EU Countries + (Israel)

About 40 people engagged

www.gridcc.org


The Grid Technologies extend the limit of a single computer (center)

Grid Technologies

Grid Technologies

User Interface

Grid Gateway

ComputingComputingElementElement

StorageStorageElementElement

ComputingComputingElementElement

Current Grid technologies offer unlimited computational power and storage capacity for scientific research and business activities

However concrete use cases demand more close interaction between various types of instruments accessible from the Grid elements


Extending the Grid Concepts

Instruments Grid Computational Grid

GridCC Project

+

Data for Model Calculations

Predictions

The goal is to build a widely distributed system that is able to remotely controls and monitors complex instruments


StorageElementsStorage

Elements

ComputingElement

ComputingElement

InstrumentElement

Instrument Element: global scenario

ComputingElement

StorageElement

InstrumentElement

InstrumentElement

Existing Grid Infrastructures

Web ServiceInterface

Virtual Control Room


Exec.

ServiceWfMS

WMS

AgrS

User direct Action

Indirect Action



(VCR)

All end user access is via

the VCR

Instrument elements

(IE)

The IE is a virtualization of the real physical instrument

Instrument elements

(IE)Instrument elements

(IE)

Of course there may be many IEs

Compute and Storage Elements (with advanced reservation)

StorageElement

(SE)

Compute element

(CE)

Of course Many CEs and SEs

StorageElement

(SE)

Compute element

(CE)StorageElement

(SE)

Compute element

(CE)

CollaborativeServices

(CS)


(VCR)

Users generally not working alone

Direct access to IE

SE (and CE) possible but often not desirable

Information and Monitoring

Services(IMS)

“Fast” all pervasive messaging system

Information System

(IS)Slowly updating information

Security Services

Security is essential to the success of the project

Global ProblemSolver

Watching (via the IMS) for problems anywhere in the system and acting to resolve them.

Execution Services

More complex workflows, including advanced reservation and QoS guarantees , allowed

The GridCC Architecture


IE Requirements

Web Services

Instrument Element

Any Protocol or physical connection

Sensor Network

Instrument

Instrument

GridGrid

ComputingComputing ElementElement

StorageStorage ElementElement

ComputingComputing ElementElement InstrumentInstrument

ElementElement

W

EF

A

B

C

D

1: Provide a uniform access to the physical device

2: Allow a standard grid access to the instruments

3: Allow the cooperation between different instruments that belong to different VOs


Instrument Element: a Black Box

IEVIG

SInstrument

Instrumentation

Fast communication channel

The term Instrument Element describes a set of services that provide the neededinterface and implementation that enables the remote control and monitoring of physical instruments.

Grid Interaction

Da

ta M

ove

r

Instruments

Quick Answers to the previous slide: 1) The VIGS provide the a uniform

instrument instrumentation way 2) The fast communication channel

disseminate the acquired information between instruments

3) The Data Mover provide a standard Grid Interface in order to be accessed by others Grids components like the SE and the CE


Instrument Element ArchitectureVirtual Instrument Grid Service (VIGS)

ResourceService

Inf & MonService

ProblemSolver

InstrumentManager

Instrument Element

Data Mover

IMSProxy

ControlManager

DataCollector

Real Instruments

Data Flow

Control Flow

State FlowError FlowMonitor Flow

• The IE entry point is constitued by a Web Service interface called Virtual Instrument Grid Service(VIGS)• The VIGS allows users to access and control the specific physical Instrument from any part in the world using a VCR.

Acc

ess

Con

trol

Man

ager

execute()

getState()

create()

destroy()

InputManager

EventProcessor

FSMEngine

ResourceProxy

Control Manager


Instrument Element Implementations

ResourceService

Inf & MonService

ProblemSolver

InstrumentManager

Instrument Element

Data Mover

Acc

ess

Con

trol

Man

ager

The IE components are typically implemented into a fully equipped Machines (e.g. dual core cpus, large memory, large disks, etc). This is true for RS, IMS and PS. For IM (and DM) there are 2 possibilities, according to the application type:• IM implemented in a fully equipped machine• IM embedded into the instrument that should be controlled

IM

RSIMS

IM

IM

IM

Embedded Web Service


Instrument Manager

IMSProxy

DataCollector

Instrument Manager

InputManager

EventProcessor

FSMEngine

ResourceProxy

Control Manager

Instruments

Data Flow

State Flow

Error Flow

Monitor Flow

Control Flow

Customizable Plug-in modules to interface to the instruments

IM is composed by 3 main components: Control Manager:

Input Manager : It handles all the input events of the IM. These includes commands from GUIs or other IMs,errors/state/log/monitor messages. Event Processor: It handles all the incoming message and decide where to send them. It has processing capabilityFSM: A finite state machine is implemented.Resource Proxy: It handles all the outgoing connections with the resources.

Data Collector: It get data from the controlled instruments and make them available to the data mover. A local storage of the data is even foreseen. IMS Proxy: It receives error/state/log/monitor information from the controlled resources and forward them to IMS.


Resource Service Architecture

The Resource Service (RS) handles all the resources of an IE and manages their partition (if any).

A resource can be any hardware or software component involved in the IE (instruments, Instrument Managers, IMS components)

RS stores the configuration data of the resources and download them to resource target when necessary

Resources can be discovered, allocated and queried. It is the responsibility of the RS to check resource availability and contention with other active

partitions when a resource is allocated for use. A periodic scan of the registered resources keeps the configuration database up to date. RS is interfaced to the WMS

DiscoveryManager

SubscribeManager

Partition&LockManager

ConfigurationManager

Available Resources

PartitionDefinitions

ConfigurationDefinitions

RS

Dat

a B

ases

Partition/Configurationretrieve methods

Partition and Locksetting methods

Configurationsetting methods

Discoverymethods


Information and Monitor System (IMS)

PUBLISHERS(Instruments nodes)SUBSCRIBERS

Errors Log infoMonitorState

The Information and Monitor Service (IMS) collects messages and monitor data coming from GRID resources and supporting services and stores them in a database. There are several types of messages collected from the sub-systems. The messages are catalogued according to their type, severity level and timestamp. Data can be provided in numeric formats, histograms, tables and other forms.

The IMS collects and organizes the incoming information in a database and publishes it to subscribers. These subscribers can register for specific messages categorized by a number of selection criteria, such as timestamp, information source and severity level.

Ins

trum

en

tM

an

ag

er

Inst

rum

ents

Ins

trum

en

tM

an

ag

er

Inst

rum

ents

Ins

trum

en

tM

an

ag

er

Inst

rum

ents


Problem Solver

IMSProxy

ControlManager

Instrument Manager

Pub/Sub

IMSProxy

ControlManager

Instrument ManagerIMS

ProxyControl

Manager

Instrument Manager

IMSProxy

ControlManager

Instrument Manager

DBData Mining Tools

Algorithms evaluations :Rule Induction, Tree, Functions, Lazy, Clusters and Associative

State FlowError FlowMonitor Flow

On Line Analisys

Problem Solver

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

iris

glas

s

brea

st c

ance

r

bupa

votin

g-re

cord

s

hous

ing

bala

nce-

scal

e

Bre

ast

Can

cer

Wis

cons

in

Pim

a-In

dian

s-D

iabe

tes

tic-t

ac-t

oe

Seg

men

t

Seg

men

tatio

n

Sic

k-eu

thyr

oid

Pag

e-B

lock

s

mus

hroo

m

Shu

ttle

(2)

Lett

erR

ecog

nitio

n

krko

pt

Shu

ttle

(1)

conn

ect-

4

dataset

accu

racy

Average Rule Accuracy

Average Tree Accuracy

Average Function Accuracy

Average Instance Accuracy

Average Cluster Accuracy

Step 1 The control manager can perform an autonomous recovery action where the cost for the determination it is not so heavy .

Step 2 Persistent information can be analyzed in order to extract knowledge

Step 3 On-line information can be analyzed in order to detect possible malfunctions


Standard Grid Interaction: Data Mover

The task of this element is to get data from the “data collector” of the IM Data can be accessed via:

Web service interface for generic data dump (e.g. slow storage, spy stream, etc.)

grid storage element (SE) and available CEs can access to the data via an SRM Interface

Http server and TCP communication for high performance had-hoc data transfer

The Data Mover exposes its methods to the IE web service and can be instrumented itself as an instrument.

Instrument Resources

DataMover

DataCollector

IM

IE Web Service Interface: get_data()

SRM interface

Http Server andTCP/IP raw socket

DataCollector

IM

DataCollector

IM


Main GRIDCC Pilot Applications

CMS detector - CERN

Power Grid - Brunel

Intrusion Detection System – NTUA/IASA

Far remote control of accelerator - Elettra


Main GridCC Pilot Applications:

The Run Control prototype is controlling and monitoring a Data Acquisition (DAQ) system of the High Energey Physical Experiment CMS at the CERN Large Handron Collider.


The CMS Data Acquisition O(104 ) distributed

Objects to control configure monitor

On-line diagnostics and problem solving capability

Highly interactive system (human reaction time - fraction of second)

World Wide distributed monitor and control

2 107 electronics channels 40 MHz

100 Hz


CMS Prototype: IEs at work

Det 1Det 1Det 1

DAQ

TTS

FedBuilder RuBuilder

FilterFarm

Trigger

TOP

GTPe

DAQ

Detector

1

8

- GridCC middleware used for CMS MTCC (Magnet Test and Cosmic Challenge)

- 11 Instrument Elements with a hierarchical topology

- Instruments are in these case Linux hosts where the cms on-line software is running

- More than 100 controlled hosts

CMS Instrument Elements

DAQ IE Instrument Managers


Power Grid

Instrument Manager

Instrument Element

..

.



Gas

Solar

Power Grid V.O

The Power Grid prototype is controlling and monitoring power generators (Solar, gas ..)


Remote Operation of an Accelerator

Elettra Synchrotron

Far remote operation of an accelerator facility in volves the planning of accelerator operations, the maintenance of the accelerator and its trouble shooting, the repair of delicate equipment, understanding and pushing performance limitations, performing studies, performing commissioning and set ups and rou tine operations.


The other GridCC pilot applications Meteorology (Ensemble Limited Area Forecasting)

Device Farm for the Support of Cooperative Distributed Measurements in Telecommunications and Networking Laboratories

Geo-hazards: Remote Operation of Geophysical Monitoring Network (see first slides)

Medical Devices need a close loop between the data acquisition and the output result


Conclusion

The GridCC project is integrating instrument into traditional computational/storage Grids.

IEs need an high interaction and interactivity between itself and the users.

The GridCC IE implementation is currently installed in heterogeneous applications


Question.....

Thank you for your time

Acknowledgement: The GridCC project is supported under EU FP6 contract 511382.

More information: www.gridcc.org

gridcc: real-time instrumentations grids

Documents