selected subjects on controls system - quality assurance p.charrue on behalf of the ab controls...

Selected Subjects on Controls System - Quality Assurance

P.CharrueOn behalf of the AB Controls Group

LHC Machine Advisory Committee

16 June 2006

16 June 2006 LHC Machine Advisory Commitee 2

Preamble

• AB/CO was asked to talk about Quality Assurance - this is a wide subject

• Today we talk about a selection of subject which are representatives in the QA domains and which deserve attention from the management


Outline

• Overview of the Controls Infrastructure

• Network Security (CNIC project)

• LHC Application production

• Post-Mortem project

• Conclusions

• Drawing here


Outline





• Conclusions


The CNIC Working Group

• The Computing and Network Infrastructure for Controls working group was created by the CERN Executive Board From the recommendations made by the Technical Network Management

Working Group (Jul 2004)

• Delegated by the CERN Controls Board (Sep 2004) “…with a mandate to propose and enforce that the computing and network

support provided for controls applications is appropriate” to cope with security issues.

Mandate covers only control systems, not office computing

• Members from all CERN controls domains and activities Service providers (Network, NICE, Linux, Computer Security) Service users (AB, AT, LHC Experiments, SC, TS)


Networking at CERN

• General Purpose Network (GN) For office, mail, www, development, … No formal connection restrictions by

CNIC

• Technical Network (TN) and Experiment Network (EN) For operational equipment Formal connection and access

restrictions Limited services available

(e.g. no mail server, no external web browsing)

Authorization based on MAC addresses Network monitored by IT/CS

Office development PC

Trusted Application Gateways

Home or remote PC

CERN FirewallConnection to Internet

INTERNET

CERN Public Gateways(LXPLUS, CERNTS)


Possible Threats

• Malicious access A hacker accessing our devices from outside A deliberate attack ‘Sniffing’ the data that transits on the TN

• Erroneous access Un-intentional errors Errors committed by CERN personnel in ignorance

• Control/Grant access from outside the CCC (Cern Control Center)

• ‘Anonymous’ traceability• Generic accounts with weak password


Malicious access

• Not much protection possible from CO side against intentional and motivated security attack from outside or within CERN

• However the TN is relatively difficult to get into from outside without a CERN account

• IT security covers protection against these type of threads. CNIC is currently studying intrusion detection on TN


What can be done

• Security enhancement and traceability are possible at four different levels : Communication Layer Accounts CNIC Applications


Communication

• Implement a ‘role-based’ access to the equipment in the communication infrastructure

• Depending on WHICH action is made, on WHO is making the call, and from WHERE the call is issued, the access will be granted or denied

• This will allow for filtering, for control and for traceability of the access to the equipment


Accounts

• Forbid ‘anonymous’ generic accounts• Enforce usage of accelerator-oriented accounts• Enforce the password change regularly• Limit operational accounts to CCC

• All these measures cost nothing• They may be seen as constraints to the

operators working habits


GeneralPurposeNetwork

TechnicalNetwork

TrustedHosts List

ExposedHosts List

Main outcomes of CNIC

• 9 January 2006 : closure of the GPN <-> TN connection No communication allowed to cross the bridge except

• from TRUSTED hosts on the GPN• to EXPOSED hosts on the TN

This reduced the TRUSTED hosts from 10’000+ to 2’000

• NICEFC and LINUXFC deployed operationally on more than 200 hosts

• Restrict access to the Network Description Database (NETOPS) via identification

• More than 40 Application Gateways deployed

• Connection to the TN requires authorization

• MAC address authentication

Operator in the CCC

Specialist access from home

Access from the office inside CERN

Office development PC

Trusted Application Gateways

Home or remote PC

CERN FirewallConnection to Internet

INTERNET

CERN Public Gateways(LXPLUS, CERNTS)

3 typical Use Cases


Pending Studies Areas

• Critical Settings encryption Discussions still on-going

• Authentication means (e.g. card readers in the consoles, bank-like authentication, …)

• Reduction of the Trusted list


Outline





• Conclusions


Frameworks for LHC Applications

• Three approaches in place to build applications

Beam based control applications• Majority of applications• Java infrastructure

Industrial control PLC/SCADA based applications• UNICOS frame based on PVSS

Post Mortem data analysis• Based on LabVIEW

LHC Java Applications and Core

LSA Controls System

Core

FESAEquipment

s

Controls Middleware

Monitoring &

Concentration

LSATrim

BeamSteering

SettingsGenerati

on

BDIApplic

FixedDisplay

s

Controls Settings

LSA API

FESAEquipment

s

FESAEquipment

s

Standard Equipment Access (JAPC)

Core applications

Equipment and instrumentation applications

Standard Equipment Access

High-Level Services

LSA Core


LHC Java Applications - Organization

• The work is done in a close collaboration with the OP group - we work in a team

• One single project in place (LSA) providing the common architecture

• Aim to use for LEIR, SPS, their transfer lines TT40, TI8, TI2, LHC HWC and operation

• Test/validate using every possible controls or operational milestone and several dry runs


Issues - Remote Access and Security

• Experts and on-call teams require access to LHC controls from outside the CCC

• Who has the right to modify LHC parameters? Control of certain devices (Schottky) from other institutes is

already requested (US-LARP collaboration)

• We need remote access and role based access policy and manpower to implement it

• Identification of the originating account and host has to be registered and propagated through all the chain (who and where from)

• Business logic between the GUI and the equipment has to react differently according to the origin of the request


Issues -Time allocated for Testing

• TT40/TI8, HWC, LEIR, CNGS and SPS ring will be used now to validate the LSA core and applications extensively

• Due to the most probable cancellation of the LHC Sector Tests end of 2006, AB/CO will : organize scalability tests for the complete controls infrastructure need well coordinated dry runs

• We request time allocated during LHC commissioning for the final tests of the deployed software


Issues - Resources

• Major core activities are staffed by temporary or departing staff• The same application developers are working for HWC, LEIR,

CNGS, PS and SPS startup • LHC applications list documented but not fully staffed clearly

showing lack of resources• See http://cern.ch/ab-project-lsa/planning/commissioning.htm

• Today 4 FTE from AB/CO/AP, 3 from OP and 1 associates are working on the LHC software production

• We need experienced Java software developers• Since Apr’05 we actively seek for 6 more associates

(3 for HWC and 3 for LSA) : Hired 1 for HWC in April’06, 1 for LSA in July’06 and 1 for LSA in

September We still miss 2 for HWC and 1 for LSA


Outline





• Conclusions


Post Mortem project mandate

After a failure during the operation of the LHC, leading to a beam abort or a power abort, a coherent set of so called “Post Mortem” information will be collected from the various sub-systems to analyze the causes of failure.

To be able to understand the failure before resuming LHC operation, the collected information needs to be analysed within a few minutes and this requires a highly automated data collection and analysis process.

The Post Mortem system is aimed at providing the operators and system experts with data visualisation tools which can combine raw data and automatically analysed data.


PMA: Data flow

QPS PICPC

LHC

Raw data files

Systems

Result data

Logging

Alarms

Othersystems…

PM viewer

PM analyser

Data basesPM server

PM used for LHC Hardware CommissioningExample: Automatic analysis of the QPS tests for

quality assurance.

1. The quench detection signal gets driven over the 100 mV threshold.

2. View of QPS signals to see that the system triggered and the quench heaters fired.

3. Automatic analysis of the quench heater discharge (log scale) showing the results.

4. Automatic analysis of the event with “passed/not passed” indication.

1

2

3

4


PM: Milestones

1. June ‘06: Data Viewer for QPS, PIC and PC data

2. Sept. ‘06: Extended PM data storage model for new clients

3. Sept. ’06: Dry run, correlation of QPS, PIC and PC data

4. Oct. ‘06: PM system scaling test, including BI, BT and RF

5. Nov. ‘06: HW commissioning analysis, as defined in

LHC-D-HCP-00026. During‘07: Analysis for Beam

Commissioning


Issues

• A successful PM system was developed for SM18 magnet quench analysis served as the base of the LHC PM system

• Recently a new Project Leader has been assigned due to succession planning and the scope has increased through data collection, storage, browsing and analysis

• Many technological choices and user interfaces still to be defined and solved

• We are rather late with the work due to the late arrival of user specifications.


Outline





• Conclusions


Conclusions

• Network and Security : Activities are well defined Reduction of the TRUSTED list is not trivial encryption, authentication and role based access need global

coordination

• LHC applications Framework well defined There are issues on resources and on time for testing Hiring JAVA experts is very difficult

• PostMortem First operational version used in HWC for QPS, PIC and PC Project changed leadership and mandate has been extended Work is late

selected subjects on controls system - quality assurance p.charrue on behalf of the ab controls...

Documents

lhc machine advisory

tn slide

lhc experiments

management slide

cern controls domains

cern controls board

network infrastructure

itcs slide