the op viewpoint on the accelerator complex controls renovation r. steerenberg on behalf of the op...

The OP Viewpoint on the Accelerator Complex Controls Renovation

R. Steerenberg on behalf of the OP group

Accelerator Complex Controls Renovation Workshop 3 December 2008

Thanks to input of: M. Benedikt, C. Carli, P. Collier, K. Cornelis, T. Eriksson, K. Hanke, D. Kuchler, M. Lamont, S. Pasinelli, R. Scrivens, F. Tecker, J. Wenninger.

AgendaScopeAim of Renovation From OP ViewpointPresent Situation (not exhaustive)OP’s Preferred Renovation StrategyRenovation OrganisationPlanningClose collaboration and specific requirementsAccelerator Operation During RenovationSide EffectsOP ContributionConclusion

3 December 2008R. Steerenberg 2

Acce

lera

tor C

ompl

ex C

ontro

ls Re

nova

tion

Wor

ksho

p

ScopeThe machines concerned are:

LINAC2PS BoosterISOLDE and REXPSAD (Recent publication: “AD consolidation for operation beyond 2010”)CTF3LINAC3LEIRSPS

The work for LINAC4 project has to be added to this, limiting the available resources to a certain extend, but is not part of the renovationSome of these machine are under the responsibility of ABP (LINAC2, LINAC3 and LEIR)


Acce

lera

tor C

ompl

ex C

ontro

ls Re

nova

tion

Wor

ksho

p

Aim of Renovation From OP Viewpoint

Modernize the present systems and increase their reliability and performance with enhanced functionalitiesHarmonisation of controls across all accelerators

Better use of resourcesMore widely spread knowledgeBetter or more available support

Very positive to add non-LHC injectors to the renovation for the above mentioned reasons

If they are left, old systems will have to be dragged along


Acce

lera

tor C

ompl

ex C

ontro

ls Re

nova

tion

Wor

ksho

p

Present situation (1)SPS has successfully run with LSA since the “big bang” at the start of 2006

Very tough year 2006Still not completely finished (some legacy SW remaining) Still fine tuning LSA

Stability is now needed to invest on the LHC as SPS and LHC teams and efforts are combined.Some worries

ROCS MUGEF caused some problems and follow up is neededPartly renovated systems need to be completedProfile measurements: the replacement SW not yet fully available. No emittance calculation anymore etc. (loss of functionality)

Additional ppm modes desirable (e.g. part. type)3 December 2008R. Steerenberg 5

Acce

lera

tor C

ompl

ex C

ontro

ls Re

nova

tion

Wor

ksho

p

Present situation (2)The PS complex is at the start of implementing INCA and eradicating X-motif

The ‘old’ control system experts are becoming scarceA 3-tier system is now necessary due to fundamental changes in the low level software (scalability issues)Main focus is now on INCA, but the renovation goes beyond

Renovation examples:BWS in PSB and PS:

Initially the design (FESA + Application) was made for LHC and later adapted to SPS to be adapted again to PS ComplexThe implementation approach during the last part of the 2008 run was good and OP requirements were and are taken into account

Vacuum control:Did and does still not fulfil OP requirementsNo prior consulting on the needs and the vacuum control is becoming an exception in the entire complex control system3 December 2008R. Steerenberg 6

Acce

lera

tor C

ompl

ex C

ontro

ls Re

nova

tion

Wor

ksho

p

Present situation (3)CTF are machines that evolve quickly and need therefore a flexible controls environment

Fast implementationThere where possible standard solutions, otherwise specificMany external collaborators developing SW, requiring good standards and support from CO (framework)

AD consolidation is under considerationPoint on the agenda of the Research Board of 5 DecemberComplete renovation of some hardware and softwareOverhaul of the timing system, cycle generation, etc...Details in :

“AD consolidation for operation beyond 2010”CERN-AB-2008-068 OP


Acce

lera

tor C

ompl

ex C

ontro

ls Re

nova

tion

Wor

ksho

p

Present situation (4)LINAC3

Renovation of ramping cavities and de-buncher control started in 2004, but never finishedProper control software is missingOnly specialist application is availableSuffered also from vacuum controls renovation

LEIRHybrid of LSA and ‘old’ PS controls and has therefore not a coherent approach for archivingBe aware that LEIR is now in a long shut down during which many developments have been ongoing (LSA)Many surprises around the corner?


Acce

lera

tor C

ompl

ex C

ontro

ls Re

nova

tion

Wor

ksho

p

OP’s Preferred Renovation StrategyThe renovation should preferably be done per system and not per machine:

Easier control over planning and resourcesLess diversity to handleBetter coherency of similar systems across all acceleratorsBetter chance to have general applications that suits the requirements of all accelerators for the system concernedLess needs to adapt systems with patches to fulfil requirements of other accelerators afterwardsIn general smaller impact on operational performance

The renovation should be organized in vertical slices containing:

Hardware equipment group concerned (+ CO)Drivers, FESA, etc equipment group concerned and CO (+ OP)INCA and Applications CO + OP


Acce

lera

tor C

ompl

ex C

ontro

ls Re

nova

tion

Wor

ksho

p

Renovation organisationThe renovation of each system (vertical slice) should be dealt with as a project:

Agreement on planning and requirements by all partiesEnsure allocated resources by and from all parties

Put mini teams together from the different groups

Make them responsible for a vertical slice to be renovatedThey will evaluate the impact of the renovation on the operational environment and establish a planning including the required resources for the system to be renovatedThey test, commission and validate the renovated system

These mini teams should report to CO3 or a steering committee for the renovation project that ensures the resources (P+M) and officially validates the outcome of the mini teams3 December 2008R. Steerenberg 10

Acce

lera

tor C

ompl

ex C

ontro

ls Re

nova

tion

Wor

ksho

p

PlanningOP has no strict requirements on the chronology of the renovation, but the planning should be based on the following criteria:

Most critical systems in terms of risk should be given priority. (see Risk Analysis of S. Baird) Obsolete hardware with no spares policyAll levels in the vertical slice should be ready or made ready to do the renovation (incl. Application)All resources (P+M) should be committed and the planning should be realistic to guarantee successful validation in timeCO3 or project steering committee coordinates and plans the whole renovation to avoid duplication or multiple use of the same resources


Acce

lera

tor C

ompl

ex C

ontro

ls Re

nova

tion

Wor

ksho

p

Close collaborationIn some cases the hardware will remain the same, but the FESA classes and the high level software will be renovatedClose collaboration with CO and OP is required to avoid losing functionalitiesExample:

The GM class POW-V will be renovated, but contains many specific requirement, which is presently dealt with using so-called treatment codesRenovation should identify all these specific cases, re-evaluate their need and provide general solutions for long term use by including enough flexibility.


Acce

lera

tor C

ompl

ex C

ontro

ls Re

nova

tion

Wor

ksho

p

Specific requirementsMore standardisation in the FESA class properties will make their use easier within applications:

example:All FESA classes containing sampler like properties should use the same conventions/protocols. As a result a general application can automatically be configured and used to display the data in different forms

Produce guidelines to be followed to obtain a standard.Technical solution, by providing standard plug-in modules

Where possible make a minimum number FESA classes per system for all machines to avoid having large numbers of FESA classes with only small differencesStandardize there where possible the naming of some more general properties to ease semi-automatic population of tables etc.


Acce

lera

tor C

ompl

ex C

ontro

ls Re

nova

tion

Wor

ksho

p

Accelerator Operation and Renovation

OP requires a minimum disturbance approach for the renovation to maximize machine performance

In case of renovation of critical systems the old system must remain as back up solution or continue to be available in parallel. (until the new system is fully validated)There where possible perform tests or pre-commissioning before the shutdown during which the new system will be deployed. This will leave time to make corrections and solve problems that showed up during these testsAllow enough time during the HW test period, cold check out period and the start up with beam period to test and validate the renovated system on all the machines concerned (have the controls system properly available)The accelerators should not be used as test bed, degrading their performance. However, request MD time


Acce

lera

tor C

ompl

ex C

ontro

ls Re

nova

tion

Wor

ksho

p

Side effectsThe renovation will also finalize the transfer of responsibility for the PS complex front-ends (hardware + software) to the equipment groups, which is already the case for SPS and LHCNevertheless the general components in the front-ends remain under the CO responsibilityOP would like to have enhanced, but simple diagnostic tools to determine in case of failure if the equipment group or CO needs to be called (e.g. at 2:00 am)What will be the future of the CO piquet service and the specialist service that works on best effort basis ?


Acce

lera

tor C

ompl

ex C

ontro

ls Re

nova

tion

Wor

ksho

p

OP ContributionOP provides a substantial amount of man power to help developing the control system (~ 8+ FTE):

The majority develops specific and general applications A substantial amount is contributing to generic software A few people are actively involved in the implementation of new or renovated systems within the controls environment

OP will continue to provide this manpower, but the resources should be planned correctly

OP people will be most available for programming during the shutdown periods and have a limited availability during the runGood support from CO for standard tools (e.g. Graphs, tables, wheel switches, documentation, etc.)Established, supported and stable programming environment by using a maximum of standard tools and procedures (framework)Establish a clear responsibility strategy for applications....


Acce

lera

tor C

ompl

ex C

ontro

ls Re

nova

tion

Wor

ksho

p

ConclusionRenovation per vertical slice as a project, using mini teamsImpact, proposed planning and resources to be evaluated and approved by the CO3 or project steering committeeOP requirements should be included they will operate the equipments daily. (impact on machine performance)OP would like to have a minimum disturbance approach

Maximize machine performancePlan enough time for testing, commissioning and validation

Standardize where possible the FESA class properties, protocols and naming, without compromising flexibilityOP provides substantial amount of manpower

Need good and stable development environment and support3 December 2008R. Steerenberg 17

Acce

lera

tor C

ompl

ex C

ontro

ls Re

nova

tion

Wor

ksho

p

The renovation can only be successful and follow the minimum disturbance approach when all parties

collaborate closelyOP is ready for doing this...

A little irony, but ..?!?…


Acce

lera

tor C

ompl

ex C

ontro

ls Re

nova

tion

Wor

ksho

p

What we actually really needed was simple

How it was supported

What was operationally

available

How we got convinced

How it was designed

How it was understood

How we specified it

How the project was

documented

How the programmer wrote

it

the op viewpoint on the accelerator complex controls renovation r. steerenberg on behalf of the op...

Documents