The ESO Very Large Telescope Observatory (VLT) at Cerro Paranal in Chile had its first light in 1998. Most of the telescopes’ electronics components are now around 20 years old. Electronic technology is advancing fast and spare parts are often no longer available on the market while new technology is evolving. This puts at risk the availability of telescope sub-systems and other operational equipment. This poster shows the analysis, approach, timeline, complications and progress in upgrades at the Paranal Observatory.

Obsolescence of Electronics at the VLTG.Hüdepohl / J.P.Haddad / C.Lucuix

European Southern Observatory"

Introduction"

Lessons Learned and Recommendations"

Conclusion"

Fig. 1 The Very Large Telescope (VLT) on Cerro Paranal with the four Unit Telescopes (UT1…4) from left to right.

To start planning the obsolescence upgrade an analysis needs to be performed to identify the most critical systems and the priority for upgrades. All telescope systems were analyzed with respect to the following criteria: For all systems: •  How many critical spare parts do we have in stock? •  How many failures has the system experienced in the last

1-2 years? As result we can extrapolate to the future and get the estimated survival time before spare parts are exhausted. For commercial off the shelf (COTS) systems: •  Is the component still available on the regular market? •  If not: is the system still available on the secondary market

(used, Ebay, repaired, etc.)?

For custom built electronics: •  Is the original supplier still able to provide spares? •  Do we have the full knowledge, documentation, specs of

the system? •  Are we able to replicate manufacture, re-design, etc. the

system?

Result of analysis: List with a ranking of priority for upgrade. If estimated survival time is <= 1 year, priority is highest (critical). See Table 1.

Custom built electronics require either internal man-power for reverse engineering and development or highly specialized companies. For the VLT this is mostly done in house at ESO headquarters (Germany) or at the observatory. For industrial equipment: •  We fully contracted out the project to specialized companies.

Disadvantage: higher cost and call for tender process. Advantage: requires less man-power from staff, design is available to implement in remaining telescopes.

•  ESO does the design, purchases the hardware and implements first telescope. For the remaining telescopes: Implementation is outsourced to the on-site contracter, hardware provided by ESO.

For custom built systems: •  Design, development and implementation done by observatory

staff at Paranal and/or headquarters.

To minimize downtime the upgrade can be broken down in smaller units, which reduces technical risks and downtime.

Table 1: Example of analysis result of systems obsolescence

Large telescopes have a lifetime much beyond 20 years. As a consequence technology upgrades are unavoidable. This is a major effort that must not be underestimated in terms of complexity, man-power, downtime and budget. Early analysis and planning is important to avoid surprises when it is too late. Consider the necessary budget. Keep in mind the above mentioned recommendations can help to avoid headaches and surprises and will help to keep large telescopes operational over decades.

•  When upgrading systems to new technology, expect teething problems and more failures during initial period, until all problems are understood and sorted out.

•  New design is first implemented in one telescope and tested for several month until robustness is proven before using it on other telescopes.

•  Time needed for design and implementation were often underestimated. Main reasons: lack of as built documentation, complex reverse engineering, unexpected technical problems. In addition: Time consuming call for tender process for external contracts and occasional hardware delivery delays.

Recommendations: •  Plan upgrades as early as possible. Even during design phase

of the telescope it could already be worth to consider this aspect. What initially may seem to save cost could later become very expensive.

•  Use industrial off-the-shelf technology wherever possible •  Avoid “home-made” electronics and prototypes where firmware

and internal design information is not fully available to the end user.

•  Avoid special products from very small companies •  For custom built equipment procure sufficient spares as long as

they are available. •  Give preference to modular design that uses standard

interfaces and standard communication protocols. •  Check obsolescence policy of equipment manufacturers •  In original design plan sufficient spare space in electrical

cabinets. Later upgrades do often (surprising but true) require more space.

Next step: Decision what technology to choose for the upgrade. This depends mostly on the original technology: •  PLC (Programmable Logic Control): upgrade to next or

latest generation PLC •  Motor drives: Use latest generation drives (upgrade from

analog to digital technology) •  Custom electronics, most difficult to choose:

1.  Find commercial electronics (preferred solution!) 2.  Redesign with today’s technology, leading again to a

custom built system. 3.  Implement the application in software where possible

•  VME: new generation boards or full redesign of hardware and software compatible boards by virtualizing, using FPGAs.

•  Relay based emergency stop and interlock chains: Replace with Safety PLCs

Once the priorities are known and the upgrade technology is selected, a cost estimate is done, at least for the high priority systems. For the full planning we need also to estimate the design and implementation time line to plan and request the Technical Time slots. Table 1 shows the technical shutdown time period that is required for some of the upgrades. Technical nights have to be requested between 6-12 months in advance in order to be coordinated with the science schedule. An approved formal change request or project request is required, before starting an upgrade project. Having four identical telescopes allows to re-use decommissioned electronics after first upgrade as spares for the remaining telescopes. This extends the lifetime before another upgrade is needed.

Having four identical telescopes has the advantage, that once the technical solution has been developed it can be tested in one telescope. Only when the new system is robust, it is implemented in the remaining telescopes. Design and development effort is therefore reduced. Old parts that were removed can serve as spares for the remaining telescopes, extending the lifetime before upgrade is necessary (gaining time). Upgrades are not aiming at improvement, but rather focus on long term availability of the telescopes. However in several cases we could improve maintainability for example by adding local touch screens.

How to tackle the upgrade mainly depends on the chosen technology, available budget and man-power. PLCs and other industrial equipment can be upgraded by specialized companies which are widely available.

Description Technology for upgr. Prio Cost/ UT downtime Comment UT1 UT2 UT3 UT4

ongoing=blue; complete=green

VLT Enclosure

Enclosure PLC fixed part PLC S7 med 47.5 kE 1 day cost includes spares and engineering (35 kE) 2009 2016 2017 2019

Enclosure Thermal Sauter PLC PLC S7 med 11 kE 1 day 2010 2016 2017 2019

Enclosure Rotation Drives Sinamics high 130 kE 2x2 days cost includes engineering (75 kE) 2010 2016 2018 2019

Enclos. PLC rotating part, OD drivesl PLC S7 high 85 kE 5 days cost includes engineering 2014 2017 2018 2019

VLT Main Structure

HBS PLC PLC S7 med 20 kE ? 2017 2017 2018 2019

HBS pump drives SEW drives high 10 kE 0 days UT1 and UT2 complete, priority reduced to med. 2010 2011 2016 2017

Altitude and Azimuth drives Sinamics high 40 kE 0 days UT4 Az Siemens Master drives, Alt in 2015 tbd tbd 2017 2016

Altitude tacho and hall sensors clone/ find alternative high clone hall board, alternative tacho 2016

Altitude cable wrap and shutter control PLC S7 low 10 kE

Azimuth cable wrap drives and control med 15kE 2 refurbished drives were bought 2016 2018 2018

E-Stop & Interlock Chain

Emergency Stop and Safety Chain Siemens Safety PLC high 75 kE 5 days final cost includes contractor: 45kE mat. + 30kE h 2015 2013 2016 2015

VLT M1Cell

VIUC and SPCMs (custom built) VME boards & SW high 20 kE 3 days 2016 2017 2016 2015

VPC (custom electronics) redesign high 5 kE 2016 2016 2016 2016

M1/M3 drives, VAU drives tbd med

Earth Quake Detector EQD Beckhoff PLC med

VLT Adapter Rotator (3 per telescope)

Direct drives for Rotator and Adapter tbd med 15kE 2-phase drives, difficult to find on market

Upgrade Technology"

Strategy"

Design and Implementation"

Planning"

Status Analysis"