D. B
ozzi
ni, E
N-E
L-H
T, A
TOP
Day
s 20
09, 4
-6 M
arch
200
9
1
ELECTRICAL NETWORK:
18 KV NETWORK REVIEW AND STATUS OF THE SPS
CONSOLIDATION PLAN
D. Bozzini on behalf of EN-EL-HT
With the contribution of EN-EL-OP section members
and in particular G. Fernqvist, G. Cumer & C. Jach
ATOP Days - Session 4 - Technical infrastructure
D. B
ozzi
ni, E
N-E
L-H
T, A
TOP
Day
s 20
09, 4
-6 M
arch
200
9
2
Outline
PART IMajor events on HV electrical cable network in 2008Major events on HV cells in 2008 Historical trend of failures on the SPS 18 kV networkCASE 1: The 2008 SPS joints crisisCASE 2: BQ1 CompensatorCASE 3: RESTO 1
PART IISPS Consolidation, status and future programBE-TI8 network layoutSMB: what do we do?Failure location on HV cables network
CONCLUSIONS
D. B
ozzi
ni, E
N-E
L-H
T, A
TOP
Day
s 20
09, 4
-6 M
arch
200
9
3
Major Events on HV Electrical cables Network
SPS 18 kV network– 6 cable failures on the 18 kV network (Including the 4 ones mentioned by P. Sollander)
• 2 on the pulsed TI8 network, stop of the SPS machine (2001)• 3 on the pulsed network, operation of SPS possible in open loop
(1970)• 1 failure on the SMB1, redundant network available (1970)
BEQ1 Compensator– Failure on a 18 kV cable head of the BEQ1 feeder in BE substation,
fortunately not in operation during failure period (2003)
RESTO 1 Meyrin– Elastimold 18 kV elbow connector on a transformer powering the general
services of Resto 1. (2001)– Near miss failure detected tanks to the preventive maintenance and the
reinforced visual inspections of the installation by the operation section.
D. B
ozzi
ni, E
N-E
L-H
T, A
TOP
Day
s 20
09, 4
-6 M
arch
200
9
4
Major Events on CERN HV cells
– ME24 cell explosion ( 1995)
• 10 days stop of the machines feed through the ISOLDE loop
– SE18 near miss ( 1995)
– Preventive action taken• Problem limited to FLUAIR cell type on Meyrin substations and SE18
• Monitoring of temperature and humidity
• Systematic Partial Discharge (PD) corona effects measurements
• Visual inspection inside cells whenever accessible
Comments: In the past 5 years, 2 similar accidents and 3 near miss !
– ME59 switch closing failure during auto-tranfer exercise ( 1968)
• Design life time 25 years, continuous operation at CERN 41 years !
Comments: Remember the 29 July 2006, same type of equipment !
ME9 Jura substation ( 1968) critical installation
D. B
ozzi
ni, E
N-E
L-H
T, A
TOP
Day
s 20
09, 4
-6 M
arch
200
9
5
Historical trend of failures on the 18 kV SPS network
1980 1985 1990 1995 2000 2005 20100
1
2
3
4
5
6
7 6 (2008)
Nature of the failuresA) Aged cables, made with XLPE affected by water-treeing effect
B) Not directly the cable but the accessories (joints, terminals)
Remarks for 20084 out of 6 failures affected recently installed SPS network segments ( 2001)
2 remaining failures on old cables
D. B
ozzi
ni, E
N-E
L-H
T, A
TOP
Day
s 20
09, 4
-6 M
arch
200
9
6
CASE 1: SPS joints crisis (1) - Failures
BE
BB3 BA4 BA51360 m 1270 m
1595 m
BB5
Foui
lle D
Foui
lle C
Gal
lerie
Foui
lle B
Foui
lle A
Reg
ard
BA
4
Ref dwg SPSEM___0003
D. Bozzini TS/EL/HT
Fault #2EDMS 944129Date:25-jui-2008
Fault 2007EDMS 846414Date:27-mai-2007
Fault #1EDMS 924713Date:24-mai-2008
SPSStable loop
SPSPulsed loop
TI8Line
Fault #3EDMS NADate:26-Dec-2008
D. B
ozzi
ni, E
N-E
L-H
T, A
TOP
Day
s 20
09, 4
-6 M
arch
200
9
7
CASE 1: SPS joints crisis (2)- Analysis
BE
BB3 BA4 BA51360 m 1270 m
1595 m
BB5
Foui
lle D
Foui
lle C
Gal
lerie
Foui
lle B
Foui
lle A
Reg
ard
BA
4
Ref dwg SPSEM___0003
D. Bozzini TS/EL/HT
Work done in 2001 by SPIE sub contractors
Work done in 2001 by SPIE technicians
Analysis report #1Cellpack PB-08-27
1 faulty jointDate: 27-jul-2008
Analysis report #2Cellpack PB-07-38
1 faulty jointDate: 25-jul-2008
SPSStable loop
SPSPulsed loop
TI8Line
Analysis report #3Cellpack PB-08-341 faulty joint2 in service jointsDate: 07-sep-2008
Analysis #4CERN1 in service jointDate: sepl-2008
Analysis #5CERN1 in service jointDate: Oct-2008
Analysis #6CERN1 in service jointDate: Nov-2008
Analysis #7CellPack1 faulty jointDate: Feb-2009
D. B
ozzi
ni, E
N-E
L-H
T, A
TOP
Day
s 20
09, 4
-6 M
arch
200
9
8
CASE 1: SPS joints crisis (3) - Analysis
Analysis of the joints
Conclusions of ALL 8 joints examinedASSEMBLY PROCEDURE NOT FOLLOWEDPOOR QUALITY OF WORK
D. B
ozzi
ni, E
N-E
L-H
T, A
TOP
Day
s 20
09, 4
-6 M
arch
200
9
9
CASE 1: SPS joints crisis (4) - Repair
Corrective actions• Repair of all joints that failed
Preventive actions• Replacement of all joints• Improvement of the shielding continuity across the joints
• 2 new joints for each repair/replacement
• Now we have 80 joints instead of 40
BE
BB3 BA4 BA51360 m 1270 m
1595 m
BB5
Foui
lle D
Foui
lle C
Gal
lerie
Foui
lle B
Foui
lle A
Reg
ard
BA
4Ref dwg SPSEM___0003
D. Bozzini TS/EL/HT
SPSStable loop
SPSPulsed loop
TI8Line
Repair #5 SPIE Oct 2008, 13 joints
Repair #4 CERN Sep 2008, 24 joints
Repair #3 CERN Jun 2008, 1 joint
Repair #2 CERN Mai 2008, 1 joint
Repair #1 CERN Mai 2007, 1joint
Repair #6 SPIE Jan 2009, 1 joint
D. B
ozzi
ni, E
N-E
L-H
T, A
TOP
Day
s 20
09, 4
-6 M
arch
200
9
10
CASE 1: SPS shield currents
Preventive action• 68/80 joints have a reinforced
shield continuity• Modified assembly procedure
approved by joint manufacturer
Standard
Reinforced shield continuity
Signs of heating confirmed by the conclusions of the joints analysis
• This triggered a campaign to measure shields currents
• Sources and reasons not yet fully understood
CNGS TI8
100 A/div
D. B
ozzi
ni, E
N-E
L-H
T, A
TOP
Day
s 20
09, 4
-6 M
arch
200
9
11
CASE 2: BEQ1 compensator – 18 kV cable terminal failure
ConclusionASSEMBLY PROCEDURE NOT FOLLOWEDPOOR QUALITY OF WORK
Failure on a 18 kV cable terminalDate of installation 2003, place BE, Indoor
Discharge to ground without load
Analysis by third party (Cellpack Laboratory, DE)
Outcome of the analysisLength of insulation dimensions not done according to the procedure
Use of inappropriate tooling (knife) causing serious weaknesses on the insulation
D. B
ozzi
ni, E
N-E
L-H
T, A
TOP
Day
s 20
09, 4
-6 M
arch
200
9
12
CASE 3: RESTO 1 - Elastimold 18 kV elbow connector failure
Failure on a 18 kV Elastimold connectorDate of installation 2003, place Bdg 500, Indoor
Discharge to ground, circuit was on load
Analysis by third party (Cellpack Laboratory, DE)
Outcome of analysisPreparatory dimensions not done according to the procedure
Use of inappropriate tooling (knife) causing serious weaknesses on the insulation
ConclusionASSEMBLY PROCEDURE NOT FOLLOWEDPOOR QUALITY OF WORK
D. B
ozzi
ni, E
N-E
L-H
T, A
TOP
Day
s 20
09, 4
-6 M
arch
200
9
13
SPS Consolidation - Status on March 2009
18 kV cable network (SMB not included) and substationsMarch-09
Segment Length Stable Pulsed[m] Done Done
BE-B3 350 350 350B3-B4 1300 1300 1300B4-B5 1350 1350 1350B5-B6 1260 B6-B7 550 550 550B7-B1 1100 B1-B2 1520 B2-BE 1100 1100 Total [m] 8530 4650 3550Done [%] 55 42
D. B
ozzi
ni, E
N-E
L-H
T, A
TOP
Day
s 20
09, 4
-6 M
arch
200
9
14
SPS Consolidation – Short term work plan
From March 2009 till December 2010• Install cables between BA6-BA6 (2X), BA1-BA2 (2X),
BA2-BE (1X)
• Digging 6.6 km
• Prepare renovation of substations BA4 and BA2
• Define and implement scheme for parallel coupling of the BE-TI8 antenna with BE-BA4 pulsed loop
During shutdown 2010-2011• Commissioning of installed cables
• Renovation and commissioning of substations in BA4 and BA2
• Installation and commissioning of parallel coupling BE-TI8 with BE-BA4 pulsed loop
D. B
ozzi
ni, E
N-E
L-H
T, A
TOP
Day
s 20
09, 4
-6 M
arch
200
9
15
SPS Consolidation – Mid term work plan
By the end of shutdown 2012• Installation and commissioning of cables between
BA4-BA1 (2X)
• Digging 1.1 km
• Renovation and commissioning of substations BA1 and BA3
By the end of shutdown 2013• Renovation and commissioning of substation BA6
D. B
ozzi
ni, E
N-E
L-H
T, A
TOP
Day
s 20
09, 4
-6 M
arch
200
9
16
BA4-TI8 network layout
Situation• Power line added in 2004 to power the
CNGS experiment• On this line we also have the SPS stable
network for BA4• Not being part of a loop -> no redundancy for
powering CNGS and SPS pulsed network in BA4
TI8
Consequences• If a fault appears on this line (ex: joint to be
replaced), one week of shut down minimum
Remedies• Implement possibility to operate the TI8 line
in // to the SPS pulsed loop from BE to BA4
D. B
ozzi
ni, E
N-E
L-H
T, A
TOP
Day
s 20
09, 4
-6 M
arch
200
9
17
SMB
Consequences• If a fault appears on this line, one week of
shut down minimum of the affected segment
How to proceed?• 1) Receive guidelines whether to include
these circuits in the EN/EL SPS consolidation mandate or:
• 2) as recommended by the ATC on 27th Oct. 2006 study and find procedures for a fast repair and shorten DTTR
Situation• Old power lines affected by water treeing
effect ( 1970) • Failure rate going up with the time• Operation is not 100 % redundant (see 4th
ATC minutes 27th October 2006)• Fully operated by TE-EPC, maintained by
EN/EL
D. B
ozzi
ni, E
N-E
L-H
T, A
TOP
Day
s 20
09, 4
-6 M
arch
200
9
18
Failure location on HV network
Current procedure• As soon as a failure is detected, ERDF (Electricite Reseau Distribution
France) is contacted and they intervene on average within three days to precisely localize the failure.
Technique• In most of t he cases reflectometry coupled with a pulsed surge generator.
Limits• Time of intervention depends on availability of ERDF. Also to consider that
EDF network diagnostic has priority with respect to CERN.
How to improve• EN/EL has almost all the equipment to perform the same type of diagnostic• A bit old but can be upgraded with a dedicated budget• Today there are weaknesses on expertise, but if necessary on a short mid
term period we can envisage to set up a EN/EL team for such diagnostic
What is needed?• Dedicated personnel, time for set-up and time for training on real cases
D. B
ozzi
ni, E
N-E
L-H
T, A
TOP
Day
s 20
09, 4
-6 M
arch
200
9
19
CONCLUSION
Failures• 6 out of 8 major failures or near miss are related to bad assembly of HV components
• For the SPS, except for the “joint crisis”, 2008 as not been a bad year, 2 faults due to aging cables. One affecting SMB. But…remember what we saw on the video!
• Cells and substations: let us not underestimate the status of ME59 and ME9
Serious problem of HV components assembly quality• Identification of specialized firms in ongoing with good results
• In-house expertise must be improved in order to follow the execution of repair
High currents in SPS shields• Require more time to investigate the sources of these currents
Consolidation of SPS 2009-2010 (assuming LHC will run over the winter)• Installation of cables on two segments (BA5-BA6 and BA7-BA1)
• Re-configure substations in BA2, BA4 and improve the TI8 network layout
• Require solid budget and availability of sufficient and fully qualified personnel
• SMB: We need to know now what to do
• Failure location: there is room for improvement of the MTTR