project management at the lhc: a 20 year journey

Project Management at the LHC: a 20 year journey

Roberto Saban

ITER Project Management Conference Cadarache September 4th 2015

Roberto Saban CERN 2

What this talk is NOT A lecture on Project

Management Basics A description of Project

Management Tools A sales talk for a Project

Management Product

… but rather

ITER September 4, 2015

OutlineCERN & the LHC ProjectQuality AssuranceConfiguration ManagementDocumentationPlanning & SchedulingEarned Value ManagementLessons Learnt


CERN was founded in 1954 as a Science for Peace Initiative by 12 European States

Member States: Austria, Belgium, Bulgaria, the Czech Republic, Denmark, Finland, France, Germany, Greece, Hungary, Israel, Italy, the Netherlands, Norway, Poland, Portugal, Slovakia, Spain, Sweden, Switzerland and the United Kingdom Associate Member: Turkey, PakistanStates in accession to Membership: Romania, SerbiaApplicant States for Membership or Associate Membership: Brazil, Croatia, Cyprus, India, Russia, Slovenia, Ukraine Observers to Council: India, Japan, Russia, United States of America; European Commission, JINR and UNESCO

2300 staff: physicists, engineers, technicians, administrative personnel1300 personnel on contract11500 users who represent more than half of the living particle physicists originating from 500 universities and institutes and of more than 80 nationalities

Annual budget 1 BCHFITER September 4, 2015

The Mission : research, develop technologies, train scientists


The technologies generate innovation


Accelerators Detectors

Hadron therapy The Computing Grid

CAT


The accelerator complex


Roberto Saban CERN 6ITER September 4, 2015

The specifications of many systems were over the state of the art. Long R&D programs with many institutes and industries worldwide were necessary.

The technological challenges of the LHC

the largest superconducting magnet system

~10’000 magnets

the highest field dipole accelerator magnets 8.3 T

the largest 1.9 K cryogenics installation

superfluid helium, 150 tons of LHe to cool down 37’000 tons of StSt

ultra-high cryogenic vacuum for the particle beams

10-13 atm, ten times lower than on the Moon

the highest currents controlled with high precision

up to 13 kA

the highest precision ever demanded from the power converters

ppm level over several orders of magnitude

a sophisticated and ultra-reliable quench detection and magnet protection system energy stored in the magnets ~10 GJ,

energy stored in the beams > 700 MJ


The LHC last week



The Medium Term Operation


LHCb b b b b b b b b b b b b b b b b b b b b o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o b b b b b b b b b b b b o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o

Injectorso o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o b b b b b b b o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o

LHCo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o b b b b b b b b b b b b o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o

Injectorso o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o b b b b b b b b b b b b o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o

LHCo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o b b b b b b b b b b b b o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o b b b b b b b b b b b b o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o

Injectorso o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o b b b b b b b b b b b b o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o b b b b b b b b b b b b o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o

Q1 Q2 Q3 Q4 Q1Q2 Q3 Q4 Q1 Q2 Q32035

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q4Q2 Q3 Q4 Q1 Q2 Q3Q4

Q1 Q2 Q3 Q4

2029 2030 2031 2032 2033 2034

Q3 Q4 Q1 Q2 Q3 Q4Q1 Q2 Q3 Q4 Q1 Q2Q3 Q4 Q1 Q2 Q3 Q4Q1 Q2 Q3 Q4 Q1 Q22022 2023 2024 2025 2026 2027 2028

Q1 Q2 Q3 Q4 Q1 Q2Q3 Q4 Q1 Q2 Q3 Q4Q1 Q2 Q3 Q4 Q1 Q22020 2021

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q3 Q42015 2016 2017 2018 2019

PHASE 1Run 2

Run 3

Run 4

LS 2

LS 3

LS 4 LS 5

PHASE 2

LS 4 LS 5Run 5

Run 3 Run 4

HL-LHC installation

LIU installation


The LHC & its lifecycle


+ th

an 2

5 ye

ars

1983 First studies for the LHC project1988 First magnet model (feasibility)1994 Approval of the project by the CERN

Council1996-1999

Series production of the magnets

1998 Declaration of Utilité Publique and start of civil engineering

1998-2000

Main production contracts

2004 Start of the installation of the LHC2005-2007

Installation of the magnets in the LHC tunnel

2006-2008

Hardware commissioning

2008-2009

Beam commissioning

2009-2035

Physics

Very long lifecycle therefore the need for internal knowledge transfer.

One generation will design build and commission the accelerators, the next one will operate, maintain and decommission it. therefore

Product Lifecycle Management is essential


Project Timeline


Project AdministrationTechnical Coordination & Planning Quality Assurance Plan Engineering Documentation Procurement Process Manufacturing Process Test Results

Earned Value Management Interfaced to accounting system Actual Costs Interfaced to contract management system Interfaced to human resource management

system In-kind contributions

Member States asked CERN Management to set up an appropriate Project Control System


A journey


We were convinced that we had a well-oiled machine from LEP construction days but we realized that the legacy tools that we had where inadequate for the project

for the control of the schedule they consisted in some planning tools where the baseline was regularly re-adjusted

for the control of the cost they consisted in precise but rudimentary accounting tools showing expenditure versus time

with no link between them

Cour

tesy

L.E

vans


LHC is a mega-project


as such it contained many project types in one project

A single PM methodology could not be imposed, instead it had to become and aggregation of practices

Cour

tesy

P.Bo

nnal


Committees


Few but empoweredPLC Parameter & Layout

The PLC established and maintained the list of parameters of the project and managed the associated functional lay-out. The parameter list included beam parameters as well as the parameters of the components of the accelerator. These were regularly reviewed in the light of the progress of the design and critically assessed in terms of global impact to the performance of the machine.

TCC Technical CoordinationThe TCC was in charge of the baseline of the project as designed to satisfy the parameter set defined by the PLC and the integration of machine components. It maintained the configuration, the documentation of the main systems and their subsystems. Controlled change management critically reviewing the impact on interface systems, on the schedule and on the cost of the project.

Machine Advisory Monitored progress, cost and availability of resources. Helped to sort out difficult technical issues.


QA in the LHC Project



Configuration Management


A set of approved and released documents that represents the definition of a product at a specific point in time

Configuration baselines are established whenever is necessary to define a reference configuration during the lifecycle of the product

The baseline is used as a starting point for further activities until it is revised in a controlled way

The responsibility of the Project Engineers is reflected by the organization in subprojects

The configuration is mirrored in a tree structure where all the documentation is contained

Insp

ired

by P.

E.Fa

uger

as


Quality | Processes and documents


Naming conventions Document types Central repository and procedures for the handling of documents the specification of parts the recording and processing of test results the follow-up of the progress the exchange of documentation with

contractors and partner institutes One lifecycle for each document type Predefined circulation and approval lists to

ensure interface system owners are aware of and approve specifications or changes.

Close follow-up of implementation of changes in terms of interfaces, schedule, budget, etc.

EngineeringDataManagementSystem


From project to installation


Functional and Engineering Specificationsto make people describe their requirements, their interfaces, their engineering

Engineering Change Request/Orderto ensure the information is up-to-date at a given time, and shared with all those participating to the projectto control changes through validation and update impacted specifications

CDR(Conceptual Design Report), TDR (Technical Design Reports), Schematics, 2D-Drawings, 3D-Mockups, Technical Notes, Technical Reports, Technical Datasheets, various lists, BoMs , Procurement Documents (TD,TQ, QC, TS, TF), Scientific Publications, illustrations, sketches, photos, videos

Cour

tesy

P.Bo

nnal

, S.C

hem

li


From installation to commissioning


Cour

tesy

P.Bo

nnal

, S.C

hem

li


Manufacturing and Test Folder


Cour

tesy

Ph.

Lebr

un


The installed collider


Cour

tesy

P.Bo

nnal

, S.C

hem

li

The installed machine evolvesComponents are exchangedNew components are installedThe configuration changes

Engineering Change Request/Order

Changes are documentedThe impact on safety, infrastructures, on neighboring systems, planning, budget is carefully studiedThe layout is updated to ensure the integration of the next changes is possible


Change Control


Cour

tesy

Ph.

Lebr

un


The Master Schedule


The strategic goals and major milestones of the project

Cour

tesy

K.F

oraz


The tactic

s


Implement and control the flow of installation following the main

milestones of the master schedule

Baseline !

Cour

tesy

K.F

oraz

Three levels of planning & scheduling

The strategy

1

2


Cour

tesy

K.F

oraz

Three levels of planning & scheduling

Sect

or 1

2

Sect

or 2

3

Sect

or 3

4

Sect

or 4

5

Sect

or 5

6

Sect

or 6

7

Sect

or 7

8

Sect

or 8

1

Inje

ctio

n Li

ne 2

Inje

ctio

n Li

ne 8 3

The detail


Review of the planning


The LEP Dismantling and the LHC Construction/Installation Schedules

The delay of the QRL imposed the reshuffling of some activities. Additional resources were added for the following phases +3 months

Hardware commissioning was included

Major non-conformities during installation: +2 months because of the QRL, +4 months because of the interconnects

Interconnect non conformities + Inner Triplet Crisis +7 months


EVM Basics


Cour

tesy

P.Bo

nnal


Industrialization & production ramp-upLHC superconducting dipole magnets



Schedule Variance


Cour

tesy

P.Bo

nnal


Cost Variance


Cour

tesy

P.Bo

nnal


Reporting to Council


Cour

tesy

L.E

vans

, Ph.

Lebr

un


EVM | Lessons learnt


Careful with the activities Number of activities stay below 500

Size of the activity 80% of the activities in range 0.2% – 2% BAC

Duration of an activity 10% project duration and 3 months

Unambiguous responsibility only one per activity

Systematics in breaking down to ease definition and taking over

Deliverable-oriented EVM “10 magnets out of 20” physical progress statement is more informative than a “50% complete” statement!


Analysis of the Risks


Each risk was assigned a measure for1.Likelihood rare, possible, likely, frequent, very frequent

2.Impact on CERN’s objectives insignificant, moderate, major, catastrophic

3.Impact on the interruption of operation hours, days, weeks, months

from which a score was derived to rank the risks.

A record was prepared for each risk giving a description, the owner, the measure which could be taken to mitigate it, its cost, its score and the new score if the mitigation measures are taken.

Based on this analysis, the Management took the decision on how to handle each risk. i.e. take mitigation measures or accept it.

LHC Project Management | in short

Strong Project Leader & control by a central team

Quality Assurance Culture Processes, documentation,

repositories Design, construction, assembly,

commissioning, operation Multi-level planning and scheduling

3 levels master, co-ordination and detailed

Deliverable-oriented EVM


Changed the culture inside the organizationContributed to cost consciousness at all levelsConveyed assurance to the stake holders that the project team was well in control of the project

✔

With many valuable contributions from P.Bonnal, S.Chemli, J. de

Jonghe, L.Evans, K.Foraz, Ph.Lebrun