greg newman - snc lavalin rail & transit - independent professional review – a value...

›The Value Preservation Postulate for Independent Professional Review

6th Annual Heavy Haul Conference 24th and 25th August 2016

Candice Augur, Section Lead, Safety & Assurance, SNC-Lavalin Rail & Transit

Greg Newman, Principal Consultant ,SNC-Lavalin Rail & Transit

Overview

2

› Introduction

› Our changing world

› Value Add c.f. Value Preservation with a Project

› What is Independent Professional Review?

› Definition

› Legislation/regulatory requirements

› When should it start?

› Case Study

› Conclusion

Globalisation of Projects

› A long time ago. . .

› All done in one place (by one company)

› But now . . .

› Multiple development sites (so design can lack cohesion)

› Multiple companies (so contracts can stand in the way)

› Multiple countries (so legislative background is different)

› Multiple time-zones (so verbal or face-to-face conversations are difficult)

› Multiple languages (so meaning can be lost in translation)

› Varying cultures (comply strictly with standards, only fulfill the contract, do what country-of-origin clients accept, . . .)

› Differing background of legal and regulatory environments (e.g. not everyone does SFAIRP)

3


› And always . . .

› Mix of new, modified and existing products in the system where system integration into a viable and operable whole is the key to success!

› Every railway is, in some way or another, unique!

4

Global Design

5


› All this adds to complexity and project risk – often also safety risk.

› System functionality is sophisticated – and hence interlinked.

› Technology is advanced:-

› GPS, inertial guidance, “learning” algorithms, increasing reliance on accurate geographical mapping, increasing integration between traditional rollingstock and train-carried signalling, interfacing to varied existing sub-systems (often of SOUP )

› Software standards are demanding.

› Suppliers (and consumers!) are competing in a global market.

› There may be global demand for your local resources

› International suppliers will naturally focus on their largest markets –this is not Australia!

6


› “Systems fall flat on their interfaces” – as a wise man once said.

› Realised Value can be lost across these multiple and complex interfaces . . .

› Varying levels of competence/understanding

› Mismatches in contracting scope and arrangements

7

Value Adding in Designing the Wheel

8

Lifecycle Stage Anticipated “Value Add”

Conceptualisation That wonderful bright idea! (Say, The Wheel)

Specification Definition of the salient properties (low friction hub, load of 1000kg, diameter 400 mm etc.)

Design The drawings and material selection

Manufacture A built wheel on the factory shelf

Installation A set of 4 these fitted to your car

Operation Being able to drive somewhere nice

9

Potential Value Vs Realised Value

10

Potential Value

Realised Value

What is Independent Professional Review?

› Put simply, it’s a second competent, knowledgeable person/group that have no involvement in, ownership of, or responsibility for a particular thing or endeavour.

› Independent Professional Review objectives are to ensure that processes are adequate and that evidence in relation to those activities is available in a structured and systematic manner.

› Technical Independent Professional Review is a process to ensure that the design meets particular standards and requirements, is comprehensive, and has not over-looked professional critical issues.

› Independent Professional Review can include:-

› Independent Safety Assessment

› Independent Verification & Validation

› Financial review

› Contract performance review

12

When to Engage? (and by Whom?)

› Works best if the IPR is engaged before the “risk” is entered into.

› Otherwise the best that can be hoped for it to tell you where the value has already gone.

› Like quality, it cannot be “painted on at the end”

› Works best if the IPR is engaged by the asset owner.

› Its their end risk!

› It’s their money!

› It’s their value to preserve!

13

Case Study: Transport for London

› Details publicly available at:-

› “Transport for London’s Signal Failure” London Assembly, March 2016 [Ref 1]

› https://www.london.gov.uk/sites/default/files/transport_for_londons_signal_failure.pdf

› “Sub-Surface Upgrade Programme Automatic Train Control Contract –Lessons Learnt” Transport for London Finance and Policy Committee, 17 July 2014 [Ref 2]

› http://content.tfl.gov.uk/fpc-20140717-part-1-item10-sup-atc-lessons-learnt.pdf

14

https://www.london.gov.uk/sites/default/files/transport_for_londons_signal_failure.pdf

http://content.tfl.gov.uk/fpc-20140717-part-1-item10-sup-atc-lessons-learnt.pdf


› In June 2011 TfL awarded a contract for the resignalling of their Subsurface lines (known as the Sub-Surface Upgrade Project or SSUP) for completion in 2018 at a cost of £354 Million (source - Railway Gazette 14 June 2011).

› The Contract required extensive development and modification of high integrity software for a complex signalling system for a unique and difficult application.

› “TfL relied on the contractor to provide an end-to-end solution. The market is not mature enough to take this approach now (March 2016) and it certainly was not five years ago” [Ref 1]

› “Reference sites in XYZ city and ABC city which the Contractor relied upon during the contract negotiations had failed to deliver against expectations” [Ref 2]

› In December 2013 they (TfL) took the decision to end the contract and pay out the contractor due to continuing delays and cost overruns.

› An alternative contractor has now been set in place, but virtually none of the work done by the previous contractor (software development) can be used by the new contractor. (i.e. little to no “value” has been preserved). Not expected to complete until 2023.

15


› The players:

16

Transport for London

Contractor

Independent Investment

Programme Advisory Group

•Only 6 or 7 part time

staff.

•Just 450 person days

per year.

•But 40 projects to

cover

•“not posses the necessary

technical expertise . ..”” [Ref 1]

•“determined by project cost . .

. rather than its risk”” [Ref 1]

•110 projects to cover

Project Management Office


› What happened and how could IPR have helped?

17

Eventually IPR Value preservation activity

The capability of the supplier to deliver these

benefits had not been adequately

demonstrated or interrogated [Ref 1]

Pre-contracting market and tender evaluation

including past project review and

technology/capability assessment by assessment

and audit

Neither TfL’s or the Contractor’s

management teams were up to the task of

managing the programme [Ref 1]

Review of project/programme management

processes, tools, capability and competence

against world-class benchmarks.

The lack of suitably skilled people in the

wider labour market should have been

another indication of the high risk nature of

the programme [Ref 1]

Assessment of competency management

arrangements in key PM and technology areas

coupled with succession planning and “depth-

of-competency/key-person” risk assessment

At certain periods of the contract a culture of

optimism and “good news" prevailed such

that underlying technical issues were not fully

confronted [Ref 2]

Vertical slice audit of data collection and

reporting processes.



18


TfL failed to take a risk-based approach to its

internal assurance work [Ref 1]

Evaluation of client, contractor and

technology solution followed by

independent risk assessment focussed on

risks to successful completion .

Ultimately the programme failed because the

contractor was unable to deliver what it said it

would deliver [Ref 1]

Competence, capability and technology

assessment BEFORE contract award (pre-

qual or Tender Evaluation short listing)

TfL should have carried out greater due

diligence on the capabilities and management

arrangements at the Contractor prior to signing

the contract. [Ref 1]

Ditto

The key to ensuring that a “good news culture”

does not persist beyond the (cancelled) contract

is to improve TfL’s assurance function. [Ref 1]

Vertical slice audit of data collection and

reporting processes.



19


key known risk areas such as the safety

engineering processes and safety integrity

levels could have been probed further, for

example through a short “vertical slice”

review through the assurance process, to

verify certification and safety integrity level

(SIL) claims. [Ref 2]

The value of good independent auditing.



20

Recommendations directly related to IPR

It (the assurance function) needs to be given the capacity and capability to provide an

unbiased view of project progress. [Ref 1]

Given the complex nature of the programme, assurance reviews should be carried out on

a more frequent and regular basis (irrespective of key stage dates), typically every six

months, examining key areas in greater detail and granularity. We would also advocate that

the services of specialised external experts are used to support the PMO including to fulfil

the technical aspect of their duties. This support should remain in place during the

procurement and subsequent delivery of the contract. [Ref 2 – Rec 23]

Undertake a sufficient resource capacity and capability analysis of the market as part of

the procurement and commercial strategy development phase.[Ref 2 – Rec 1]

Undertake deep due diligence of the supply chain organisations in relation to their

effectiveness of internal operational management and integration of their relevant global

businesses. Particular attention should be paid to assessing the capability of the supplier

leadership and management team. An option could be to interview key individuals on the

supplier leadership and management team [Ref 2 - Rec 4]


› Value not preserved [Ref 1]:-

› £85 million in settlement costs

› £271 million in lost/delayed revenue (5 years delay)

› £886 million less in the capital works budget

› Inconvenience to the travelling public (11 million fewer journiesper year [Ref 1])

› Reputational damage.

21

Conclusion

› IPR preserves potential value, it does not add value.

› IPR comes in many different flavours, ISA, IV&V, Financial, Contract performance etc

› The IPR should be engaged by the owner, not the Project (It’s the Owner who carries the final consequences and pays the bills)

› The scope and degree of detail in IPR should be agreed between the IPR provider and the owner.

› Should begin as soon as the exposure to risk is committed.

› The degree of independence and level of oversight should be proportional to the overall risk (safety risk, financial risk, environmental risk, project completion risk etc)

22

›“With an annual budget of just over £0.5 Million, yet the potential to save TfL hundreds of millions, we need to understand how the quality and value of its (IIPAG’s) work would increase if it was given more resources”. (Speaking of the independent assurance function) [Ref 1]

›For the financially minded, that’s a potential 20,000%+ ROI!

greg newman - snc lavalin rail & transit - independent professional review – a value...

Business