l1-che-man-001 engineering management systems (ems ... · engineering management systems (ems)...
TRANSCRIPT
Engineering Manual Systems Engineering
L1-CHE-MAN-001
Engineering Management Systems (EMS) Methodology
Version: 1
Issued: 20 February 2014
PRINTOUT MAY NOT BE UP-TO-DATE; REFER TO METRO INTRANET FOR THE LATEST VERSION
ENGINEERING MANAGEMENT SYSTEM (EMS)
METHODOLOGY
Type: Manual
Author: Engineering Systems Manager
Page: 2 of 59
L1-CHE-MAN-001(1) Ap proval Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Table of Contents
1 Introduction .................................................................................................. 3 1.1 Purpose ........................................................................................................................... 4
1.2 Referenced Documents ................................................................................................... 4
1.3 Abbreviations/Acronyms ................................................................................................ 5
1.4 Definitions ...................................................................................................................... 7
1.5 Overview....................................................................................................................... 13
1.6 Systems Engineering – Acquisition, Development & Operation ................................. 15
1.6.1 Systems Engineering Phases................................................................................. 15
1.6.2 Systems Engineering Management....................................................................... 16
1.7 Components of the Engineering Management System - General ................................. 17
1.7.1 Project Plans ......................................................................................................... 17
1.7.2 Authorisation ........................................................................................................ 18
1.7.3 Process Control ..................................................................................................... 18
1.7.4 Prime Phase Based Process .................................................................................. 20
1.7.5 Support Processes ................................................................................................. 22
1.8 Key Concepts of the Engineering Management System .............................................. 24
1.8.1 The Scope of an Engineering Project ................................................................... 24
1.8.2 Key Components .................................................................................................. 26
1.8.3 Project Management Phases ................................................................................. 28
1.8.4 Executive Management Oversight ........................................................................ 31
1.8.5 Systems Engineering Phases................................................................................. 33
1.8.6 Software Engineering Phases ............................................................................... 38
1.8.7 Life Cycle Models ................................................................................................ 42
1.8.8 Selecting the type of software development strategy ........................................... 43
1.8.9 Engineering Reviews ............................................................................................ 47
1.8.10 Action Item and Problem Tracking ...................................................................... 52
1.8.11 Prototype and Concept Demonstrator Development - Software .......................... 54
Appendix 1: Project Engineering Manager – Role description......................................... 56
Appendix 2: Criteria for Establishing PEM Role on MTM Projects ............................... 58
EMS METHODOLOGY
3 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
MTM’s project management and engineering methods and processes are evolving and
maturing, in some cases from a relatively low base, or early stage, of maturity.
Whilst the concepts contained in this ‘EMS Methodology’ may appear complex or unfamiliar, it
attempts to capture methods that are generally and universally applied across various industries that
successfully develop and deliver systems and software engineering products/solutions within
projects, particularly those that are complex, safety and/or service critical.
This Methodology can be tailored and scoped to suit MTM’s individual projects and
appropriate, associated systems engineering and software engineering acquisition and
governance approaches.
This EMS Methodology presents a universal, contemporary 3-tier model (in Figures 8 to 19
inclusive) of Project Management, Systems Engineering and Software (acquisition) Engineering
that depict their respective activities and process interactions. This 3-tier model is used in
support of contemporary engineering approaches underpinning this EMS Methodology.
This Methodology will, itself, need to maintain alignment with MTM’s evolving engineering
and related business policies and approaches. It may undergo change and improvement over
time as MTM progressively matures its engineering and associated project management
processes to attain the requisite levels of engineering process maturity whilst necessarily
meeting its ongoing rail engineering regulatory (ARO) and safety obligations.
EMS METHODOLOGY
4 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1 INTRODUCTION
1.1 Purpose
The purpose of this systems engineering methodology is to provide a ‘high level’ engineering
framework, and outline and describe a ‘top down’ and ‘bottom up’ structure of the proposed
Engineering Management System (EMS) and methodology to be implemented within MTM
for its Project and Infrastructure engineering activities, particularly those projects or
acquisitions that involve complex and/or service/safety critical engineering solutions.
This EMS provides a high level, systems and software (latter with MTM as an “acquirer”)
engineering lifecycle approach for addressing governance, management and technical aspects
of engineering work applied to MTM projects and renewals etc. As such, its engineering
coverage provides for application to a wide range of MTM projects and works of differing
scope, size, risk (technical, safety and commercial), complexity, standards and technical
challenge. Accordingly, its implementation will provide for ‘tailoring’ to meet the specific
management and technical needs relevant to the project or Infrastructure activity, ensuring that
process overheads are planned and minimised whilst meeting contractual, technical, risk, safety
and other associated management requirements in a consistent, repeatable and profitable
manner.
Whilst numerous, changing standards, methodologies and emerging technologies continue to
redefine engineering approaches for systems and software solutions, an EMS provides a stable
foundation upon which MTM can base its engineering approach. Such an approach aims to
encapsulate ‘good engineering practice’ that embodies the principles espoused by
contemporary industry and will respond to lessons learned through MTM’s project and
Infrastructure experience as an ARO in the rail industry. It will be continually be reviewed,
improved and matured, to retain its relevance to, and meeting, MTM’s rail industry regulatory
requirements and MTM’s broader strategic, business needs.
The EMS forms part of the overall MTM integrated management system, which has been
independently 3rd
Party certified to ISO 9001 as well as safety and environmental Standards.
The EMS must comply with all safety system assurance requirements set by MTM including
the System Safety Assurance Framework and Safety Management System to meet the safety
requirements of the State as well as reinforcing MTM’s compliance with its ARO obligations.
1.2 Referenced Documents
IEEE STD 1220 Systems Engineering - Application and management of the
systems engineering process (ISO/IES 26702)
ISO 9001:2008 Quality Systems - Model for Quality Assurance in design,
development; production, installation and servicing
ISO/IEC 15288 Systems and software engineering – System life cycle
processes
ISO/IEC 12207 Information Technology – Software life cycle processes
EN50126 Railway applications – The Specification and demonstration
of Reliability, Availability, Maintainability and Safety
(RAMS)
EMS METHODOLOGY
5 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
EN50128 Railway applications – Communication, signalling &
processing systems -SW for railway & protection systems
TBA (draft completed) Engineering Work Breakdown Structure (WBS) Guideline
PRINCE 2 Projects IN Controlled Environments, second revision
Project Management for Business. (CCTA, UK)
L0-SQE-MAN-002 MTM Safety Management System Manual
L1-ASY-PRO-001 Engineering Change Procedure
L1-NAM-PRO-001 MTM Signalling Design Technical Review Procedure
L1-NAM-PRO-002 MTM Design and Technical Review Procedure
L1-PRJ-PRO-001 Project Management Propcedure
L1-PRJ-PRO-002 Gate Procedure for Project Managers
L0-SQE-PRO-031 Enterprise Risk Management Procedure
L1-SQE-PRO-001 Management of Change Process
1.3 Abbreviations/Acronyms
AM Asset Management
AMP Asset Management Plan
ARO Accredited Rail Operator
CCB Configuration Control Board
CDR Critical Design Review
CI Configuration Item
CM Configuration Management
COTS Commercial Off The Shelf
CSCI Computer Software Configuration Item
D&C Design and Construction
DDR Destailed Design Review (software)
EC Engineering Change
EMS Engineering Management System
FAT Factory Acceptance Testing
FER Formal Engineering Review
FIS Final Impact Statement
FOR Final Operational Requirements
HMI (HFE) Human Machine Interaction (Human Factors Engineering)
HSEQ Health Safety Environment & Quality
HV High Voltage
IADT Inspection, Analysis, Demonstration, Testing
EMS METHODOLOGY
6 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
MoC Management of Change
M&R Maintenance & Renewal
MTM Metro Trains Melbourne
NAM Network Asset Management
NDT Non-Destructive Testing
OHS Occupational Health & Safety
PAM (draft) Project Authorisation Matrix
PCM (draft) Process Compliance Matrix
PDP Project Development Plan (for project planning phase)
PDR Preliminary Design Review
PEM Project Engineering Manager
PM Project Manager
PMB Performance Management Baseline
PMP Project Management Plan (for project delivery phase)
PRR Preliminary Requirements Review
QM Quality Manager
RAM Responsibility Assignment Matrix
RAMS Reliability, Availability, Maintainability and Safety
SCR System Change Request
SEMP Systems Engineering Management Plan
SIL Safety Integrity Level
SMS Safety Management System
SOR Statement of Requirement
SOW Scope of Works
SPR System Problem Report
SRA Stakeholder Requirements Assessment
SRR System Requirements Review
SRS Software Requirements Specification
SSAF Systems Safety Assurance Framework
STD Software Test Description
TA Type Approval
T&C Test and Commissioning
TRR Test Readiness Review
TSI Test Support Item
VDD Version Description Document
WBS Work Breakdown Structure
EMS METHODOLOGY
7 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1.4 Definitions
Allocated Baseline Comprises all the specifications down to the lowest level
specification in the system (allocating sytem requirements to lower level) and is established
during a review of the design (usually Preliminary Design Review) when the
development/performance requirements specifications are approved by the User or Client.
Baseline: The approved definition of a product at a selected point in
its life-cycle and establishes a reference basis for evolution and enhancements. The Baseline
plus approved changes constitue the current, approved configuration items at a point in time.
Baseline Management: Schedules control points for an in-process review of the
development progress and enables optimised scheduling for the completion of requirements
and design. (Refer to Figure 1 below for example of Baselines and their lifecycle.)
Client: MTM’s client, usually Public Transport Victoria.
Configuration Baseline: (1) The currently approved and released configuration
documentation, or suite of documentation, at a specific revision that describe the agreed-to
description of the attributes of a system of products, at a point in time, that serves as a basis for
defining status and managing change. (2) A released suite of files comprising a software
version and associated configuration documentation.
Configuration Item: The fundamental structural unit (hardware or software) of a
configuration management (CM) system. A Configuration Item may be an aggregation of
hardware, software, processed materials, services or any of its discrete portions that are
designated for CM and treated as a single entity in the CM process.
Configuration Management (CM): CM comprises configuration planning, identification,
control, status & accounting and audit (FCA and PCA). It is an essential, fundamental element
of engineering management and control. The EC process is a current, key process that applies
aspects of CM within MTM (viz. configuration identification and control).
Design Baseline: Established at successful completion of the CDR and captures all
specification, design documents etc that comprise the detailed design approved at CDR.
Engineering Change (EC): The EC is an engineering process used within MTM to initiate,
develop, approve, implement and audit changes to (asset) configuration, maintenance and
technical standards associated with MTM Rolling Stock and Infrastructure. Key aspects of the
EC process are risk assessments, continuity with the MoC process and assuring that adequate
engineering methods are applied as part of the EC lifecycle and alignment with asset
management stratgey (as applicable) is achieved. The EC is currently a principle element of
configuration management within MTM.
Engineering Reviews and Baselines: Depicted below is shown the proposed MTM system
engineering lifecycle showing the various engineering Baselines, their association with systems
(hardware and software) development and formal engineering Reviews, and when informal
versus formal configuration management is applied during the systems engineering lifecycle.
EMS METHODOLOGY
8 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Functional Baseline: Represents the top level, system requirements (defined in systems
specification and interface documents) and is the foundation for configuration management by
the organisation during subsequent phases of the product/system development cycle, usually
established at the System Requirements Review (SRR). The Functional Baseline
documentation describes the systems functional, performance, interoperability and interface
requirements and the verifications required to demonstrate the achievement of those specified
requirements.
Functional Configuration Audits (FCA): FCAs are performed to formally examine (viz.
testing or demonstration) the functional characteristics of a configuration item or system, prior
to acceptance or commissioning, to verify that the item has achieved the requirements specified
in its functional and allocated configuration documentation. FCAs precede PCAs.
Product Baseline: Describes all functional, physical, interoperability, production, test and
support characteristics for the CI and is usually established at the completion of the PCA or its
equivalent. The Product Baseline incorporates the Allocated Baseline documents that describe a
CI’s functional, performance, interoperability and interface requirements and the verifications
required to confirm the achievement of those specified requirements. The Product Baseline
comprises the complete product definition package that includes such things as engineering
drawings, installation and construction drawings and instructions, material and process
specifications, approved designs, approved test documentation, approved Engineering Changes,
approved Waivers and special test equipment designs.
Physical Configuration Audit (PCA): PCAs are performed to formally examine (by
inspection or analysis) the ‘as built’ configuration of a CI against its approved design
documentation. PCAs usually follow FCAs (of which there may be several before readiness to
establish the Product Baseline). FCAs precede PCAs.
Performance Management Baseline (PMB): The PMB is a time-phased budget plan
against which contract performance is measured. The PMB also includes budgets assigned to
higher level WBS elements. (Refer to section 1.8.3.2 of this Methodology).
Stakeholders Requirements Assessment (SRA): The SRA is conducted during the pre-
award ‘Development’ phase to ensure that System Requirements are assessed early, at a high
level, against operational, technical, interface, quality, safety and maintenance requirements
(refer to Figure 2 “Stakeholder Requirements Assessment”).
System Requirements: ‘System requirements’ constitute the engineering requirements
and comprise 1) ‘Functional Requirements’ that describe what the system/solution is required
and capable to do including interfacing and environmental aspects; 2) ‘Performance
Requirements’ that describe how well the system/solution performs in measurable terms; and
3) ‘Non-Functional Requirements’ (also called ‘ilities’ (i.e. quality factors), attributes or
characteristics) that describe constraints that affect design solutions.
NOTE 1: Requirements need to be unique, complete, unambiguous, implementable,
verifiable and remain consistent with all other requirements (via requirements traceability).
NOTE 2: Based on the Projects Agreement, MTM establishes, during ‘Project Planning
& Development’ phase, a hierarchy of ‘operational’ requirements and impact statements viz.
Preliminary Operational Requirements (POR), Preliminary Impact Statements (PIS), Final
Operational Requirements (FOR) and Final Impact Statements (FIS) – captured in a ‘Project
EMS METHODOLOGY
9 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Brief’ (that may also include project draft Pre-Award concept design, as well as costs and
risks). System engineering requirements derive from these front-end operational requirements.
NOTE 3: ‘Functional’ and ‘Performance’ requirements essentially describe the
capability of the system/solution being produced. ‘Non-Functional’ requirements may include
the following categories:
o Physical requirements (dimensions, weight, mass, mounting points etc)
o Interface requirements (interaction of the system with external item(s))
o D&C requirements (drawn from applicable specification(s) and Standard(s))
o Quality factors requirements (e.g. safety, security, reliability, maintainability,
accessibility, availability, human factors/HMI, testability, interoperability)
o Environmental requirements (temperature, vibration, humidity, EMI/EMC,
pressure, contamination, etc)
o Training requirements (incl. system familiarisation)
o Documentation requirements (document deliverables quantities, format etc)
o Quality control & assurance requirements (inspections, special examinations and
testing (incl. NDT, sampling, ‘special processes’)
o Delivery requirements (incl. packaging, handling, marking, labelling, storage)
User: For MTM Infrastructure, Head of Network Asset Management. For MTM
Rolling Stock, the GM of Rolling Stock.
Verification: The process of demonstrating that the system and subsystems are built
and function according to the requirements and design documents i.e. “We have built the
system right”.
Validation: The process of assuring that the high-level user requirements’ behaviour
and performance is acceptable under operating conditions (in the user environment) i.e. “We
have built the right system”.
NOTE: Prior to system integration and testing and T&C, the traceability of
technical/engineering requirements (derived from the contracted requirements) to verification
and validation requirements (IADT) should be established to form the basis for planning,
IADT in order to objectively confirm these contracted requirements have been satisfied.
Applying a VCRM (Verification Cross Reference Matrix) is a useful approach that can provide
this traceability and is used, for example, as part of MTM testing and commissioning
Works Readiness Reviews: An MTM governance program that utilises “T- (weeks)” gates to
asses and ensure planned major works packages are achieving readiness criteria. Engineering
reviews (SRR, CDR and TRR) and the post-award Concept Design Gate are integrated with
WRR and are shown in the Figure 1 “Overview of EMS Baseline and Review Lifecycle”.
NOTE: Figure 3 depicts the combined SRA that precedes, and is linked to, the EMS Baseline
and Review lifecycle.
EMS METHODOLOGY
10 of 59
L1-CHE-MAN-001(1) Approval Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Handover/
Deployment
Acceptance
Review
Informal (‘Engineering’) CM Formal CM
System
Concept
Requirements
System/HW Reqts
Analysis and
Definition
System/SW Reqts
Analysis and
Definition
HW
Specification(s)
Development
Preliminary
Design
Detailed Design
SW
Specification(s)
DevelopmentArchitectural
(Preliminary)
Design
Detailed Design
System
Integration and
Testing
Testing and
Commissioning
SW
Implementation,
Unit Testing,
Integration &
Testing
Fabrication of HW
Product to
Qualified
Design
HW Product
Testing
SW Product
Testing
SRR PDR CDR TRR
DDR
CDR
CDR Critical Design Review
CM Configuration Management
DDR Detailed Design Review
HW Hardware
PDR Preliminary Design Review
PRR Preliminary Requirements Review
SRR System Requirements Review
SW Software
TRR Test Readiness Review
WRR Works Readiness Review
HW development
SW development
Functional Baseline Allocated (Development)
Baseline
Design Baseline Product Baseline
Preliminary Definition & Design Detailed Design Qualification and Fabrication System Integration, T&C and Deployment
PRR
Concept Definition & Analysis
Occupation
X XX
T-20 T-12 T-5
Mandatory, Formal Review
Optional Review
X Works Readiness Gate
Concept
Design
Formal Design Gate
Figure 1: Overview of EMS Baseline and Review Lifecycle
EMS METHODOLOGY
11 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
The SRA is held pre-award, during the “Development” phase. Two SRAs are usually held, one prior to drafting of the POR and PIS (to evaluate multiple options), and the second to assess the requirements of the selected, single, final option.
Stakeholders at these SRAs will include representatives from Network Strategy & Development Group (NSD),
Engineering, Quality, Safety, Operations, Infrastructure Delivery and from other MTM areas as required.
Figure 2: Stakeholders Requirements Assessment
Multiple
Options
PROPOSED
POR &
PISProject Brief
FOR &
FIS
Stakeholders Requirements
Assessment (“SRA”)
APPROVED
Single,
Final
Option
NPD, NAM, ENGINEERING, SAFETY
EMS METHODOLOGY
12 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Figure 3: SRA and Engineering Reviews Lifecycle
Multiple
Options
PROPOSED
POR &
PISProject Brief
FOR &
FIS
Stakeholders Requirements
Assessment (“SRA”)
APPROVED
Single,
Final
Option
NPD, NAM, ENGINEERING, SAFETY
System
Concept
Requirements
System/HW Reqts
Analysis and
Definition
System/SW Reqts
Analysis and
Definition
HW
Specification(s)
Development
Preliminary
Design
Detailed Design
SW
Specification(s)
DevelopmentArchitectural
(Preliminary)
Design
Detailed Design
SW
Implementation,
Unit Testing,
Integration &
Testing
Fabrication of HW
Product to
Qualified
Design
SRR PDR CDR
DDR
CDR
CDR Critical Design Review
CM Configuration Management
DDR Detailed Design Review
HW Hardware
PDR Preliminary Design Review
PRR Preliminary Requirements Review
SRR System Requirements Review
SW Software
TRR Test Readiness Review
WRR Works Readiness Review
HW development
SW development
Functional Baseline Allocated (Development)
Baseline
Design Baseline
PRR X X
T-20 T-12
Mandatory, Formal Review
Optional Review
X Works Readiness Gate
Concept
Design
Formal Design Gate
EMS METHODOLOGY
13 of 59
L1-CHE-MAN-001(1) Approval Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1.5 Overview
This methodology provides an overview of a proposed MTM company-wide approach to
engineering by describing the various components of an EMS, based in part on external
standards and on industry experience upon which its described structure, methodology,
processes and documents suite (plans, templates etc.) are based and supported.
The EMS processes themselves will progressively be developed and documented as
engineering policies, procedures, plans, forms and templates that will form part of MTM’s
integrated management system accessible on its Intranet and captured on an engineering
‘Process Compliance Matrix’ (PCM). When referenced within this methodology, the titles of
those procedures are ‘italised’ (e.g. Project Management Procedure). This methodology aims
to provide the context within which to read and enact those procedures (current and to be
developed/issued).
An overview of the various components of the proposed Engineering Management System
(EMS) is provided at Figure 4 below.
These components of the EMS are the basic building blocks that should be available to be
used for managing any generic project comprising engineering effort.
A tailored approach will be established to meet the specific engineering needs of each project.
This basic engineering approach does not change, only the breadth and extent to which it is
employed, based on the complexity of the project or renewal works as well as its technical
scope, contractual obligations and conditions to be met for particular projects and/or
Infrastructure works.
Another key activity associated with the EMS methodology and phases described in this
Methodology is the Performance Measurement Baseline (PMB) that is established to bound
the scope, schedule and budget for an MTM project. For engineering, a key aspect of the
PMB is the technical/engineering scope that describes the engineering activity to be
performed for the particular project as well the scheduling, resourcing and milestones that are
incorporated within the PMB against which engineering can plan and implement its activities
and tasks.
EMS METHODOLOGY
14 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Implement in Stages (Design, Development,
and Construction) Project Start Up
Project Closure &
Handover
PROJECT MANAGEMENT (proposed PRINCE2 based)
Systems Engineering
Control
System Architectural
Design
System Requirements
Analysis
System Integration And Test
SYSTEMS ENGINEERING ( IEEE1220 & ISO15288)
System Qualification
Test
Project Initiation & Planning
Software Detailed Design
Software Architectural
Design
Software Requirements
Analysis
Software Integration And Test
Software Qualification
Test
Software Code
And Test
CONFIG MGT
SUPPORT PROCESS PHASE BASED PROCESS
Asset Management Planning (RAMS, M&S planning / training and facilities)
Commissioning
& Deployment Analysis
Cost and Schedule - Performance Management Control
Project Executive/Project Steering Committee
Nomination
Activity
LIFECYCLE
Maintenance & Support
Process Compliance
Matrix
Project Management Plan
Engineering Plans: SEMP (with SDP,CMP, ITP etc.)
Project HSEQ Plan
PROJECT PLANS PROCESS CONTROL
Project Authorisation
Matrix
AUTHORISATION
SEE ALSO
Standards Work Instructions Guidelines Forms Templates
Local Process
Instructions
SOFTWARE ENGINEERING ( ISO12207)
Configuration Management
Baseline Management
Change Management
Formal (External) Tech Review
Software Configuration Management
Configuration Control Board
Configuration Audits
Peer
Review
Risk Management
Engineering
Change
Waivers
Figure 4: EMS Overview
EMS METHODOLOGY
15 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1.6 Systems Engineering – Acquisition, Development & Operation
1.6.1 Systems Engineering Phases
Figure 5 below depicts three types of distinct systems engineering phases viz. system
acquisition, system development and system operation. MTM’s role in these three phases may
vary depending on the engineering and management activity e.g. MTM will usually always be
an acquirer and user of software engineered solutions but not the developer of this software.
The system acquisition process is undertaken on behalf of all stakeholders in the product
system, in accordance with the commercial policy and practices of MTM. It thus has
paramount responsibility for achieving an operational solution that satisfies MTM User needs.
This is accomplished through its interaction with the system development process.
Sequential lifecycle
Userrequirements
definition
Systemrequirements
definition
Architecturaldesign
COMPONENTDEVELOPMENT
Integration& verification
Installation& validation
Review
Review
Test
Acceptancetests
Systemtest
Change
Change
Change
a8
ReviewSYSTEM
DEVELOPMENT
SYSTEMOPERATION &
SUPPORT
SYSTEM
ACQUISITION
Change
Change
Figure 5: Systems Engineering Phases 1
1 Figures 2 and 3 are based on Business Management System reference model
EMS METHODOLOGY
16 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1.6.2 Systems Engineering Management
The model in Figure 6 below depicts, at a high level, the process relationship between
systems acquisition and systems development phases, with interrelated process flows shown.
One of the key process flows depicted occurs between architectural design and user
requirements as part of the ‘triangle’ relationship linking User requirements definition,
system requirements definition and architectural design. Once this activity has evolved to
a level acceptable to the User (usually MTM e.g. NAM), requirements can be allocated to
“component development” i.e. hardware, software, interfaces and associated documentation.
This flow carries information about the proposed architectural design(s) and the extent to
which a design is capable of satisfying the User requirements down to the component level.
It is important to note that requirements need to be unique, complete, unambiguous,
implementable, verifiable and consistent with all other requirements (and remain
consistent through the life of the project (as requirements may change) through requirements
traceability).
A useful rule to remember is that requirements should be ‘SMART’ i.e. Specific, Measurable,
Achievable, Relevant (clearly linked to objectives) and Time-based (linked to schedule and
ability to track progress).
Once agreement has been achieved (between system developers and acquirers), it is possible
to proceed. At this point components are developed/built (including COTS) and supplied for
integration and verification, leading to a configured system which can be installed and
validated against the User’s need, and then accepted into operation.
This forms the essence of the 3 tiered EMS model constituting the engineering management
system covering, and linking, the project management, systems engineering and software
engineering tiers as outlined in this EMS Methodology (as depicted in Figure 4).
Basic system creation lifecycle
Userrequirements
definition
Componentdevelopment
Installation &validation
a2
Proposedcharacteristics
Proposedcharacteristics
Allocatedrequirements
System
acquisition
process
System
development
process
Component
development
process
Systemrequirements
definition
Architecturaldesign
Integration &verification
Figure 6: Systems Acquisition vs System Development Phases
EMS METHODOLOGY
17 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1.7 Components of the Engineering Management System - General
The various components of the EMS as depicted in Figure 4 are outlined in this Section.
1.7.1 Project Plans
Each project should be planned from three management perspectives: Project Management,
Engineering/Technical Management and HSEQ Management. This helps ensure that each of
these aspects is thoroughly considered and integrated, and the separate needs are not confused
or overlooked. This is enacted by nominating a separate Project Manager (PM), Project
Engineering Manager (PEM) and HSEQ Manager for each project. Refer to Appendix 1 for
an outline of the purpose and key responsibilities of the PEM role.
On the smallest of projects, the PM and PEM roles might be merged but the HSEQ role
should always be performed independently. This results in the following suite of management
plans representing the means by which each of these responsibilities has been planned to be
enacted on each project:
Project Management Plan (PMP);
Systems Engineering Management Plan (SEMP) for development/construction projects.
NOTE: The SEMP should be developed for projects particularly where there is complex
engineering development or where the engineering scope includes systems and software
engineering that may require appropriate management of the interfaces between the
systems and software engineering activities and resulting solutions and where safety risk
is deemed to be high; and
Project HSEQ Plan.
On smaller projects, the planned approach for all aspects of the engineering scope can be
outlined in the SEMP or have the engineering covered in a broader PMP. On larger projects,
the SEMP will usually be supplemented by other supporting plans (stand-alone of
incorporated in the SEMP) that provide a greater breadth and depth of planning for various
major aspects of the engineering task. For example, the following plans are typical in larger
projects involving development and acquisition of systems/software solutions:
Systems Engineering Management Plan (SEMP)
Software Development Plan (usually created by the actual software Developer/Contractor)
Configuration Management Plan (CMP)
Integration and Test Plan (ITP)
Test & Commissioning Plan (T&C Plan)
Health, Safety, Environmental & Quality (HSEQ) Plan
EMS METHODOLOGY
18 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1.7.2 Authorisation
To ensure efficient and effective engineering management of projects, it is necessary to
establish an integrated team environment within which all project and supporting
departmental and Client staff comprehend their individual engineering responsibilities and
authority in relation to project execution. It is planned that Engineering will address this need
by developing and promulgating a Project Authorisation Matrix (PAM) early in the project.
This matrix includes all key engineering organisational roles associated with the project and
identifies the key project management team appointments and their approved delegates. The
key financial, contractual, managerial, technical and process related documents are listed and
cross referenced in the PAM so that it is clear as to who is responsible and authorised to
create, review and approve each listed document.
1.7.3 Process Control
1.7.3.1 Engineering ‘Process Compliance Matrix’ (PCM)
The process assets available to be employed on new projects will continue to increase in
number and mature as MTM addresses a variety of Client needs, across different domains and
technologies as part of its M&R and project activities. A core suite of engineering processes
(“Project Phase Based Process” and “Support Processes”) is depicted in Figure 4 and further
outlined below. These processes can be described in a suite of generic procedures that can be
employed on any project (e.g. Configuration Control Board (CCB) Procedure). The specific
needs of, or differences in, various projects may also generate a variety of underlying project
specific procedures, work instructions, standards etc., that can be re-used on other projects
with similar domain, technology and related process needs in the future.
Engineering Systems may propose MTM manages the tailoring of available process, and the
universal visibility of the employment of process on its various projects, via an engineering
Process Compliance Matrix (PCM). The PCM will list all available engineering process
documentation in MTM, whether generic or project specific, and indicate applicability by
MTM’s functional departments and current projects. This would provide the following:
The standardisation of generically applicable process;
A means to authorise departure from standard process to meet project specific needs;
The avoidance of redefining or recreating processes where not warranted;
A means to ensure that project-specific process developed for individual projects is
disseminated back into the wider MTM business for potential re-use; and
A means to continually improve engineering processes without affecting active projects
by allowing them to continue to work to an earlier version of a process document when
warranted and approved by the process document’s sponsor. This would be reflected in
the applicable project plan.
A simple example of a PCM is depicted in Figure 7 below. The example shows how each
listed process document is mapped to MTM’s departments and the projects against which
they are to be compliant. It also shows how the situation is managed where one project
(PROJ1) has developed a project specific version of the “PROJ1 Software Engineering
Procedure” whereas the other projects utilise a generic version.
EMS METHODOLOGY
19 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Ident DOCUMENT Rev INFRA1 INFRA2 INFRA3 INFRA4 INFRA5 PROJ1 PROJ2 PROJ3 PROJ4
ENGINEERING DEPARTMENT
SYSTEMS ENGINEERING EMS Methodology x x x x x x
SOFTWARE ENGINEERING Software Engineering Policy x x x PROJ1 Software Engineering Procedure x
Testing & Commissioning
Integration & Test Procedure x Problem Reporting x x x x
Design
Design Review x x x x x x x x
Configuration Management
CCB Procedure x x x x x x x x
ENG ‘PROCESS COMPLIANCE MATRIX’
Figure 7 Simple representation of MTM Process Compliance Matrix
EMS METHODOLOGY
20 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1.7.3.2 Project Process Instructions
While standardisation and control of processes are a key influencing factor in the maturity of
MTM’s capability, it is also recognised that each project may differ in some way from
another. The differences in size, complexity, tools, technology, etc., will likely lead to the
need to establish local, detailed project instructions, affecting only that project and its
personnel.
The development of local, standard engineering instructions on MTM projects will be
encouraged but will ensure that they are properly authorised and well promulgated by having
them captured in a single, universally accessible location. The project management team
controls the contents and would be involved in authorising its applicability.
1.7.4 Prime Phase Based Process
1.7.4.1 Project Management
All projects need to be planned, initiated, executed, commissioned, completed (“practical
completion”) and handed over to its operational/asset management phase. That is, they need
to be project managed such that each management phase is performed in a timely, controlled,
coordinated and thorough manner. The project management process (covering planning,
development and delivery) employed at MTM is described in L1-PRJ-PRO-001 and L1-PRJ-
PRO-002. From an EMS perspective, these procedures cover both the project ‘planning and
development’ phase – where FOR, FIS, Project Brief and finalisation of the Business Case
that seeks funding for the project are established – and the project ‘delivery’ phase – where
the project is actually implemented.
With respect to project ‘delivery’ (i.e. project implementation/execution), the aforementioned
2 procedures are useful in describing and summarising the sequences, tasks, documents,
deliverables and aspects to be addressed; however, they are not tightly integrated nor
'workflow' based that one could expect in the form of a project management methodology (or
“PMM”) such as provided by a contemporary methodology, such as with PRINCE2.
Essentially, a PMM is designed to describe the project management lifecycle phases and
elements, and interlink concurrent, different business functions, particularly engineering
(systems/software) during particular project implementation phases for effective management
and control.
This EMS Methodology essentially adopts a PMM perspective via a 3-tiered view of project
management, systems engineering and software (acquisition/governance) engineering
showing generic (albeit contemporary practice) project and engineering lifecycle phases, how
they interact, their interdependencies, and with some relevance to current MTM arrangements
(that will broaden and improve in relevance over time).
Elements of the EMS Methodology are being initiated before other elements will/should, such
as engineering reviews (SRR, CDR and TRR) that have been integrated into the existing
MTM Works Readiness Review (“T- n”) program in collaboration with NPD. Like a PMM,
this EMS Methodology particularly focuses on the MTM project ‘Delivery’ phase.
EMS METHODOLOGY
21 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
This systems engineering approach utilises a methodology supported by current multi-
industry used PM methods (such as ‘PRINCE2’). PRINCE2 emphasises a structured PMM
approach by dividing the project into manageable, interlinked and controllable phases. The
phases are scheduled and budgets are established for each associated element of work via
engineering ‘Work Packages’ covering engineering activities (refer to Engineering WBS
Guideline).
On all but the largest projects, the PM cost/schedule control implementation can be scaled
down to meet the basic and specific cost and schedule control needs of the project. This
provides for a consistent ‘earned value’ measurement and reporting approach across all
projects, regardless of project size/complexity/scale and scaled to practical levels for MTM’s
projects.
NOTE: For proper planning and execution, the EMS needs to be phase-based and driven,
with Entry and Exit gates, to and from these project/engineering phases and not a Gates
driven approach.
Once planned and under way, the project’s progress status is given to, and oversighted by, a
Project Steering Committee (‘Project Executive’) to whom the PM reports. Monthly (TBC)
project status meetings are held for all projects, the PM reporting overall status and any
exceptions to the agreed planning baseline. The following minimum management measures
should be incorporated within the monthly project status reports:
Cost and schedule variance and trends (i.e. performance and actual versus plan)
Identification of the source of variance(s)
Identified milestones (contractual and technical) and their scheduled and actual
achievement
Critical path identification (should include any Contractor related critical milestones)
Completion status of all engineering phases
Project risk management activity
Problem report (“SPR”) handling metrics
Staffing/resourcing profile.
Each Project Manager should be required to document their planned approach in a Project
Management Plan.
1.7.4.2 Systems Engineering
1.7.4.2.1 Systems Engineering Phases
Where the scope of a project encompasses the engineering (or re-engineering) of the system
itself, a methodical systems engineering approach is proposed for MTM. The systems
engineering phases (per this EMS Methodology and established systems engineering
procedures) are consistent with the development process of ISO/IEC15288 with the addition
of a Systems Engineering Control phase which covers the activities performed following
architectural design and prior to integration and test of the system (e.g. specification and
EMS METHODOLOGY
22 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
design control, performance modelling and monitoring, integration and test planning, etc.).
These processes are supported by the use of IEEE 1220/ISO26702 as a companion standard
that provides more detailed planning guidance in the implementation of the various systems
engineering activities. Essentially, ISO15288 takes a broad and general view of the entire
lifecycle of the system whereas IEEE1220 focuses on the development of a system, including
planning.
Each project PEM will be required to document the planned systems engineering approach in
a SEMP which is supported by lower level and more detailed plans when warranted. The
PEM will also be responsible for the development and management of the Engineering WBS.
1.7.4.2.2 Asset Management
The asset management aspects related to a project or AMP planned renewal are planned and
performed as required within the systems engineering phases. This may encompass RAMS
activity (planning for the required Reliability, Availability, Maintainability and Safety of the
system) addressing requirements of EN 50126 and all aspects of training related to the
operation and maintenance of the system. The asset management discipline aspects of
renewals work (e.g. structures, signal, electrical, facilities, track etc) may also be covered by
this EMS approach, as applicable.
1.7.4.3 Software Engineering
MTM is an ‘acquirer’ of software solutions and does not develop software. However, it
should employ established, industry software engineering practices, linked to the system
engineering lifecycle approach as outlined in this methodology, for its embedded/integrated
systems acquisition, governance and management. This is particularly vital for the acquisition
of safety critical software solutions (SIL 3 and SIL 4) that are often part of
products/technology acquired by MTM as part of signals engineering elements.
Where the scope of a project encompasses the acquisition and 3rd
Party development (or re-
development, amendment, maintenance, etc.) of software, a methodical software engineering
approach must be employed by MTM. The software engineering phases will be consistent
with the development process described in ISO/IEC12207.
PEMs are required to document the planned software engineering approach in the SEMP
unless a separate Software Development Plan (SDP) is warranted (or mandated by the Client).
For MTM projects, the SDP would usually be produced, and submitted to MTM for review
and approval, by the Developer (contractor) of the software solutions.
1.7.5 Support Processes
The prime phase-based process is supported by other related support processes such as
configuration management, peer review, baseline management and Enterprise risk
management procedures (refer to Figure 4). Also to be provided are associated supporting
standards, work instructions, forms, templates and guidelines.
EMS METHODOLOGY
23 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1.7.5.1 Configuration Management
A suite of CM procedures are required to outline the necessary steps for ensuring that all of
the engineering work is performed in a controlled manner, to a known specification derived
from baseline requirements, and that all final products are verified through independent
review and audits (functional (FCA) and physical (PCA)) to have met that specification. This
encompasses the principles of data management, version control, baseline management,
configuration control and change management. Configuration Control Boards (CCBs) are a
highly effective mechanism on projects to manage engineering baseline through proper
configuration control and should be established on complex engineering projects (particularly
those that are safety critical such involving design/modification of SIL3 or SIL4 signalling
systems) to formally establish and then control the configuration and content of the various
engineering baselines (Functional, Allocated, Design and Product).
Each project PEM would be required to document the planned configuration management
approach in the SEMP unless a separate Configuration Management Plan is warranted, such
as large, complex projects, or mandated by the Client.
1.7.5.2 Other Processes
Visibility of all process assets available at MTM may be provided to MTM project staff via
the PCM. This includes all supporting procedures, standards, work instructions, forms and
templates related to project work. Examples include other engineering life cycle processes
(e.g. Enterprise Risk Management and Peer Review procedures) and HR management (e.g.
timesheets and leave forms). In time, key elements of an ‘Earned Value’ approach to project
management such as cost/schedule variances and work package planning/management could
add rigour to the management of MTM’s projects and systems engineering.
EMS METHODOLOGY
24 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1.8 Key Concepts of the Engineering Management System
This Section provides further information about the key concepts of the EMS. Each
sub-section provides narrative related to an associated figure depicting the key concepts being
described.
1.8.1 The Scope of an Engineering Project
Figure 8 depicts the scope of an engineering project as required to be estimated, scheduled,
funded and accordingly included within the engineering WBS. Refer to Engineering WBS
Guideline. The scope includes the following activity:
Pre-planning by the designated project management team (the Start-Up Phase) to ensure
that the resources (staff and equipment) required to fully plan the project are well
considered and that the scope of the project is fully understood and agreed with the Client
prior to committing further resources.
Comprehensive project planning by the project management team and key support staff
(the Initiation Phase) which will finalise the project’s management plans and establish a
‘performance measurement baseline’ (PMB) while the necessary staff are being gathered
for the technical implementation activity. The output of this activity is a fully planned and
documented project, with work packages and associated input material established against
which allocated staff can commence their assigned responsibilities.
The technical implementation itself (i.e. the systems and software engineering activity) as
required to develop the products to be delivered to the MTM User or Client in full
satisfaction of the project scope. This activity commences as soon as there is sufficient
planning to support it and as staff become available, and completes when the MTM User
or Client accepts the installed products.
Post-acceptance activity (the Close-down Phase) required to finalise any outstanding
anomalies that were identified during acceptance testing and to perform all necessary
‘clean-up’ including the positioning of staff, equipment and records and the creation of
final reports (e.g. metrics, lessons learnt, etc.). Budget is allocated and resources identified
as required to support contractual warranty and latent defect provisions. The financial
records are finalised and the project formally closed.
Provision for support during the contractual warranty and latent defect period (the
“Warranty” phase).
With MTM assuming post delivery maintenance (i.e. operate, maintain, service, etc.), this
aspect may be treated as a separate Maintenance and Support project in its own right. That is,
it will be separately staffed, planned and executed to avoid any potential conflict in resource
with the development project, ensuring a smooth, well planned transition into operation.
EMS METHODOLOGY
25 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Implement Maintenance and Support Project
Systems Engineering Control
System Architect'al
Design System Reqts
Analysis System
Integration And Test
System Qualification
Test
Software Detailed Design
Software Integration And Test
SOFTWARE ENGINEERING PHASES Software
Qualification Test
Software Code
And Test
Deployment SYSTEMS ENGINEERING PHASES
PROJECT MANAGEMENT STAGES
ENGINEERING MANAGEMENT SYSTEM
Project Initiation
Project Closure
Software Reqts
Analysis Software
Architect'al Design
Warranty and LD Provisions
Acceptance of Installed
System End of
Contract Responsibility
Project Management
Team Appointed
Resources allocated
to support Project
Planning
Project is Fully
Planned. Resources allocated to implement
Client Acceptance
Gained. Resources retained to
close anomalies.
Practical Project Completion
Satisfied. Acceptance anomalies
closed.
Contractual Limit of
Warranty & Latent Defect
Liability Reached
Scope of the Engineering development project to be estimated and included in the WBS
THE SCOPE OF AN ‘ENGINEERING’ PROJECT
STARTUP STAGE
INITIATION STAGE IMPLEMENTATION STAGE(S)
CLOSE DOWN STAGE
WARRANTY STAGE
Some contracts include post acceptance activity to provide maintenance, support, servicing or the like. This in-service component may be managed as a separate ‘engineering project’
Project Start Up Implement in Stages
Startup M&S Project
Initiate M&S Project
Close M&S
Project
Performance Measurement
Baseline (PMB) In Place
Contract Being
Finalised
Contract Finalised
Figure 8: Scope of an ‘Engineering’ Project
EMS METHODOLOGY
26 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1.8.2 Key Components
Figure 9 depicts the key components of the EMS as follows:
A ‘Project Executive’ (comprising one or more appointees dependent on the complexity,
scale, risk etc. of the project), responsible to the Chief Executive Officer for providing
executive management oversight to ensure that both the Client and company expectations
are being realised NOTE: PTV may mandate the establishment of a “Project Steering
Committee’ per Clause 4.2 of the Projects Agreement.
A Project Manager role, responsible to the Project Executive for the execution of the
project, providing day to day management of project resource, scope, budget and schedule
A Project Engineering Manager role, responsible to the Project Manager for control of the
technical/engineering solution, ensuring that the product being developed is fit for purpose
and meets the full scope of the Client’s technical requirements, SOW, specification(s) etc.
Thorough planning through the Start-Up and Initiation Phases to achieve the necessary
authorisations, project plans and process identification to meet the immediate needs of the
project in its early implementation activity. This includes control of project scope through
Contract Baseline establishment, and establishing a budget and schedule (Master and
Intermediate level schedules as required).
As required, implementation of some aspects of an ‘earned value’ approach to
performance measurement with periodic cost and schedule status to the Project Executive.
This incorporates the development of (possible Planning Packages) and/or Work Packages
used to authorise and status current and near term activity through which the forward
budget and work scope is planned and distributed. On larger projects, these elements of
work may be grouped under multiple Cost Accounts which are managed by one or more
Cost Account Managers (CAM). This structured approach to managing the project
ensures that work is performed in the planned order and that sufficient budget and
schedule is allocated to all activities. As a result, the potential effects of project risks, the
impact of delays and the true status of the project can be managed effectively.
Thoroughness in the comprehension and agreement of project scope with the Client
through the development of a Functional Baseline specifying all technical system
functionality, performance, interfacing and acceptance requirements.
Thoroughness in the development of a complete system solution through the creation of
an Allocated Baseline for all parts of the system that are to be separately developed and
integrated, with strong change management and traceability against the Functional
Baseline.
Thoroughness in the control of developed system components through the establishment
of Product Baseline, ensuring that they are comprehensively identified, verified against
the specification and fully documented in support of operations and ongoing maintenance
activity.
EMS METHODOLOGY
27 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
PP PP PP PP PP WP WP WP WP Performance Measurement
Baseline
Systems Engineering Control
System Architect'al
Design System Reqts
Analysis System
Integration And Test
System Qualification
Test
Software Detailed Design
Software Integration And Test SOFTWARE ENGINEERING
PHASES Software
Qualification Test
Software Code
And Test
Deployment
Project Executive / Project Steering Committee
Process Compliance
Matrix Project Mgt Plan
Project HSQE
Plan
PROJECT
PLANS PROCESS Project
Authorisation Matrix
AUTHORISATION
Periodic Cost Collection against Open Work Packages
SYSTEMS ENGINEERING
PHASES
PROJECT MANAGEMENT STAGES
ENGINEERING MANAGEMENT SYSTEM
EXECUTIVE
MANAGEMENT
PERFORMANCE MEASUREMENT
Project Initiation Project
Closure
Software Reqts
Analysis Software
Architect'al Design
Warranty and LD Provisions
KEY COMPONENTS
PROJECT MANAGER
PROJECT ENGINEERING MANAGER
PROJECT EXECUTIVE
Functional Baseline
Allocated Baseline
Product Baseline
Product Baseline
Cost and Schedule-Performance Management Control System
Project Start Up
Project Engineering Management
Plan
Implement in Stages
Routine Project Status Reporting
WP WP WP PP PP PP PP PP PP PP Planning Packages for future work - converted to Work Packages in a rolling wave as the planning details become firmer.
Contract Baseline
Figure 9: EMS Key Components
EMS METHODOLOGY
28 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1.8.3 Project Management Phases
Figure 10 depicts the key project management phases described below.
1.8.3.1 Start-Up Phase
An engineering development project is commenced as soon as a mandate for the project is
established. A request to commence project activity is made by the designated PM by
applying to the nominated Project Executive for interim funding ahead of a secure mandate
(via a Project Commencement Checklist). For external (contracted) projects, this coincides
with advice that an MTM tender has been approved finalising of contract negotiations.
A project management team is appointed as required to perform the Start-Up Phase activities.
This will always include the designated Project Manager and Project Engineering Manager
such that they can be involved in finalising the project requirements with the Client (e.g.
contract or scope negotiations).
A thorough examination of the Client requirements is performed (contract review) prior to
agreement to ensure that they are fully comprehended, consistent and achievable within the
resources available to MTM. During the Start-Up Phase the project management team will
secure the project mandate (e.g. finalise contract or scope negotiations) and determine, based
on the scope of the project, the resources and schedule required to perform thorough planning.
On larger projects, additional staff may be required to support them.
1.8.3.2 Initiation & Planning Phase
The Initiation Phase is commenced after a Contract has been signed (or similar project
commitment has been provided). The Contract Baseline is established ensuring that the
following associated aspects are fully identified and managed from the outset:
The Contract itself, including all amendments and change proposals
All Client furnished items to be provided
All documents referenced by the Contract by exact issue, as applicable to the work scope
All identified obligations under the contract.
The project is comprehensively planned to completion ahead of commencement of the initial
technical activity to ensure that all effort is applied effectively. This includes the following:
Minimum 3 management plans (Project, HSEQ and Engineering)
Project Authorisation Matrix (PAM)
Engineering Process Compliance Matrix (PCM)
Preliminary product (or ‘Configuration Item’ (CI)) hierarchy (finalised within the System
Architectural Design phase)
EMS METHODOLOGY
29 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Identification of all types of documents to be developed, including their inter-relationships
(the document tree).
This Phase also results in establishment of a PMB incorporating the following items:
Engineering WBS and associated dictionary (Engineering WBS is a key, primary
document supporting EMS implementation – refer to the Engineering WBS Guideline)
Master Project Schedule (MPS), identifying all management phases, technical phases and
their associated major milestones
The Responsibility Assignment Matrix (RAM), allocating scope and budget to Cost
Account Managers (CAMs) for work to be performed and completed
A time phased, distributed budget across the Cost Accounts through a structure of Work
Packages (WPs) and possibly Planning Packages (PPs).
1.8.3.3 Implementation Phase(s) – Design, Development & Construction
The technical products are analysed, architected, designed in detail, developed, integrated,
constructed, tested, qualified, commissioned and deployed. Refer to the Systems and
Software Engineering phases for further detail (sections 1.8.5 and 1.8.6 respectively).
Throughout the Implementation Phase, project risks are routinely assessed and reviewed and
necessary abatement actions determined and executed to minimise impact on the project
schedule, budget and technical solution.
1.8.3.4 Closure & Handover Phase
Following Client acceptance, the project is brought to a close, finalising outstanding
anomalies and positioning all project resources. This includes the following:
Capturing of lessons learnt to feed back into process improvement and corporate
knowledge repositories
Calculation of final project metrics which can be utilised to improve the project estimation
process to the benefit of future project enhancement activity or like project development
Identification and allocation of all follow on action needed
Archiving of all project records for future reference.
1.8.3.5 Warranty Phase
The PM must ensure that provision has been made to support contractual warranty and latent
defect provisions following the practical completion through the period of liability that would
likely cover operational phase. This may include establishing budget and planning for the
means by which suitable resource (staff and development environment) would be made
available to support such activity, should it eventuate.
EMS METHODOLOGY
30 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Archived Records
Follow-on Actions
Project Metrics
Contract Baseline
Systems Engineering Control
System Architect'al
Design System Reqts
Analysis System
Integration And Test
System Qualification
Test
Software Detailed Design
Software Integration And Test
SOFTWARE ENGINEERING PHASES Software
Qualification Test
Software Code
And Test
Deployment
Process Compliance
Matrix Project Mngnt Plan
Project Quality
Plan
PROJECT PLANS PROCESS Project
Authorisation Matrix
AUTHORISATION
SYSTEMS ENGINEERING PHASES
PROJECT MANAGEMENT STAGES
ENGINEERING MANAGEMENT SYSTEM
Project Initiation
Project Closure
Software Reqts
Analysis Software
Architect'al Design
Warranty and LD Provisions
Project Start Up
Doc Tree-P Config Item Hierarchy-P
Implement in Stages
Project Engineering Management
Plan
PROJECT MANAGEMENT STAGES
Referenced Documents
List
Contract Amendments & CCP List
Client Furnished Items List
Contract Obligations
Matrix
Contract Review
Lessons Learnt Report
Performance Measurement Baseline
Work Breakdown Structure
Master Project
Schedule Responsibility Assignment
Matrix Time-Phased Distributed
Budget (Cost Accounts)
Work Packages Planning
Packages
Contract
Figure 10: Key Project Management Phases
EMS METHODOLOGY
31 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1.8.4 Executive Management Oversight
Figure 11 depicts aspects of the Executive Management function which may be exercised at
MTM to oversee each engineering project, routinely monitoring its health based on corporate
and Client expectations and ensure overall strategic compliance.
In this model, a Project Executive (nominally chaired the General Manager – Projects) is
assigned to each project and is responsible to the Chief Executive Officer for the well being of
the project. The Project Executive appoints the key project personnel and approves the
assignment of budget to commence the project planning processes (Project Start-Up) as soon
as a project mandate is expected (e.g. a Contract is expected to be signed) via the Project
Commencement Checklist. Early expenditure on the project is closely monitored to ensure
that there is a balance between the need to perform preparatory work and the risk of
unnecessary effort should the expected project mandate not come to fruition. The main
activity to be performed by the project management team is to finalise the Contract scope
details with the Client and to plan the activities and resources needed to perform thorough
project and technical management planning during the Initiation Phase. Once the mandate has
been achieved (e.g. Contract signed), the Project Executive releases budget and resources to
the PM for this activity via the Project Start-Up Phase Completion Checklist.
The Project Executive ensures that the Initiation Phase results in a thoroughly planned project
with an agreed PMB established to set the bounds of the scope, schedule and budget for the
project. Exception reporting levels are established at which the PM must report to the Project
Executive. The thoroughness of planning and therefore the readiness to move into execution
is checked via the Project Initiation Phase Completion Checklist.
The various project management phases and technical/engineering phases of the project are
then routinely monitored with the Project Executive, chairing a routine (say, monthly) review
meeting at which the PM delivers a Highlight Report to the Project Executive. The Highlight
Report summarises the project from the Project, HSEQ and Technical/Engineering
management perspectives and status’s progress against the PMB. The Project Executive
consists of various MTM senior managers whose responsibilities relate and contribute to the
success of the project. Senior ‘User’ Representatives review the strategic, financial and Client
perspectives knowing that a successful project provides a stepping stone to new and expanded
business opportunities. Senior ‘Supplier’ Representatives ensure that the various support
areas of the Company are providing the required service to the PM to enable project success
(Human Resources, Contracts, staff allocation, environment, process, tools, etc.). Similarly,
senior external representatives may be invited to the project status reviews. At the time that
the project is offered to the Client for acceptance, the PM produces the Project
Implementation Phase Completion Checklist which is used to ensure that all required project
activity is completed prior to releasing staff for other duties.
The Close-Down Phase completes all outstanding activity remaining at acceptance, creates
project records, releases resources and where appropriate, hands over responsibility to a
Maintenance and Support team. A Close-Down Phase Completion Checklist is used so the
Project Executive can ensure that all project activity has been finalised.
EMS METHODOLOGY
32 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Technical Status
Project Executive / Project Steering Committee
Systems Engineering Control
System Architect'al
Design System Reqts
Analysis System
Integration And Test
System Qualification
Test
Software Detailed Design
Software Integration And Test SOFTWARE ENGINEERING PHASES
Software Qualification
Test Software
Code And Test
Deployment
Project Executive / Project Steering Committee
SYSTEMS ENGINEERING PHASES
PROJECT MANAGEMENT STAGES
ENGINEERING MANAGEMENT SYSTEM
EXECUTIVE MANAGEMENT
PERFORMANCE MEASUREMENT
Project Initiation Project
Closure
Software Reqts
Analysis Software
Architect'al Design
Commencement Checklist
SU Stage Completion Checklist
Init Stage Completion Checklist
Impl Stage n Completion Checklist
Closure Stage
Completion Checklist
Impl Stage 2 Completion Checklist
Impl Stage 1 Completion Checklist
Warranty and LD Provisions
PROJECT MANAGER
PROJECT ENGINEERING MANAGER
PROJECT EXECUTIVE
Cost and Schedule-Performance Management Control System
Project Start Up
EXECUTIVE MANAGEMENT OVERSIGHT
Routine Highlight Reports
Project Executive
PTV Rep Senior
Internal User Rep
Senior External
Supplier Rep(s) Senior External
User Rep(s)
By invitation By invitation
Cost and Schedule
Status
Give Ad-hoc direction Advise
Exceptions
Implement in Stages
Project Issues
Figure 11: Management Oversight
EMS METHODOLOGY
33 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1.8.5 Systems Engineering Phases
Figure 12 provides a more detailed view of the systems engineering phases and how they
interact with the software engineering activity. The systems engineering activity on a project
commences as soon as sufficient planning has been completed and staff allocated to the task.
The expected inputs are as follows:
From the Client: the Contract and a documented operational concept
From the Initiation & Planning Phase: an initial CI hierarchy, list of document types to be
developed (document tree) and the SEMP.
This work is managed within distinct Systems Engineering phases as follows:
1.8.5.1 System Requirements Analysis
System requirements comprise:
1. ‘Functional Requirements’ that describe what the system/solution is required
and capable to do including interfacing and environmental aspects;
2. ‘Performance Requirements’ that describe how well the system/solution
performs in measurable terms; and
3. ‘Non-functional Requirements’ (also called ‘ilities’, attributes, characteristics or
quality factors) that describe constraints that affect design solutions.
These requirements are developed during the Development phase by NSD and undergo
review and assessment at the Stakeholders Requirements Assessment (SRA) (refer Figure 2).
SRA participants are selected based on the scope, scale and complexity of the project/works.
A Client’s requirements should be fully analysed resulting in a preliminary Systems
Specification and External Interface Document which is usually presented to, and approved
by, the Client to become the Functional Baseline (established at SRR), that is a complete and
unambiguous definition of the functional (incl. interface), performance and non-functional
requirements of the system.
It is the specification against which the final acceptance/commissioning testing is performed.
A preliminary System Test Plan and the ‘architectural concept’ for the system are developed,
supporting the Functional Baseline’s development and review.
1.8.5.2 System Architectural Design
If required by the System Requirements Review (SRR), the system concept design is
architected (physical/functional system architecture) and presented at the Post Award
Concept Design Gate (CDG) to satisfy the Functional Baseline, struck at successful
completion of the SRR. As a result, the allocation of system requirements to the various
engineering system components is determined and preliminary versions of Hardware,
Interface and Software specifications are then developed for each CI that lead preliminary
design completion becoming the initial Allocated Baseline) achieved at successful
Preliminary Design Review (PDR).
EMS METHODOLOGY
34 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Successful completion of PDR initiates the start of detailed design (hardware and, as
applicable, software), the essential completion of which culminates in a Critical Design
Review (CDR) to confirm detailed design satisfies its system requirements at which time the
Design Baseline is struck.
NOTE: Apart from the formal, design reviews described above as part of the systems
engineering methodology, MTM conducts lower-level design and technical reviews in
accordance with L1-NAM-PRO-002, and its companion procedure for signalling L1-NAM-
PRO-001. These design and technical reviews are conducted by engineering to consider the
design from the MTM Asset Manager’s and Infrastructure Delivery Manager’s perspective, in
particular the impact on the existing inspection and maintenance regime and future operation
and maintenance access requirements. The technical reviews are not a substitute for the
checking and independent review processes that form part of the preparation of signalling
drawings.
The following includes typical approved documents that are finalised as the basis for ongoing
engineering control of the project:
a. Scope of Works
b. Specification(s) (MTM and 3rd
Party (e.g.VicRoads, VicTrack, Yarra Trams)
c. Design Documents (incl. drawings, design architecture (e.g. CBI architecture design))
d. Engineering Configuration and Qualification (MoC, ECs, Type Approvals, Standard
Waivers)
e. Interface Document(s) (MTM system solution related and 3rd
Party (e.g. VLine,
ARTC))
f. Certificates (all MTM technical Disciplines, external Suppliers, NATA, etc.)
g. Safe Working authorisation
h. Test Plan(s) and Report(s) (internal MTM such as T&C Plans, Supplier FAT, other 3rd
Party)
i. CI Hierarchy
j. Document Tree.
1.8.5.3 Systems Engineering Control
While the various software CIs are being developed, continuing systems related activity must
be performed in readiness for back-end integration and test.
The hardware specifications must be finalised in consultation with the Contractor’s (software
developers) as they finalise their software specifications. This ensures that sensible trades are
performed to allocate budget and effort to best address the underlying needs of performance,
throughput, etc. The associated key technical performance measures are determined and
monitored throughout development. All specialty engineering needs are addressed, including
EMS METHODOLOGY
35 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Human Machine Interaction (HMI), safety, security, etc. Design verification activities are
performed where warranted (e.g. simulation, prototyping, modelling, throughput analysis,
etc).
System integration and test planning is performed by the Contractor and system test
procedures prepared against the Functional Baseline with any identified errors, omissions or
inconsistencies addressed and corrected in consultation with MTM (and possibly the Client).
A Configuration Control Board is established within the project to manage this process.
The various hardware components are incrementally tested as they are received from the
suppliers and the complete hardware architecture is built up as necessary to integrate and test
a fully representative system.
The asset management/logistics aspects are addressed. This may include facility planning,
physical layouts, loadings, footprints and cabling plans. The RAMS aspects are reviewed
within the design. Operator and maintainer training and support documentation needs are
addressed.
1.8.5.3.1 SSAF and Safety Management System
The MTM SSAF and SMS are described in L0-SQE-MAN-002. Together, they constitute an
important component of what the EMS is required, and designed to, deliver as part of its
engineering solution(s) in M&R and Projects. This is particularly the case with integral rail
safety applications that may present as engineering complexity and high OHS risk such as
with signal engineering and Electrical HV aspects.
As such, the EMS must address and include safety as part of system requirements, risk
assessment and management, specifications, design, fabrication, configurations and
qualification, testing, installation, integration, reviews and commissioning throughout its
engineering lifecycle.
1.8.5.4 System Fabrication, Qualification and Installation Testing
Within this phase, the system is built from the bottom up, integrating its various pre-tested
components (CIs). The CM of the system enters a formal phase (previously having been
under developmental CM) to ensure that the exact configuration under test is known at all
times. Depending on the size and complexity of the system, this integration might occur
through a hierarchy of subsystem layers, each requiring testing in its own right. In each such
layer, the basic architecture is first established and tested prior to exercising and testing the
related operational functionality. This involves testing the hardware interconnection, the
operating system and other associated firmware, and any middleware (developed or purchased
software) upon which the various software CIs rely for inter-process communication,
exception handling, etc.
The previously developed system test procedures are now exercised and finalised in
preparation for qualification testing. Test Readiness Review (TRR) reports are prepared and
kept as a record in support of system qualification. Refer to section 1.8.9 for a description of
Engineering Reviews, including TRR.
EMS METHODOLOGY
36 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1.8.5.5 System Integration and Testing
Within this phase product, installation, integration, factory acceptance testing is undertaken,
possibly in the Client’s presence, to prove that all of the requirements in the Functional
Baseline have been met. Again, depending on the size of the system, this may need to be
performed in an incremental manner, from the bottom up through the various layers (e.g.
subsystems) of the total system. The details of the configuration under test are recorded and
the test results documented in a System Test Report.
Any anomalies found during testing are registered, categorised, prioritised in regard to their
urgency, and passed back to the Developers for correction. As a result, corrected systems will
again be presented for testing to close out the anomalies, and regression testing will be
performed as necessary.
The functional and physical completeness of the various system components (e.g.
subsystems) that have been subjected to test are independently audited to ensure they meet all
of their intended functionality and are comprehensively documented. The resulting Product
Baselines are formally established by the CCB and very carefully controlled and managed
from that point.
1.8.5.6 Testing and Commissioning (T&C)
Broadly, the objective of T&C of new or modified systems is to ensure that before the system
(solution) is used for the control and conduct of train operations, the system solution is
validated as satisfying all specified requirements, including safe working standards and
applicable specific requirements such as operational (such as FOR), functional and
performance requirements.
As an engineering risk reduction strategy, T&C should be scoped to only address those
requirements that are necessary to be validated in the MTM (‘user’) operating environment
(i.e. during occupation) to achieve commissioning objectives, with other testing (such as
factory acceptance testing, integration testing and majority of correlation and principles
testing) to be conducted prior to the T&C activity.
Consequently, as part of up-front project and engineering planning, and verified at the TRR,
the overall testing strategy should aim to verify as many of the system requirements during
the project lifecycle as possible, with only the minimum, mandatory tests to be conducted at
T&C in order for it to become more a ‘high confidence’ testing activity, achieved within the
allocated test (time) budget during occupation.
Standard T&C Plan templates (by Discipline) are being developed to get improved
consistency in describing scoping and planning these T&C activities across projects.
1.8.5.7 Deployment
The various proven components of the system are deployed to MTM operational railway
network sites and installed in accordance with their Product Baselines. Representative testing
is performed to verify that the qualified items are functioning/operating/performing as
required and expected.
EMS METHODOLOGY
37 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
KEY:
P = Preliminary F = Final
Doc Tree-F Config Item Hierarchy-F
HW Spec-P SysHW Arch SSDD
SysTest Plan-F Arch Concept
SysTest Plan-P Ext I/FD -F
Developmental CM Formal CM
Systems Engineering Control
System Architect'al
Design System Reqts
Analysis System
Integration And Test
System Qualification
Test
Software Detailed Design
Software Integration And Test
SOFTWARE ENGINEERING PHASES Software
Qualification Test
Software Code
And Test
Deployment
Project Engineering Management
Plan
SYSTEMS ENGINEERING PHASES
PROJECT MANAGEMENT STAGES
ENGINEERING MANAGEMENT SYSTEM
Project Initiation
Project Closure
Software Reqts
Analysis Software
Architect'al Design
Project Start Up
IRS -P SRS - P
Qualified CI
Software prototypes
SYSTEMS ENGINEERING PHASES
Ext I/FD-P
Operational Concept
Document
Contract (Ts&Cs SOW)
System Spec-F
Tested Hardware
CIs
Functional Baseline Control
System I&T Plan
System Test
Procedures
HW Spec-F
Specialty Engineering Docs as rqd
PCA Report FCA Report
Sys Config Sys Test Rpt
TRR Reports
Design Verification Docs as rqd
System Spec-P
Logistics, Maint & Spt Docs as rqd
HW Test Reports
Doc Tree-P Config Item Hierarchy-P
Hardware procurement
Implement in Stages
Figure 12: Systems Engineering Phases & software interaction
EMS METHODOLOGY
38 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1.8.6 Software Engineering Phases
MTM is an acquirer of software solutions (software, firmware and associated documentation)
as part of products (systems), not a software ‘developer’. Nonetheless, the principles applied
by MTM, albeit from a software management, control and governance perspective, in
acquiring software solutions are the same as should be applied by developers themselves to
produce software solutions (incl. requirements development and management, integration
testing, configuration management, baseline management, reviews and delivery etc.),
particularly for safety critical solutions. MTM should review, approve and audit the
Contractor(s) software engineering management system and processes early to ensure
satisfactory methods, processes and controls are applied throughout the project’s life.
Figure 13 provides a more detailed view of the software engineering phases and how they
interact with the systems engineering activity. The software engineering activity on a project
commences as soon as sufficient systems architectural design has been completed and staff
allocated to the task. On some projects, early software prototyping might be performed to
assist in deciding the systems architecture. On other projects, the developer may be acting as a
software subcontractor to the prime contractor, in which case software engineering activity
will commence with the delivery of the specification against which to develop. Typical inputs
to the Software Engineering activity are as follows:
a. Preliminary Software Requirements Specification (SRS)
b. Preliminary Interface Requirements Specification (IRS).
When acting as a software subcontractor, the prime contractor may refer to these
specifications as final. Even so, it is expected that a requirements analysis activity will be
performed and specifications updated by agreement to ensure that they are fully
comprehended and that the values of uniqueness, completeness, consistency and testability are
present such that efficient development and qualification is possible. For safety critical
software (e.g. signalling), where SIL3 or SIL4 may apply, EN50128 should be used in
conjunction with this EMS Methodology for the development of embedded (i.e. firmware
based) or other software solutions acquired by MTM for integration within its railway asset
base.
Unless otherwise specified by the Client, the software documentation may be based on the
MTM cited Standards and particularly ISO12207, tailored as appropriate to the project.
This work is usually managed within distinct Software Engineering Phases as follows:
1.8.6.1 Software Requirements Analysis
The allocated requirements are fully analysed resulting in final Software Requirements and
Interface Requirement Specifications across all software CIs, which may require approval by
the MTM/the Client to become the agreed Allocated Baselines, being complete and
unambiguous definitions of the functional, performance and interface requirements of each
software CI. These are the specifications against which the software qualification testing is
later performed. A preliminary Software Test Plan is created supporting the Allocated
Baseline development and review.
EMS METHODOLOGY
39 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1.8.6.2 Software Architectural Design
The software system is fully architected to identify the generic environment within which all
CSCIs (or standalone software program/items) will be developed. This activity may have
commenced within the System Architectural Design phase and must be completed at this
time. The operating system, related system software services and utilities, language, inter-
process communication layer, exception handling mechanisms, and any related design
constraints will be identified, documented and disseminated to the various software CI
development teams.
Each CSCI is ‘laid out’ within that architecture and documented in a preliminary Software
Design Description and associated Interface Design Description. The related physical
hardware on which the software is to reside is identified and confirmed within the system
architectural design. As a result, the allocation of functionality to the various software layers
and components is determined, the preliminary design being described using either a
functional (e.g. structured analysis and design) or object-oriented methodology.
Where user interfaces are required to be developed, any firmware to be utilised is identified,
the ‘style’ associated with the screen layouts is determined, and the number and utility of the
expected presentations defined. MTM/Client involvement in this HMI work is paramount to
ensure that a solid foundation for design is agreed.
The Software Test Plan is finalised and any associated Test Support Items (TSIs) needed –
such as software Drivers, Stubs and test harnesses - to be developed are identified.
Consideration of the source of required input test data is made as is the means by which
output test data can be verified (analysis, comparison, inspection, etc.). TSIs themselves will
likely need to be under formal configuration management by the Developer.
1.8.6.3 Software Detailed Design
The individual software components (e.g. tasks, routines, classes, objects, etc.) are designed in
detail and documented in final versions of the Software Design and Interface Design
Descriptions.
The preliminary version of the Software Test Description (STD) is generated with a
determination of the types of test cases required to qualify the software against the Allocated
Baseline. The specification of any associated TSIs is finalised.
Where HMI is involved, say where output is displayed to an Operator, the detailed design of
each of the screens to be presented to the operator is determined, with all input choices,
bounds, ranges, responses and the like defined. Again, significant MTM/Client interaction is
sought to ensure that the operational aspects of the system are well founded.
1.8.6.4 Software Code and Test (Unit Test)
Post coding, unit testing is one of the key, highly important activities in software engineering.
The various software components (e.g. tasks, routines, classes, objects, etc.) and the TSIs are
coded, de-bugged and unit tested individually. A key document that should be established by
the Developer is a software Coding Standard that is used as part of the software design
process. Developmental control is exercised by the programmers, and supported by integrated
software tools, to perform the software CM of these components. Where possible, integrated
test management tools are utilised to provide for automated ‘regression’ testing (that ensures
no impact on desired functionality). Records of testing should be held in associated test logs.
EMS METHODOLOGY
40 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Walkthroughs or peer reviews (another key activity) of the software source/object code and
documentation for this software development phase are highly recommended. However, this
should be mandated by MTM for any safety critical software solutions (SIL3 or SIL4).
An updated version of the STD should be generated, documenting the details of the
procedures to be followed during subsequent qualification testing against the CSCI.
1.8.6.5 Software Integration and Test
The various software components (e.g. tasks, routines, classes, objects etc.) are integrated,
again by the individual programmers under their local development configuration control, to
create a fully tested CSCI along with its related TSIs. Records of testing are held in the
associated test logs.
At the end of this phase, the CSCI should have been fully tested by its developer, who has
exercised all test procedures in the STD, and corrected any anomalies found, so that a final
version of the CSCI and its STD are now available to support qualification testing. This is
known as a Test Readiness Review (TRR). A preliminary Version Description Document
(that identifies and describes the software as delivered, provides instructions to install the
software, and also provides references on how to maintain the software) is produced and all
documents are passed into independent (formal) CM control ready for qualification testing.
1.8.6.6 Software Qualification Test
Each CSCI is tested against its Allocated Baseline in accordance with its STD by an
independent tester. In many circumstances, the CSCI will need to have been firstly integrated
with other system components to provide an environment within which it has defined inputs,
operational systems context, etc. In other circumstances, TSIs will be used to harness the
CSCI to provide a representative simulation of its user/operating environment.
Any anomalies found during testing are registered, categorised, prioritised against their
urgency to be addressed, and passed back to the developers for correction. As a result,
corrected systems should again be presented for testing to resolve the anomalies and any
required regression testing should be performed to ensure that pre-tested functionality has not
been adversely affected by the changes made.
The functional and physical completeness of the various CIs that have been subjected to test
are independently audited to ensure that they meet all of their intended functionality, and are
comprehensively documented, consistent with the item as tested. The resulting Product
Baselines are formally established / endorsed by the CCB and very carefully controlled and
managed from that point onwards.
At the completion of the Software Engineering activity, each CSCI will have been qualified
and either separate or combined (in a Product Delivery Notebook) documentation produced as
follows:
TRR Report
Software Test Report
Final Version Description Document (VDD)
Functional and Physical Configuration Audit (FCA and PCA) reports.
The software CI has accordingly been proven to be complete and robust and ready to be
integrated into the system (if that has not already occurred i.e. incremental development).
EMS METHODOLOGY
41 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
KEY:
P = Preliminary F = Final
Dev/mental CM Formal CM
Formal CM Developmental CM
Systems Engineering Control
System Architect'al
Design System Reqts
Analysis System
Integration And Test
System Qualification
Test
Software Detailed Design
Software Integration and Test SOFTWARE ENGINEERING PHASES
Software Qualification
Test Software
Code and Test
Deployment SYSTEMS ENGINEERING PHASES
PROJECT MANAGEMENT STAGES
ENGINEERING MANAGEMENT SYSTEM
Project Initiation
Project Closure
Software Reqts
Analysis Software
Architect'al Design
Project Start Up
HMI Screens HMI
Layout
PCA Report FCA Report VDD-F
Test Log STD-F
VDD-P
Test Log TSI Spec-F TSI Spec-P STD(Cases) STP - F STP - P
IRS -P SRS - P
IRS - F SRS - F
IDD - P SDD - P
IDD - F SDD - F
Qualified CI
Software prototypes
Allocated Baseline Control
STD(Proc)
Tested Software
Units & TSIs Tested CI STR
TRR Report
SOFTWARE ENGINEERING PHASES
Implement in Stages
Tailor to single
Product Delivery
Notebook (PDN) where
possible
Figure 13: Software Engineering Phases
EMS METHODOLOGY
42 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1.8.7 Life Cycle Models
While the above proposed Systems and Software Engineering phases are performed on an
engineering project, and in the order in which they are depicted in Figure 12 and Figure 13,
there can be considerable variation in the cyclic nature of their development from one project
to another. This is an important aspect of the overall project and technical planning that must
be determined case-by-case during the initiation phase of a project.
A number of different lifecycle approaches are available to suit the different circumstances
within which an engineering system solution might be developed. Three classic lifecycle
software development approaches exist, Waterfall, Incremental and Iterative Development,
as depicted in Figure 14, Figure 15 and Figure 16, respectively.
The Waterfall Development approach is only suitable in environments that have high stable,
clear requirements, architecture and technology (with possible precedence). An example
would be the upgrade or enhancement of a small to medium sized system previously
developed/constructed by MTM. It is the least favoured of the three approaches because of the
inherent risk in developing all system components in parallel with the expectation that they
will successfully integrate (i.e. original requirements were clear, adequate and complete and
the system design was ‘flawless’). A large development would rarely be approached this way
without introducing significant risk, particularly on ‘fixed-firm’ contracts.
Waterfall. The waterfall strategy, also called “once-through”, comprises performing the
development process a single time. Simplistically: determine user needs, define requirements,
design the system, construct the system, test, fix, and deliver.
The more desirable approach is that of Incremental Development that is applied whenever
possible. This approach ensures that the system specification (Functional and Allocated
Baselines) is comprehensively established at the start and the system is then developed from a
solid foundation of requirements analysis and design. A series of software releases are
planned, with the system and software architecture being proven in the first such release.
Depending on the size and complexity of the problem, the system functionality is then added
incrementally, providing basic, intermediate and then complete functionality. The number,
and the content, of the increments are determined based on the inter-relationship of the
functionality (i.e. what must come first) and its inherent size, complexity and therefore risk.
Incremental. The incremental strategy determines user needs and defines the system
requirements, then performs the rest of the development in a sequence of builds. The first
build incorporates part of the planned capabilities, and so on, until the system is complete.
However, it is recognised that certain projects may have requirements that are not fully
defined or understood at project commencement, with new and complex domains that may
also require a measure of trial and error during the development/build. This might be well
recognised by the Client who prefers an evolutionary acquisition approach, requiring
performance of analysis, then design and finally implementation tasks, each being defined
after completion of the latter phase, with correction or enhancement to the Functional and
Allocated Baselines being performed, particularly during early project phases. This is known
as Iterative/Evolutionary Development as it recognises that the specification, design and
implementation will iterate towards the best Client solution over time. Consequently, this is
not a suitable approach for a fixed price contract situation.
EMS METHODOLOGY
43 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Iterative (or Evolutionary) . The iterative or evolutionary strategy also develops a system in
builds but differs from the incremental strategy in acknowledging that the user need is not
fully understood and all requirements cannot be defined upfront. In this strategy, user needs
and system requirements are partially defined at the beginning, and then are further defined in
each succeeding build.
Where possible, these aspects should be considered during early negotiations with the Client
to ensure that the terms of contractual arrangements recognise the type of environment within
which the project will be developed, and particularly where an evolutionary, risk sharing
approach or a fixed/firm contract needs to be taken. The Project Manager and Project
Engineering Manager must determine the life cycle approach most suitable to the project
environment in the very early planning project phases.
1.8.8 Selecting the type of software development strategy
The following Tables provide guidance comparing development strategies is provided.
Risk analysis for determining the appropriate software development strategy:
EMS METHODOLOGY
44 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Systems Engineering Control
System Architect'al
Design System Reqts
Analysis System
Integration And Test
System Qualification
Test
Software Detailed Design
Software Integration And Test
SOFTWARE ENGINEERING PHASES Software
Qualification Test
Software Code
And Test
Deployment
System Requirement
Review System Design Review
Formal Qualification
Tests Acceptance
Review
SYSTEMS ENGINEERING PHASES
PROJECT MANAGEMENT STAGES
ENGINEERING MANAGEMENT SYSTEM
Project Initiation
Project Closure
Software Reqts
Analysis Software
Architect'al Design
Project Start Up
Possible Project Management control Stages during implementation
Finalise software across all CSCIs and qualify each independently.
Software requirements analysis commences when the system architecture has stabilised.
System requirements analysis is commenced as soon as engineering resource is available (and not required to support planning).
HW and Firmware reqts are finalised
Software Detailed Design
Software Integration And Test
Software Qualification
Test Software
Code And Test
Software Reqts
Analysis Software
Architect'al Design
Software Detailed Design
Software Integration And Test
Software Qualification
Test Software
Code And Test
Software Reqts
Analysis Software
Architect'al Design
Architectural Design System Build System Completion
WATERFALL DEVELOPMENT
All hardware and firmware available
All CSCIs developed independently to specification and then integrated.
The waterfall development model can be effectively implemented in certain types of projects only. It is particularly applicable in small to medium sized projects within which the technology, architecture and requirements are well known and stable. An example would be the upgrade or enhancement of a system previously acquired by MTM. The larger the project, the more unfamiliar the domain, the more complex the problem and the less solid the Client’s operational concepts, the more unsuitable is this approach. These types of projects need to be progressed in a 'learning environment', where initial design concepts are developed and tested prior to committing greater effort into detailed design and full scale implementation.
CSCI 1
CSCI 3
CSCI 2
Figure 14: Waterfall Development
EMS METHODOLOGY
45 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Systems Engineering Control
System Architect'al
Design System Reqts
Analysis System
Integration And Test
System Qualification
Test
SOFTWARE ENGINEERING PHASES
Deployment
System Requirement
Review System Design Review
Formal Qualification
Tests Acceptance
Review
SYSTEMS ENGINEERING PHASES
PROJECT MANAGEMENT STAGES
ENGINEERING MANAGEMENT SYSTEM
Project Initiation Architectural Design Project
Closure Project Start Up
Software Integration And Test
Software Code
And Test
INCREMENTAL DEVELOPMENT
Software Detailed Design
Software Integtn
And Test Software Qualif'n
Test Software
Code And Test
BUILD 2
HW and firmware is delivered to support the build cycle needs.
Initial System Build System Completion
Possible Project Management control Stages during implementation
Develop software across all CSCIs in support of Build 1 functionality.
Finalise software across all CSCIs and qualify each independently.
Detailed Design is started when the complete software architectural design has stabilised.
Next Build cycle is commenced when supportable by staff released from Build 1 activity.
Software requirements analysis is commenced when the system architecture has stabilised.
System requirements analysis is commenced as soon as engineering resource is available (and not required to support planning).
Establish the system requirements and stabilise the system architectural design. Next, establish the software and hardware requirements and stabilise the software and hardware architectural design. Then develop and integrate the system in multiple build cycles, the first establishing the architecture with subsequent builds providing additional system functionality, until complete. While all system and lower level requirements and interfaces are intended to be stable across the build cycles, related problems that are identified within a build are addressed prior to commencement of the next build cycle.
HW and Firmware reqts are finalised
Complete software requirements analysis and architectural design across all CSCIs to provide a stable 'build-to' baseline.
CSCI 2
BUILD 1
Software Reqts
Analysis Software
Architect'al Design
Software Detailed Design
Software Integration And Test
Software Code
And Test
CSCI 1 CSCI 2
CSCI 3
Figure 15: Incremental Development
EMS METHODOLOGY
46 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
ENGINEERING MANAGEMENT SYSTEM
Software
Reqts
Analysis
Software
Archit
Design
BUILD 1
Possible Project Management control Stages during implementation
ITERATIVE DEVELOPMENT
Systems Engineering
Control
System
Architect'al
Design
System
Reqts
Analysis
System
Qualification
Test
Deployment
Software
Detailed
Design
Software
Integration
And Test
Software
Code
And Test
Software
Reqts
Analysis
Software
Archit
Design
BUILD 2
System
Reqts
AnalysisSoftware
Reqts
Analysis
Software
Archit
Design
BUILD 3
CSCI 3
CSCI 2
CSCI 1
System
Archit
Design
System
Reqts
Analysis
System
Integn
And Test
System
Integn
And Test
SOFTWARE ENGINEERING PHASES
SYSTEMS ENGINEERING PHASES
Software
Detailed
Design
Software
Integn
And Test
Software
Code
And Test
System
Integn
And Test
Project
ClosureInitial Architectural Design System Completion
Software
Detailed
Design
Software
Integn
And Test
Software
Code
And Test
Software
Qualifcn
Test
ITERATIVE CYCLES
COM
PLE
TE S
ET O
F SYSTE
MS A
ND
SO
FT
WA
RE
EN
GI N
EE
RI N
G
PHASES O
N EA
CH C
YCLE
An Iterative Development approach is used when the customer requirements are unstable, or when the technology is complex or
unfamiliar to the design team. Such a project will normally have a high R&D component and be performed under some kind of
evolutionary acquisition arrangement with the customer, each sharing and managing the associated risks.
It involved a substantial early effort to establish the initial system requirements and determine the initial system architectural
design. A complete systems and software lifecycle is then performed to develop an initial implementation of basis system
functionality in order to confirm initial concepts and to learn as much as possible about the technology. As a result, a complete
re-iteration of the system and software requirements and design is performed and another build cycle completed to upgrade the
system and enhance its functionality. This process is repeated as many times as is necessary, with the system requirements
and design being stabilised as early as possible, the hardware and software development then being able to be finalised in the
last cycles.
System Build 2System Build 1
System
Archit
DesignSystem
Requirements
Review
System
Design
Review
Formal
Qualification
Tests
Acceptance
Review
Figure 16: Iterative Development
EMS METHODOLOGY
47 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1.8.9 Engineering Reviews
The engineering performed by MTM may be subject to a range of both internal management
and technical peer review, and external management and technical reviews to suit the Client’s
possible involvement. Figure 17 depicts the following engineering gate/reviews, and others to
be considered, when planning a project:
Joint Management Reviews
Formal Engineering Reviews (FER) comprising:
Preliminary Requirements Review (PRR)
Systems Requirements Review (SRR)
Post Award Concept Design Gate (CDG)
Preliminary Design Review (PDR)
Critical Design Review (CDR)
Test Readiness Review (TRR)
Joint Management Reviews are periodic project status reviews held with the Client’s project
management team to monitor the progress from a schedule, cost, strategic and relationship
perspective. Project or contract performance reports would normally be provided with agreed
content and structure ahead of a meeting chaired by the Client. Any management issues of
concern would be aired and resolved immediately where possible, or assigned actions taken to
resolve them within agreed timeframes. MTM’s approach should encourage an open,
cooperative and honest dialogue with the Client, exposing all project issues and resolving any
associated project risks to cost or schedule at the earliest opportunity.
The following formal engineering reviews (FERs) – internal or external - are held within
MTM, or between MTM, Designer, Contractor and/or the Client to deal with issues associated
with technical specification, progress, quality and compliance. As a minimum, the following
FERs will be conducted, as applicable, on selected projects.
Preliminary Requirements Review (PRR) - Optional PRR assesses whether the User
requirements are sufficiently acceptable and mature, to proceed to SRR and subsequent
engineering specification and design phases with acceptable risk, within cost and schedule
requirements and constraints.
The objective of this review is to ensure that high-level (concept) User requirements
(functional, performance (and possibly non-functional) and constraints are unique, succinct,
clear, complete, unambiguous, implementable (e.g. from asset management perspective),
verifiable, non-conflicting, consistent with all other User requirements and adequately and
realistically describe the engineering/technical User requirements captured in the Scope of
Works (SOW) document.
Criteria for deciding whether to plan a PRR, and what to assess during its conduct, include:
- the deemed maturity of the engineering/technical system concept requirements as
described in the SOW;
- required system objectives, behaviour and performance are clear;
EMS METHODOLOGY
48 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
- whether the system concept (User) requirements are organised in a clear (and desirably
modular) structure (functional and performance requirements as a minimum);
- requirements state what is wanted without defining the design solution;
- the relationship and traceability between high level, related requirements are clear and
appear valid;
- whether all User requirements are verifiable; and
- whether any User requirements may or may not be met.
Systems Requirements Review (SRR) to agree & baseline established ‘system
requirements’ (functional, performance and non-functional) pre design. SRR confirms
MTM’s comprehension of the work scope and formally agree the initial version of the
documents that will form the Functional Baseline. SRR is included as part of Works
Readiness checklist.
The SRR is a formal review led by MTM with participation by Designers and Client. It is
conducted when the significant portion of the System Requirements (i.e. functional,
performance and non-functional) are clearly established, analysed and defined and establishes
agreement to proceed to specification development and preliminary design. A refined ‘Scope
of Works’ document would normally be an output of the SRR – and would form the basis of
the ‘system requirements’ that initiates the core Engineering lifecycle activity (engineering
WBS, specification(s), design etc.)
NOTE: The engineering SOW will usually be generated earlier than at SRR (e.g. at concept
stage by State) and be assessed at PRR – however, it must precede CDG and subsequent
specification development and design activity.
Concept Design Gate (CDG) The need for a post-award CDG is determined at SRR. The
Designer(s) present(s) the concept design (before approval to proceed to preliminary and
detailed design) to confirm requirements clarity and acceptability/validity of proposed design
approach and solution preceding development of specifications that will drive the design. The
CDG is included as part of Works Readiness checklist.
Preliminary Design Review (PDR) The PDR determines that the preliminary design meets
systems requirements agreed at SRR with acceptable risk and within cost & schedule
requirements /constraints and is sufficiently mature to proceed to detailed design.
The Allocated (Development) Baseline is established at successful completion of the PDR.
Critical Design Review (CDR) confirms detail design is essentially complete and satisfies
system requirements from SRR. At CDR, the Designer will expose to MTM the overall
design, development and qualification testing approach and thereby seek any comment from
the Client related to their expectations against the Functional Baseline which will be finalised
as a result. CDR is included as part of Works Readiness checklist.
Purpose of CDR is to determine that detail design under review satisfies specified system
(HW & SW) requirements (incl. interface/compatibility) in accordance with development
specification(s). Assess system/Product(s) risk aspects (on technical, cost and schedule basis).
The CDR is usually conducted for each major Product/suite/subsystem when the detail design
EMS METHODOLOGY
49 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
is essentially completed. For software, the CDR (‘Detailed Design Review’) should focus on
determining acceptability of detail design, performance and test characteristics of the design
solution; and the adequacy of the SW’s safety, operational and support documents (e.g. Safety
Cases), particularly for new/modified safety critical SIL 3 and SIL4 software.
Test Readiness Review (TRR) reviews the status of:
♦ installation/testing;
♦ open ECs, open Waivers, open test problem reports, open reqts & design issues;
♦ readiness to proceed to, and review planning for, remaining ‘system’integration
and T&C (T&C should be a formal, ‘high confidence event’).
TRR is conducted to 1) Assess subsystems (& Products’) readiness to be integrated as the
‘System’; 2) Assess status of ongoing installation testing; and 3) Plan (and assess
preparedness for) system integration testing and T&C ‘event’ (high confidence). TRR is
included as part of Works Readiness checklist.
NOTE: T&C should primarily validate the Client’s Operational Requirements, as a
minimum; whereas Integration Testing should focus more on verifying low level subsystem
requirements (and validating selected system requirements e.g. tests of necessarily longer
duration prior to T&C).
Acceptance Review (post Test & Commissioning (T&C)) where MTM’s, and possibly the
Client’s, project management team review the completeness of all contractual activity to
provide for formal acceptance/commissioning of the completed system.
Ad-hoc FERs, where necessary, to jointly manage and control technical issues that must be
resolved in an efficient and timely manner to ensure that development can progress to
schedule. These might take the form of working groups, such as special external interface
control working groups, HMI working groups etc. – sometimes when a Client wishes to
integrate their specialists into the project team to provide for additional domain expertise or
authoritative review, verification etc. This can be a very useful strategy.
For software solutions, FERs should be held between the contractor and acquirer (MTM) to
evaluate the software solutions/products viz.:-
o status/completion against agreed requirements and controlled
schedule/milestones;
o compliance with requirements, standards and specifications (system/software);
o all changes have been properly implemented ('transparent'), effectively
('controlled'), properly managed ('authorised') and internally reviewed (‘peer
reviews’ or ‘walkthroughs’) and approved ('configuration management');
o readiness for advancing to the next scheduled phase/activity; and
o development, operation and maintenance are being conducted according to
approved contract/project plans, schedules, standards, processes and guidelines.
EMS METHODOLOGY
50 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Monthly Project Status Reviews are held between the MTM Project Manager and the MTM
Project Executive to internally monitor the project progress and achievement of Company and
Client expectations. If a Client provides a project ‘resident’ technical team to work with the
MTM project Delivery team in an integrated teaming arrangement, the senior Client
representative would be invited to attend. Similarly, invited attendance by senior
subcontractor representatives might be appropriate.
Internal Peer Reviews (incl. Formal Peer Reviews) should be held at appropriate times
throughout the Systems and Software Engineering phases of the project to enable adequate
review of the developing system components. Their purpose is to identify errors as early as
possible in the development/construction lifecycle when their correction has the least cost and
schedule impact. These might take the form of either peer reviews or structured walkthroughs,
depending on the nature and maturity of the product under review (e.g. plan, specification,
design, code, etc.). It is recommended that Formal Peer Reviews be held to review all formal
project documents/deliverables or other important artefacts, prior to submitting to the Client.
NOTE: For software reviews (usually at the software Developer’s facility), ‘structured
walkthroughs’ are a universally adopted software technique for engineering reviews of
software design and code and are typically held at the end of each software engineering phase
to confirm completeness and consistency with related products. Structured walkthroughs are
independently moderated and all anomalies documented, categorised and closed out in a
controlled manner.
EMS METHODOLOGY
51 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Test
Systems Engineering Control
System Architectural, Preliminary & Detailed Design
System Reqts
Analysis System
Integration And Test
System Qualification
Test
Software Detailed Design
Software Integration And Test SOFTWARE ENGINEERING PHASES
Software Qualification
Test Software
Code And Test
Deployment
System Requirement
Review Critical Design Review
Readiness
Review Acceptance Review
Formal Engineering Reviews
Internal Peer Reviews
Monthly Project Status Reviews
Joint Management
Reviews
SYSTEMS ENGINEERING PHASES
PROJECT MANAGEMENT STAGES
ENGINEERING MANAGEMENT SYSTEM
Internal Peer Reviews
Project Initiation
Project Closure
Software Reqts
Analysis Software
Architect'al Design
Other Joint Technical Reviews and WGs as required
Test Readiness Review(s)
Project Start Up
REVIEWS
Implement in Stages
Contract Review
Detailed Design Review (Software)
Figure 17: Project Reviews
EMS METHODOLOGY
52 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1.8.10 Action Item and Problem Tracking
A regime of action items and problem tracking should be employed at MTM to ensure that
projects are kept on track, identified anomalies are addressed as early as possible, and
important activities are not overlooked due to the pressures of deadlines and associated
milestone achievement. Figure 18 identifies the tracking associated with each of the review
types depicted in Figure 17 as described in the above section of this EMS Methodology.
For Joint Management Reviews and Formal Technical Reviews, an agreed action item
tracking mechanism should be implemented such that each item raised is registered and
routinely reported until closed. As a minimum, these action items should be managed to
closure through the minutes of such reviews.
For Monthly Project Status Reviews, action items raised by the Project Executive should be
tracked from meeting to meeting until closed by incorporation within the associated Highlight
Reports.
For internal Formal Peer Reviews, the reviews themselves, attendance, results and all
identified anomalies should be tracked in a review database, with each entry being controlled
by the independent review Moderators and checked by HSEQ representatives.
For project’s products that have been accepted into formal configuration management
(i.e. ‘baselined’), a regime of System Problem Reports (SPRs) should be implemented to
enable any project team member to identify an issue at any time including, but not limited to,
final qualification testing activity. Each SPR must be formally investigated by an appointed
engineer, with recommendations for its resolution forwarded to the identified and authorised
MTM technical/engineering authority for approval.
SPRs will be raised for engineering/technical failures, faults, defects or any technical non-
conformances, physical (hardware/software) or functional/performance, and determined as a
result of testing, inspection, analysis or demonstration. Technical problems associated with
design, and associated requirements, would be captured as ‘Issues’ and addressed as part
requirements/design reviews and by Standard Waivers, Engineering Change and risk
management.
Once an SPR has been approved for implementation of a change, the update of each product
or system affected by the change should be managed (this may be done via a form such as a
system change request ‘SCR’ (to be agreed/developed)). This thoroughness of approach will
ensure that every engineering (system, hardware or software) problem, once registered, is
addressed and that every authorised system change is tracked to completion.
EMS METHODOLOGY
53 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Review Anomalies - Tracked in Review DB
Review Anomalies - Tracked in Review DB
Review Anomalies - Tracked in Review DB
System Problem Reports/System Change Requests - Tracked via development tools (only raised against Baselined products)
Project Action Items - Tracked on Highlight Report
Customer Action Items - Tracked by agreed mechanism
Customer Action Items - Tracked by agreed mechanism
Systems Engineering Control
System Architect'al
Design System Reqts
Analysis System
Integration And Test
System Qualification
Test
Software Detailed Design
Software Integration And Test
SOFTWARE ENGINEERING PHASES Software
Qualification Test
Software Code
And Test
Deployment
System Requirements
Review System Design Review
Formal Qualification
Tests
Acceptance
Review Joint
Technical Reviews
Internal Peer Reviews
Monthly Project Status Reviews
Joint Management
Reviews
SYSTEMS ENGINEERING PHASES
PROJECT MANAGEMENT STAGES
ENGINEERING MANAGEMENT SYSTEM
Internal Peer Reviews
Project Initiation
Project Closure
Software Reqts
Analysis Software
Architect'al Design
Other Joint Technical Reviews and WGs as required
Test Readiness Reviews
Project Start Up
ACTION ITEM AND PROBLEM TRACKING
Implement in Stages
Baseline Change Control Mechanism
Contract Review
Commissioning
Figure 18: Action Item and Problem Tracking
EMS METHODOLOGY
54 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
1.8.11 Prototype and Concept Demonstrator Development – Software (Example only)
Figure 19 depicts an example of tailoring the standard, proposed MTM systems and software
engineering lifecycle to reduce the process overhead associated with the development of
prototype systems or concept demonstrators. While such systems still need to be engineered
within a controlled environment (e.g. specification, design, implementation and test), the
necessary rigour is reduced given that such systems are not expected to be delivered to a
Client for use in an operational environment. In that regard, the various Specifications (e.g.
SRS) and Descriptions (e.g. Software Design Description) required to be created would be
managed within a development tool and not extracted to form associated formal
documentation unless this was a specified Client requirement.
Such a project would be commenced with documentation outlining the scope of the
development work. Depending on the scope and size of the project, the development might
require a full systems approach or it might merely require the development of a small
software system on a pre-defined hardware architecture/platform.
Where a full systems approach is required, a detailed System Specification (suite of
requirements) will need to be developed to control the project scope (the Functional
Baseline). Any interfacing requirements would be included in this singular specification
rather than being documented separately. A System Design Description will need to be
developed, with associated target system hardware architecture defined. This assumes that the
system is to be developed as a number of separately developed software CIs that must be
integrated and tested as an interacting system. The functionality assigned to each software CI
would be separately identifiable in one or more SRS documents. Again, any associated
interface requirements would be included in that specification. Whilst test planning is still
required, it is expected that this can occur through the development of a Systems Test
Procedures without first creating a System Test Plan. Given the rapid development nature of
such a project, developmental configuration control (rather than independent, formal CM)
provides sufficient rigour. The system qualification testing might also be exercised by the
development team rather than by an independent authority, verifying that the system operates
in accordance with its Functional Baseline. The configuration of the system under test and the
test results would be recorded.
Where only a small software development approach is required, the project scope statement
is analysed and a single SRS developed, any interfacing requirements being included.
Otherwise, the preliminary SRS developed under the full systems approach would be finalised
for each software CI involved. The AGILE method often is used for this sort of scale.
The design of the software, including its interfaces, would be documented in one or more
Software Design Descriptions, incorporating all interface, test support item and HMI design
information. The software test planning would be managed through the development of the
STD, with no necessity to first develop a Software Test Plan. All software development
would be managed under developmental configuration control by the software staff, without
the need for independence in its qualification against the Allocated Baseline (SRS) or a need
for formal CM. The configuration of the software under test and the test results would be
recorded by the development team.
EMS METHODOLOGY
55 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Developmental CM
Developmental CM
SysHW Arch SSDD
Systems Engineering Control
System Architect'al
Design System Reqts
Analysis System
Integration And Test
System Qualification
Test
Software Detailed Design
Software Integration And Test
SOFTWARE ENGINEERING PHASES Software
Qualification Test
Software Code
And Test
SYSTEMS ENGINEERING PHASES
PROJECT MANAGEMENT STAGES
ENGINEERING MANAGEMENT SYSTEM
Project Initiation
Project Closure
Software Reqts
Analysis Software
Architect'al Design
Project Start Up
VDD-F
VDD-P STD(Cases)
SRS - P SRS - F SDD - P SDD - F
Allocated Baseline Control
STD(Proc)
Tested Software
Units & TSIs Tested CI STR
System Spec-F
Functional Baseline Control
System Test
Procedures Sys Config
Sys Test Rpt System Spec-P
Implement in Stages
Prototype or Concept
Demonstrator Scope
PROTOTYPE AND CONCEPT DEMONSTRATOR DEVELOPMENT
Figure 19: Prototype and Concept Demonstrator Development
EMS METHODOLOGY
56 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
APPENDIX 1: PROJECT ENGINEERING MANAGER – ROLE DESCRIPTION
Position Purpose:
Provide engineering and technical management and coordination required for the project to deliver its
technical obligations and objectives. The PEM is responsible for the delivery of the engineering
solution/product and data to meet project requirements. The PEM provides engineering support,
advice and assistance to the MTM Project Manager (PM) and has engineering oversight of the project
in accordance with the MTM ‘Engineering Management System (EMS) Methodology’, tailored to
meet the specific, planned needs of the project.
The PEM will engage with, and support, MTM project, engineering and other MTM Business Units (e.g. Finance and Commercial) in the identification and resolution of any trade-offs between cost, schedule, quality, safety and performance based, in part, on processes, methods, and information produced by the EMS to deliver solutions compliant with engineering requirements as part of engineering and project risk management.
Key Responsibilities:
In conjunction with the Project Manager (PM) and MTM Engineering Systems Manager, plan the
execution of the engineering scope of the project and tailor and apply the MTM Engineering
Management System (EMS) methodology as required to meet the project’s objectives and
priorities.
Contribute to the development and review of technical/engineering requirements, specifications
and systems engineering solutions to ensure compliance with applicable contract obligations,
safety regulations and MTM policies.
In conjunction with the PM, participate in planning/estimation activities and plan and manage
delivery of the project's engineering product to cost and schedule including acceptance by the
customer and forecast engineering resourcing requirements through the project lifecycle phases.
Responsible for technical liaison and negotiation of technical issues with client and contractor(s)
representatives associated with the project.
Lead, manage and provide input into the performance management of a team of project
engineering personnel assigned to the project.
Provide assurance to both the PM and Chief Engineer that the project is technically sound and
ensure the end product is safe and fit for purpose by providing guidance and direction on
appropriate levels of engineering to be undertaken, as planned, by the project.
When necessary, arrange for Independent Technical Assessments (ITA) of any aspect of the
engineering process at any time during the project.
Monitor and assess the project engineering effort to ensure the project engineering activities are
conducted correctly and at the appropriate level for the specific, planned engineering tasks per the
SEMP and CMP and, similarly engineering effort performed by contractors, including contractors
responsible for design.
EMS METHODOLOGY
57 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Manage technical risk and engineering system safety and assess and report project risks and
opportunities (e.g. design, design margin); actively contribute to the project’s risk management
processes and project risk register.
Oversee, coordinate, chair and manage formal, technical milestones of the project (e.g. SRR,
CDR and TRR) and ensure all associated ‘Entry’ and ‘Exit’ criteria are satisfied, with any carry-
over, open issues and technical risks being properly agreed (incl. engineering), documented and
assigned/handed over throughout project phases (e.g. at Project Closure).
Perform engineering/technical documentation reviews and approvals and oversee/manage the
configuration baseline on the project in accordance with the MTM EMS methodology, policies
and procedures.
In particular, collaborate with Engineering Systems Manager and PM to implement the EMS, the
project engineering scope of which will include, as applicable:
Requirements Development & Management incl. providing input on
constructability, ongoing maintenance and renewal impacts
Development of project engineering Plans (incl. SEMP & CMP) and overseeing
development of contractor engineering plans (e.g. SDP, Software CMP)
Engineering WBS development and management
As required, prepare/review MTM design and technical construction
specifications and standards
Prepare/ review project design briefs
Oversee and manage the project system design and authorise and issue of design
certification under delegation from the Chief Engineer
Configuration Management processes incl. Configuration Control Boards
Engineering Baseline Management
Design & Test Management & Reviews (SRR, CDR and TRR), including
ensuring that engineering safety requirements are clearly specified and addressed
through design, design reviews and V&V activities
MOC, Engineering Change and Type Approval activities related to the project
Earned Value management (engineering Work Plans, performance analysis and
variance reporting)
Engineering Safety (per SSAF and SMS)
Integration, testing & commissioning – planning, coordination and conduct
Promote innovation and strive to complete all activities on or ahead of schedule
whilst challenging accepted practices that may impede project success
Verification & Validation (V&V) planning and coordination
EMS METHODOLOGY
58 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
APPENDIX 2: CRITERIA FOR ESTABLISHING PEM ROLE ON MTM PROJECTS
Overview
The requirements for the PEM role on MTM Projects are based on the 5 criteria shown in
Table 1 below.
Criteria for establishing a PEM role on a particular project should be based on more than just
contract cost or subjective assessments of a particular project’s complexity or risk level.
Instead, establishing the PEM role on a project should be based on objective criteria for a
particular project. Table 1 provides the criteria to be used for this purpose based on project
classification. This approach is intended to provide a clear, consistent and unambiguous
means for determining the need for, and establishing, the important PEM role.
(NOTE: A generic PEM Position Description has been created and is available for tailoring to
a project’s requirements once the need for the PEM role has been determined.)
Rules for applying PEM role to MTM Projects
The criteria are worked through independently, in sequence, with the criterion a “YES” or
“NO” for each criterion.
If one or more of the criteria satisfies the requirement for the PEM role being established on a
project, the project will be deemed to require this role. However, it is possible to further
assess this requirement (say, where only one or two of the criteria are marginally satisfied). In
such cases, a part time PEM role or indirect engineering support for this project function may
be options. In all such instances where it is decided not to establish the PEM role on a project
– despite satisfying one or more “YES” criteria of Table 1, documented risk controls (in the
risk register) must be established and shall be put in place to mitigate the absence of this
engineering role.
Justification will also be required to remove the requirement for the PEM on projects after
assessment against the Table 1 criteria. This can only occur after review and prior agreement
by the GM Projects, NAM and Chief Engineer (or authorised delegate(s)).
Table 1: Criteria for MTM PEM role.
Ref Criterion (refer to Guidance Notes below)
YES NO
1 Cost (total contract)
>$50m (Signal, Electrical)
>$100m (all disciplines/types)
<$50m (Signal, Electrical)
<$100m (all other disciplines/types)
2 Contract Type Type 2 Type 1
3 Risk Assessment 12 to 25 <12
4 Design involvement Yes (D&C) No
5 Cross-Discipline Projects Yes (at least 2 Disciplines) No
EMS METHODOLOGY
59 of 59
L1-CHE-MAN-001(1) Approv al Date: 20/2/2014
Approving Manager: Engineering Systems Manager
Guidance Notes on PEM Criteria (for Table 1):
1. Cost (total contract)
A preliminary assessment of the total contract cost of the project will be made (values/ranges
are shown in Table 1 above). However, a final validation of the total contract cost should be
made and used for this criterion, including project governance and margins associated with
level of risk associated with such things as contract type.
2. Contract Type
Type 1 Purchase Order (PO), PO plus Schedule of Rates, Lump Sum, Cost
Reimbursable.
Type 2 Type 1 plus D&C, Fixed/Firm, JV, Alliance, PPP, ECI, Performance
Specified or other complex engagement agreements.
3. Risk Assessment
An assessment of the contract/project risk, as part of this PEM criterion evaluation, will be
based on the “Composite score table” contained in Appendix 1 of the MTM Enterprise Risk
Management Procedure.
4. Design involvement
The requirement for design, or otherwise, including performance requirements, will be
assessed.
For guidance, if design is not a requirement of the project’s scope, this rating will default to a
“NO”.
If the contract does have design requirements as part of its scope, the rating will default to
“YES”. This includes:
MTM does not directly undertake the design but employs design
consultants;
projects involving software product development/modification with
software integrity level rated at either SIL3 or SIL4; or
MTM directly undertakes design for the project, even ‘minor’ design.
5. Cross-Discipline Projects
At least 2 of the following Discipline categories are within the scope of the project:
Electrical (Overhead, Substation(s), Bonding, SCADA)
Signalling (incl. major communications, OCS)
Track, Structures and/or Facilities
MURL
Approval for PEM role
Approval of the PEM role for a project will be required from GM Projects, Head of NAM and
the Chief Engineer.