a framework to assess project technical progress
TRANSCRIPT
General Engineering Maturity Measure
A framework to assess project technical progress
General Engineering Maturity Measure◦ A practical measure for Technical development activities◦ Based on Systems Engineering principals
◦ Provide independent and unambiguous measure of progress Not subjective estimate of completion
Not CMMI◦ Measures the activity, not the business
What is GEMM?
◦ A practical and pragmatic measure for projects and programmes
◦ Scalable across technical problems With modest tailoring
◦ Easy to deploy Little training burden
Obvious and intuitive Driven by existing or easily derived data
From PLM, MBSE and other common SE tools
Objectives in Building GEMM
An evidence based measure of programme maturity◦ For technical development activities◦ Provides Evidence based decision making for:
Risk reduction Resource management
Any programme◦ Supports unambiguous P3M measures◦ Great value on large complex programmes
In the context of broader business processes
What is GEMM for?
Widely Applicable◦ Broader engineering processes◦ Built on Systems Engineering principals
Reduce dependence on high levels of SE knowledge◦ Straight forward application against familiar document set◦ Naturally available from SE tools and techniques
Controlled review points◦ Unambiguous objective measures◦ Against planned expectations
Potential for Automation
Objectives
Derived from the Principals of EPaRT ◦ (Became GEAR)
EPaRT = Engineering Planning and Review Tool◦ Planning and Review◦ Status Audit◦ Progress reporting
GEAR – Guidance to Engineering Activity and Review◦ Focussed on Engineering artefacts◦ In a procurement context
Background
Sites within Generalised Business Model
Objective & Blue-Print
Organisation & Governance
Stakeholder managemen
t
Data Management
Resource Management
Uncertainty (Risk)
Management
Culture
Communication
Commercial Management
Financial Management
Schedule Management
Requirements & Acceptance
Technology Management
Architecture
Interfaces
Supply Management
Safety & Security
Production
IntegrationSustainment & Environment
Verification & Validation
Deliverables
Decision Processes
Customer
Customer
Business
Poj/Prog Man
Engineering
Development of business
GEMMP3MBSM
Enshrined experience of Systems Engineering◦ A more objective basis of assessment◦ With recognisable Systems Engineering taxonomy
Based on tangible outputs ◦ Defined metrics◦ Defined documentation set
Robust, Tailorable and Scalable◦ Dual Independent measures◦ Loosely coupled sources◦ Common Tailoring model
Codification of SE principals
Metrics◦ That reflect progress against dimension◦ That are unambiguously measureable◦ Based on stability (not absolute value)
Documentation◦ Document status – what documents are expected at reviews◦ Document Content – how documents reflect to progress
Intuitive to progress◦ Graphically and logically
Measures
OpenStableChilledFrozenLocked
Planning
0
100
200
300
400
500
600
700
800
900
1000
KICK-OFF SRR SDR PDR CDR PRR TRR FCA/PCA DELIVERY
Metrics PlanReq&Acc Technology ArchInterface Sust&Env S&SProduct Integrate V&V
0
100
200
300
400
500
600
700
800
900
1000
KICK-OFF SRR SDR PDR CDR PRR TRR FCA/PCA DELIVERY
Document PlanReq&Acc Technology Arch
Interface Sust&Env S&S
Product Integrate V&V
0102030405060708090
100Req&Acc
Technology
Arch
Interface
Sust&EnvS&S
Product
Intergate
V&V
Documents
Current Expected Next
0102030405060708090
100Req&Acc
Technology
Arch
Interface
Sust&EnvS&S
Product
Integrate
V&V
Metrics
Current Expected Next
Early Risk Identification
0102030405060708090
100Req&Acc
Technology
Arch
Interface
Sust&EnvS&S
Product
Intergate
V&V
Documents
Current Expected Next
Technology and Architecture is
more stable than Requirements
Document Set StatusCu
rren
t Rev
iew
Document
Review .
Revi
ew N
eede
d
Stak
ehol
der M
anag
emen
t Pla
n
Risk
Site
Sur
vey
Gene
ric C
onO
ps
Data
and
Con
figur
ation
Man
agem
ent P
lan
Prod
uct a
nd S
yste
m R
equi
rem
ents
Doc
umen
t
Arch
itect
ure
Desc
riptio
n Do
cum
ent
Tech
nolo
gy S
trat
egy
Tech
nolo
gy m
atur
ity re
port
Safe
ty a
nd S
ecur
ity a
naly
sis
Faul
t Tre
e An
alys
is
Failu
re M
odes
Eff
ects
and
Criti
calit
y An
alys
is
Inte
rfac
e M
anag
emen
t pla
n
Inte
grati
on p
lan
Bill o
f Mat
eria
ls (a
s pro
cure
d)
Proc
urem
ent s
ched
ule
Spar
es h
oldi
ng A
naly
sis
Syst
em T
est S
trat
egy
Infr
astr
uctu
re D
evel
opm
ent p
lan
Prod
uctio
n Pl
an
Verifi
catio
n pl
an
Legi
slatio
n an
d ce
rtific
ation
pla
n
Syst
em T
est S
ched
ule
Test
Pro
cedu
res
Trai
ning
nee
ds a
naly
sis
Trai
ning
mat
eria
l
Envi
ronm
enta
l Im
pact
Ass
essm
ent
Soft
war
e In
tegr
ation
pla
n
Syst
em In
tegr
ation
Pla
n
Inst
alla
tion
plan
Tran
sition
to u
se p
lan
Thro
ugh
Life
Supp
ort p
lan
Valid
ation
pla
n
Fact
ory
Acce
ptan
ce P
lan
Cust
omer
Acc
epta
nce
Plan
Verifi
catio
n Cr
oss R
efer
ence
Mat
rix
Mai
nten
ance
Sch
edul
e
Mai
nten
ance
pro
cedu
res
x Document Needed X x X X x X X X X X X X x x X X X X X x X X X X X X X X X X X X X X X X Xx Kick-off x O O O
SRR xSDR xPDR xCDR xPRR xTRR x
FCA/PCA xDelivery x
Documentation requirement - titles are subject area guidance; subject areas may be covered by single or multiple documents in any combination
Filter
Unfilt
Tailored Document Set and Reviews
Curr
ent R
evie
wDocument
Review .
Revi
ew N
eede
d
Stak
ehol
der M
anag
emen
t Pla
n
Risk
Gene
ric C
onO
ps
Data
and
Con
figur
ation
Man
agem
ent P
lan
Prod
uct a
nd S
yste
m R
equi
rem
ents
Doc
umen
t
Arch
itect
ure
Desc
riptio
n Do
cum
ent
Tech
nolo
gy S
trat
egy
Tech
nolo
gy m
atur
ity re
port
Inte
rfac
e M
anag
emen
t pla
n
Inte
grati
on p
lan
Bill o
f Mat
eria
ls (a
s pro
cure
d)
Proc
urem
ent s
ched
ule
Syst
em T
est S
trat
egy
Verifi
catio
n pl
an
Syst
em T
est S
ched
ule
Test
Pro
cedu
res
Envi
ronm
enta
l Im
pact
Ass
essm
ent
Soft
war
e In
tegr
ation
pla
n
Syst
em In
tegr
ation
Pla
n
Inst
alla
tion
plan
Valid
ation
pla
n
Fact
ory
Acce
ptan
ce P
lan
Cust
omer
Acc
epta
nce
Plan
Verifi
catio
n Cr
oss R
efer
ence
Mat
rix
x Document Needed X x X X x X X X X x x X X x X X X X X X X X X Xx Kick-off x O O O
SRR xSDR xCDR xTRR x
Delivery x Docu
men
t Sta
tus
Documentation requirement - titles are subject area guidance; subject areas may be covered by
Filter
Unfilt
At SRRRequirements and Acceptance
No. of Product requirements (PRs) C
No. of System Requirements (SRs) C % of SRs traced to PRs S % of SRs classified S
% of SRs with acceptance Strategy O 57
Techology Management
No. of technology lines tracked S
% of Tech Lines traced to Tech Strat S
% of Tech Lines traced to procurement O
% of Tech Lines traced to maturity plan O
Current system technical maturity O 29
ArchitectureNo. of HW CIs O No. of SW CIs O % of CIs traced to SRs O
No. of dependencies between CIs
No. of connections between CIs 9
Interfaces No. of Interfaces managed
No. of interfaces traced to CIs
No. of Interfaces specified (ICDs)
No. of interfaces traced to owner
No. of Interfaces agreed 0
Sustainment and Environment
No. of Environmental impacts tracked
No. of Env impacts controlled
No. of Faults tracked in FTA Reliability MTBF
No. of CIs tracked to MTTR 0
Safety & SecurityNo. of threats tracked No. of hazards tracked
No. of threats mitigated
No. of Hazards mitigated
No. of Failure modes tracked in FMECA 0
Production % of CI specifications complete
% of CIs tracked to production schedule
No. of CIs tracked to procurement No. of CIs on time
% of Sub-system (CI) compliance 0
IntegrationNo. of dependencies on integration sequence
% Configuration defined
% system configuration frozen No. of faults reported % System integrated 0
Verification and Validation
No. of SRs covered by Test Procedures
No. of SRs with compliance % test complete
% of certification complete No. of concessions 0
At CDRRequirements and Acceptance
No. of Product requirements (PRs) L
No. of System Requirements (SRs) F % of SRs traced to PRs F % of SRs classified F
% of SRs with acceptance Strategy F 96
Techology Management
No. of technology lines tracked F
% of Tech Lines traced to Tech Strat F
% of Tech Lines traced to procurement C
% of Tech Lines traced to maturity plan C
Current system technical maturity S 82
ArchitectureNo. of HW CIs C No. of SW CIs C % of CIs traced to SRs C
No. of dependencies between CIs S
No. of connections between CIs S 71
Interfaces No. of Interfaces managed C
No. of interfaces traced to CIs C
No. of Interfaces specified (ICDs) S
No. of interfaces traced to owner S
No. of Interfaces agreed O 57
Sustainment and Environment
No. of Environmental impacts tracked S
No. of Env impacts controlled S
No. of Faults tracked in FTA S Reliability MTBF O
No. of CIs tracked to MTTR 33
Safety & SecurityNo. of threats tracked S No. of hazards tracked S
No. of threats mitigated O
No. of Hazards mitigated O
No. of Failure modes tracked in FMECA 26
Production % of CI specifications complete O
% of CIs tracked to production schedule O
No. of CIs tracked to procurement O No. of CIs on time
% of Sub-system (CI) compliance 9
IntegrationNo. of dependencies on integration sequence O
% Configuration defined O
% system configuration frozen No. of faults reported % System integrated 6
Verification and Validation
No. of SRs covered by Test Procedures O
No. of SRs with compliance % test complete
% of certification complete No. of concessions 3
At DeliveryRequirements and Acceptance
No. of Product requirements (PRs) L
No. of System Requirements (SRs) L % of SRs traced to PRs L % of SRs classified L
% of SRs with acceptance Strategy L 100
Techology Management
No. of technology lines tracked L
% of Tech Lines traced to Tech Strat L
% of Tech Lines traced to procurement L
% of Tech Lines traced to maturity plan L
Current system technical maturity L 100
ArchitectureNo. of HW CIs L No. of SW CIs L % of CIs traced to SRs L
No. of dependencies between CIs L
No. of connections between CIs L 100
Interfaces No. of Interfaces managed L
No. of interfaces traced to CIs L
No. of Interfaces specified (ICDs) L
No. of interfaces traced to owner L
No. of Interfaces agreed L 100
Sustainment and Environment
No. of Environmental impacts tracked L
No. of Env impacts controlled L
No. of Faults tracked in FTA L Reliability MTBF L
No. of CIs tracked to MTTR L 100
Safety & SecurityNo. of threats tracked L No. of hazards tracked L
No. of threats mitigated L
No. of Hazards mitigated L
No. of Failure modes tracked in FMECA L 100
Production % of CI specifications complete L
% of CIs tracked to production schedule L
No. of CIs tracked to procurement L No. of CIs on time L
% of Sub-system (CI) compliance L 100
IntegrationNo. of dependencies on integration sequence L
% Configuration defined L
% system configuration frozen L No. of faults reported L % System integrated L 100
Verification and Validation
No. of SRs covered by Test Procedures L
No. of SRs with compliance L % test complete L
% of certification complete L No. of concessions F 99
Benefits
Specific Review Agendas◦ Clarity of expectation at all review◦ In the same form
Common graphical presentation◦ Supports immediate planning ◦ Supports trend analysis◦ Project to project comparison
Connects to Project and Programme Management◦ Support to Earned Value Management
Unambiguous input to EVM/P3M processes Traceable evidence based reviews
◦ Removes anecdotal and subjective maturity assessment◦ Helps identify risks early
Relates to wider business context◦ Communications… Data Management… Decision Making…
GEMM capability
Questions
Dr Kevin Howard is Engineering Director at Optima Systems Consultancy providing strategic guidance to the business in its application of Systems Engineering across defence and civilian consultancy. He has been Vice President for Systems Engineering at AGT based in Zurich and head of Systems and Sensors for Thales.
Kevin is Member of IET and INCOSE, Visiting fellow at Bristol University and a Guest lecturer at Cranfield University.
He has reached this position through a practical engineering route in positions of Chief Engineer for various major programmes across defence and civilian world including armoured fighting vehicle, space borne radars, and software data fusion engines.
GEMM is based on this pragmatic and practical backdrop to systems engineering approach that is underpinned by a deep understand across a range of technologies from automotive mechanics, through electronics, RF and antennas to pure software. He is a strong proponent of Model Based Systems Engineering and GEMM is closely aligned to a Model Based approach. This is supported by a PhD in optimising complex systems by minimising their entropy.
Dr Kevin Howard