the eia-731 capability maturity model, & its relationship to other systems engineering standards...
TRANSCRIPT
Systems Engineering Capability Assessment
09.15 Introduction to session and speakers Paul Davies
Racal Thomson Defence
09.20 EIA-731 Capability Maturity Model and its relationship to other SE Standards
Paul Davies Racal Thomson Defence
09.45 UK MoD Experiences in the use of EIA-731 in Soothsayer contractor selection
John Keenan
DERA
10.05 Racal experiences from the Soothsayer Capability Assessment, and CMM-based Process Improvement
Dick Allen-Shalless
Racal Thomson Defence
10.30 Refreshment Break
10:45 Differences in practice between UK and US CMM-based Capability Assessment; can this lead to a more efficient assessment?
Valerie Gundrum,
Lockheed Martin
11.10 Beyond EIA 731 – moving towards integrated assessments with other disciplines (CMMI)
Hillary Sillitto,
Pilkington Optronics
11.35 Who’s doing what with capability assessments – a worldwide summary
Paul Davies
Racal Thomson Defence
11.45 Panel Discussion and Questions from the Floor All speakers, moderated by) Paul Davies
12.30 End of Session
The EIA-731 Capability Maturity Model, & its The EIA-731 Capability Maturity Model, & its Relationship to Other Systems Engineering Relationship to Other Systems Engineering StandardsStandards
Paul Davies
Thomson Racal Defence Ltd
ThemesThemesWhat Standards are ‘Out There’?Where did they come from?What topics do they cover?What’s their current status?Who uses them, and what for?Where are they going?Which one is best to use now?How do they support business process improvement?
The ‘Frameworks Quagmire’The ‘Frameworks Quagmire’ - - What Standards are out there?What Standards are out there?
SW-CMMMIL-Q -9858
Trillium Baldrige
IEEE Stds. 730,828 829, 830,1012,1016
1028,1058,1063ISO 15504(SPICE)
People CMM
IPD-CMM*
DOD IPPD
SECAMAF IPD Guide
SDCCR
SCE
NATO AQAP1,4,9
BS5750
MIL-STD-
498
DOD-STD -2167A
ISO/IEC12207
IEEE1220 ISO 10011
SDCE
SECM(EIA/IS 731)
EIA/IS 632
ISO 9000Series
EIA/IEEEJ-STD-016
IEEE/EIA 12207
MIL-STD-1679
IEEE 1074
TickITSSE-CMM
ISO 15288*
EQA
CMMI**
PSP
Q9000
DOD-STD-2168
FAA-iCMM
DO-178B
SW-CMM
SOFTWAREPRODUCTIVITYCONSORTIUM
Also see www.software.org/quagmire
* Not released** CMMI-SE/SW released IPPD not yet released
MIL-STD-499B*
SA-CMM
SE-CMM
EIA 632
The ‘Frameworks Quagmire’The ‘Frameworks Quagmire’ - - What Standards are out there?What Standards are out there?
SW-CMMMIL-Q -9858
Trillium Baldrige
IEEE Stds. 730,828 829, 830,1012,1016
1028,1058,1063ISO 15504(SPICE)
People CMM
IPD-CMM*
DOD IPPD
SECAMAF IPD Guide
SDCCR
SCE
NATO AQAP1,4,9
BS5750
MIL-STD-
498
DOD-STD -2167A
ISO/IEC12207
IEEE1220 ISO 10011
SDCE
SECM(EIA/IS 731)
EIA/IS 632
ISO 9000Series
EIA/IEEEJ-STD-016
IEEE/EIA 12207
MIL-STD-1679
IEEE 1074
TickITSSE-CMM
ISO 15288*
EQA
CMMI**
PSP
Q9000
DOD-STD-2168
FAA-iCMM
DO-178B
SW-CMM
SOFTWAREPRODUCTIVITYCONSORTIUM
Also see www.software.org/quagmire
* Not released** CMMI-SE/SW released IPPD not yet released
MIL-STD-499B*
SA-CMM
SE-CMM
EIA 632
Color Meaning of color
Red A Capability Maturity Model
Green A U.S. Government or military document
Purple An international standard
Blue Commercial, professional, or industrial association document (mostly U.S.)
Black Other
The ‘Frameworks Quagmire’The ‘Frameworks Quagmire’ - - What Standards are out there?What Standards are out there?
SW-CMMMIL-Q -9858
Trillium Baldrige
IEEE Stds. 730,828 829, 830,1012,1016
1028,1058,1063ISO 15504(SPICE)
People CMM
IPD-CMM*
DOD IPPD
SECAMAF IPD Guide
SDCCR
SCE
NATO AQAP1,4,9
BS5750
MIL-STD-
498
DOD-STD -2167A
ISO/IEC12207
IEEE1220 ISO 10011
SDCE
SE-CMMSECM(EIA/IS 731)
EIA/IS 632
ISO 9000Series
EIA/IEEEJ-STD-016
IEEE/EIA 12207
MIL-STD-1679
IEEE 1074
TickITSSE-CMM
ISO 15288*
EQA
CMMI**
PSP
Q9000
DOD-STD-2168
FAA-iCMM
DO-178B
SW-CMM
SOFTWAREPRODUCTIVITYCONSORTIUM
Also see www.software.org/quagmire
* Not released** CMMI-SE/SW released IPPD not yet released
Software Development and Documentation Standards, including Lifecycle models
The ‘Frameworks Quagmire’The ‘Frameworks Quagmire’ - - What Standards are out there?What Standards are out there?
SW-CMMMIL-Q -9858
Trillium Baldrige
IEEE Stds. 730,828 829, 830,1012,1016
1028,1058,1063ISO 15504(SPICE)
People CMM
IPD-CMM*
DOD IPPD
SECAMAF IPD Guide
SDCCR
SCE
NATO AQAP1,4,9
BS5750
MIL-STD-
498
DOD-STD -2167A
ISO/IEC12207
IEEE1220 ISO 10011
SDCE
SE-CMMSECM(EIA/IS 731)
EIA/IS 632
ISO 9000Series
EIA/IEEEJ-STD-016
IEEE/EIA 12207
MIL-STD-1679
IEEE 1074
TickITSSE-CMM
ISO 15288*
EQA
CMMI**
PSP
Q9000
DOD-STD-2168
FAA-iCMM
DO-178B
SW-CMM
SOFTWAREPRODUCTIVITYCONSORTIUM
Also see www.software.org/quagmire
* Not released** CMMI-SE/SW released IPPD not yet released
Quality, Quality Managementand Quality Audit Standards
The ‘Frameworks Quagmire’The ‘Frameworks Quagmire’ - - What Standards are out there?What Standards are out there?
SW-CMMMIL-Q -9858
Trillium Baldrige
IEEE Stds. 730,828 829, 830,1012,1016
1028,1058,1063ISO 15504(SPICE)
People CMM
IPD-CMM*
DOD IPPD
SECAMAF IPD Guide
SDCCR
SCE
NATO AQAP1,4,9
BS5750
MIL-STD-
498
DOD-STD -2167A
ISO/IEC12207
IEEE1220 ISO 10011
SDCE
SE-CMMSECM(EIA/IS 731)
EIA/IS 632
ISO 9000Series
EIA/IEEEJ-STD-016
IEEE/EIA 12207
MIL-STD-1679
IEEE 1074
TickITSSE-CMM
ISO 15288*
EQA
CMMI**
PSP
Q9000
DOD-STD-2168
FAA-iCMM
DO-178B
SW-CMM
SOFTWAREPRODUCTIVITYCONSORTIUM
Also see www.software.org/quagmire
* Not released** CMMI-SE/SW released IPPD not yet released
Integrated Product(& Process)
ImprovementStandards
The ‘Frameworks Quagmire’The ‘Frameworks Quagmire’ - - What Standards are out there?What Standards are out there?
SW-CMMMIL-Q -9858
Trillium Baldrige
IEEE Stds. 730,828 829, 830,1012,1016
1028,1058,1063ISO 15504(SPICE)
People CMM
IPD-CMM*
DOD IPPD
SECAMAF IPD Guide
SDCCR
SCE
NATO AQAP1,4,9
BS5750
MIL-STD-
498
DOD-STD -2167A
ISO/IEC12207
IEEE1220 ISO 10011
SDCE
SE-CMMSECM(EIA/IS 731)
EIA/IS 632
ISO 9000Series
EIA/IEEEJ-STD-016
IEEE/EIA 12207
MIL-STD-1679
IEEE 1074
TickITSSE-CMM
ISO 15288*
EQA
CMMI**
PSP
Q9000
DOD-STD-2168
FAA-iCMM
DO-178B
SW-CMM
SOFTWAREPRODUCTIVITYCONSORTIUM
Also see www.software.org/quagmire
* Not released** CMMI-SE/SW released IPPD not yet released
Quality-Related StandardsLinked to IPPD &
Business Capability
The ‘Frameworks Quagmire’The ‘Frameworks Quagmire’ - - What Standards are out there?What Standards are out there?
SW-CMMMIL-Q -9858
Trillium Baldrige
IEEE Stds. 730,828 829, 830,1012,1016
1028,1058,1063ISO 15504(SPICE)
People CMM
IPD-CMM*
DOD IPPD
SECAMAF IPD Guide
SDCCR
SCE
NATO AQAP1,4,9
BS5750
MIL-STD-
498
DOD-STD -2167A
ISO/IEC12207
IEEE1220 ISO 10011
SDCE
SE-CMMSECM(EIA/IS 731)
EIA/IS 632
ISO 9000Series
EIA/IEEEJ-STD-016
IEEE/EIA 12207
MIL-STD-1679
IEEE 1074
TickITSSE-CMM
ISO 15288*
EQA
CMMI**
PSP
Q9000
DOD-STD-2168
FAA-iCMM
DO-178B
SW-CMM
SOFTWAREPRODUCTIVITYCONSORTIUM
Also see www.software.org/quagmire
* Not released** CMMI-SE/SW released IPPD not yet released
Software CMM’s and Capability EvaluationStandardsSA-CMM
The ‘Frameworks Quagmire’The ‘Frameworks Quagmire’ - - What Standards are out there?What Standards are out there?
SW-CMMMIL-Q -9858
Trillium Baldrige
IEEE Stds. 730,828 829, 830,1012,1016
1028,1058,1063ISO 15504(SPICE)
People CMM
IPD-CMM*
DOD IPPD
SECAMAF IPD Guide
SDCCR
SCE
NATO AQAP1,4,9
BS5750
MIL-STD-
498
DOD-STD -2167A
ISO/IEC12207
IEEE1220 ISO 10011
SDCE
SE-CMMSECM(EIA/IS 731)
EIA/IS 632
ISO 9000Series
EIA/IEEEJ-STD-016
IEEE/EIA 12207
MIL-STD-1679
IEEE 1074
TickITSSE-CMM
ISO 15288*
EQA
CMMI**
PSP
Q9000
DOD-STD-2168
FAA-iCMM
DO-178B
SW-CMM
SOFTWAREPRODUCTIVITYCONSORTIUM
Also see www.software.org/quagmire
* Not released** CMMI-SE/SW released IPPD not yet released
Systems Engineering StandardsEIA 632
MIL-STD-499B*
The ‘Frameworks Quagmire’The ‘Frameworks Quagmire’ - - What Standards are out there?What Standards are out there?
SW-CMMMIL-Q -9858
Trillium Baldrige
IEEE Stds. 730,828 829, 830,1012,1016
1028,1058,1063ISO 15504(SPICE)
People CMM
IPD-CMM*
DOD IPPD
SECAMAF IPD Guide
SDCCR
SCE
NATO AQAP1,4,9
BS5750
MIL-STD-
498
DOD-STD -2167A
ISO/IEC12207
IEEE1220 ISO 10011
SDCE
EIA/IS 632
ISO 9000Series
EIA/IEEEJ-STD-016
IEEE/EIA 12207
MIL-STD-1679
IEEE 1074
TickITSSE-CMM
ISO 15288*
EQA
CMMI**
PSP
Q9000
DOD-STD-2168
FAA-iCMM
DO-178B
SW-CMM
SOFTWAREPRODUCTIVITYCONSORTIUM
Also see www.software.org/quagmire
* Not released** CMMI-SE/SW released IPPD not yet released
Systems EngineeringCapability Maturity Models(including Evaluation Methods)
SE-CMM SECM(EIA/IS 731)
The ‘Frameworks Quagmire’The ‘Frameworks Quagmire’ - - What Standards are out there?What Standards are out there?
SW-CMMMIL-Q -9858
Trillium Baldrige
IEEE Stds. 730,828 829, 830,1012,1016
1028,1058,1063
People CMM
IPD-CMM*
DOD IPPD
SECAMAF IPD Guide
SDCCR
SCE
NATO AQAP1,4,9
BS5750
MIL-STD-
498
DOD-STD -2167A
ISO/IEC12207
IEEE1220 ISO 10011
SDCE
SE-CMMSECM(EIA/IS 731)
EIA/IS 632
ISO 9000Series
EIA/IEEEJ-STD-016
IEEE/EIA 12207
MIL-STD-1679
IEEE 1074
TickITSSE-CMM
ISO 15288*
EQA
CMMI**
PSP
Q9000
DOD-STD-2168
FAA-iCMM
DO-178B
SW-CMM
SOFTWAREPRODUCTIVITYCONSORTIUM
Also see www.software.org/quagmire
* Not released** CMMI-SE/SW released IPPD not yet released
Integrated Capability Maturity Models(“the whole nine yards”)
Status of ‘Lead’ Standard in each FrameworkStatus of ‘Lead’ Standard in each Framework
Standard Framework PublicationDate
Expiration /Status
Purpose
MIL-STD-498
SoftwareDevelopmentandDocumentation
5 December1994
Canceledas ofJune1998
to establish uniform requirements forsoftware development anddocumentation. It specifies activitiesand products that may be required.The products are described in detail in22 Data Item Descriptions (DIDs).
SW-CMM CapabilityMaturity Model(CMM) forSoftware
February1993
Version 1.1 “CMM” is a model used byorganizations for appraising thematurity of their software processesand for identifying practices that willincrease the maturity of thoseprocesses. It was developed by theSoftware Engineering Institute, in cooperation with industryrepresentatives.
Status of ‘Lead’ Standard in each FrameworkStatus of ‘Lead’ Standard in each Framework
Standard Framework PublicationDate
Expiration /Status
Purpose
EIA - 632 SystemsEngineeringStandard:“Processes forEngineering aSystem”
November1998
Version 1.0 “to enable an organization to...engineer and produce quality productsand deliver them on time at anaffordable price... “ Also to benchmarkprojects / processes against therequirements of EIA 632 forengineering a system
SECM(EIA-731)
SystemsEngineeringCapabilityModel
January1999?
Version 1.0(Interimstandard forTrial-Use)
This model is a merger of the SE-CMM (from the Enterprise ProcessImprovement Collaboration - EPIC)and the SECAM (from INCOSE)
CMMI CapabilityMaturity ModelIntegration
September2000
Version 1.0 The CMMI effort will merge thesoftware, systems engineering,acquisition and IPD models.
What EIA 632 is aboutWhat EIA 632 is about
EIA 632 OverviewEIA 632 Overview
EIA 632 Process RequirementsEIA 632 Process Requirements
What EIA-731 is aboutWhat EIA-731 is aboutDefines Focus (Process) areas generally felt to be of
benefit for most businessesDefines capability levels for each FASuggests questions to ask to evaluate whether the
maturity level has been reached by the organisation performing the activity
Provides recommended scoring schemeProvides recommended conduct of an evaluationAllows tailoring of FA’s and questions to specific
business needs
How is EIA 731 arranged?How is EIA 731 arranged?SE Domain - 3 Categories
19 Focus AreasThemes - 1-5 per Focus area
Specific Practices within each theme
Capability Domain Generic Practices
Applied to any SE Domain Focus Area / Theme‘An activity which, when applied…, enhances the capability to perform those practices’Grouped by Maturity Levele.g. GP2.2 Verify Compliance with approved plans & processes...
Generic Attributes‘Effectiveness’ and ‘Value’ of products achieved by SE Domain ThemesDescriptions of Effectiveness & Value for each Maturity LevelApplicable to any Focus Area
SECM FAs for TC 1.0 SE Technical SECM FAs for TC 1.0 SE Technical FA 1.1 Define Stakeholder and System Level RequirementsFA 1.2 Define Technical ProblemFA 1.3 Define SolutionFA 1.4 Assess and SelectFA 1.5 Integrate SystemFA 1.6 Verify SystemFA 1.7 Validate System
SECM FAs for TC 2.0 SE Management SECM FAs for TC 2.0 SE Management FA 2.1 Plan and OrganizeFA 2.2 Monitor and ControlFA 2.3 Integrate DisciplinesFA 2.4 Coordinate with SuppliersFA 2.5 Manage RiskFA 2.6 Manage DataFA 2.7 Manage ConfigurationsFA 2.8 Ensure Quality
SECM FAs for TC 3.0 SE EnvironmentSECM FAs for TC 3.0 SE EnvironmentFA 3.1 Define and Improve the Systems Engineering
ProcessFA 3.2 Manage CompetencyFA 3.3 Manage TechnologyFA 3.4 Manage Systems Engineering Support
Environment
SECM Capability LevelsSECM Capability LevelsLevel 0 InitialLevel 1 PerformedLevel 2 ManagedLevel 3 DefinedLevel 4 MeasuredLevel 5 Optimizing
Level 0 “Initial”Level 0 “Initial”Level 0 “Initial”Level 0 “Initial”
Development Process characterised as
Chaotic or Ad Hoc
General Properties
Not all practices accepted as enabling Systems Engineering are performed
General failure to perform activities
No easily identifiable work products
No evidence tasks are accomplished
Level 1 “Performed”Level 1 “Performed”Level 1 “Performed”Level 1 “Performed”
Development Process characterised as
Informal
General Properties
All basic practices accepted as enabling Systems Engineering are performed
Planning and tracking of practices is inconsistent
Dependence upon individual’s knowledge (“heroes”)
Level 2 “Managed”: CharacteristicsLevel 2 “Managed”: CharacteristicsLevel 2 “Managed”: CharacteristicsLevel 2 “Managed”: Characteristics
Development Process characterised as
Formal
General Properties
Activities are planned, tracked and verified
Processes are defined but programme specific
Adherence to specified process is monitored and enforced
Level 3 “Defined”: CharacteristicsLevel 3 “Defined”: CharacteristicsLevel 3 “Defined”: CharacteristicsLevel 3 “Defined”: Characteristics
Development Process characterised as
Institutionalised / Established
General Properties
Standard SE development process across the organisation
Tailored if required for specific programmes
Process metrics are collected
Qualitative process improvement occurs
Levels 4 & 5: CharacteristicsLevels 4 & 5: CharacteristicsLevels 4 & 5: CharacteristicsLevels 4 & 5: Characteristics
Level 4 ProcessImproving
Measurable quality goals established for work products
Processes are quantitatively improved
Level 5 ProcessOptimised
Continuous quantitative process improvement
Which model is ‘Best’ to use?Which model is ‘Best’ to use?It depends which you are most comfortable with731 is very specific - perhaps good if you don’t know
where to start, and want a list of possibles to selectCMM’s give you a guideline for process improvement;
but perhaps assume a prior level of maturity?How do you want to measure yourself:
At intervals (staged)? Or continuously - harder work, but quicker feedback
Let’s hear the speakers…. And debate it afterwards!