march 2002 cosysmo: constructive systems engineering cost model ricardo valerdi usc annual research...
Post on 19-Dec-2015
223 views
TRANSCRIPT
March 2002
COSYSMO: COnstructive SYStems Engineering Cost MOdel
Ricardo Valerdi
USC Annual Research Review
March 11, 2002
March 2002 2
Outline• Background on COSYSMO
• EIA632
• Approach
• Delphi Survey
• Delphi Round 1 Results
• Analysis/Conclusions
• Lessons Learned/Improvements
• The Next Step
• Q & A
March 2002 3
“All models are wrong, but some of them
are useful”
- W. E. Deming
Source: www.deming.org
March 2002 4
What is it?
The purpose of the COSYSMO project
is to enhance the current capability of
the COCOMO II model by accounting
for costs that are outside the realm of
software engineering by introducing
system engineering drivers.
March 2002 5
The Challenge
To develop a preliminary model for estimating the cost impact of front-end System Engineering tasks in the design of software intensive systems.
These include system definition, integration, and test activities as defined in standard EIA632.
March 2002 6
Approach
• Begin with front-end costs of information systems engineering
• Follow 7-step modeling methodology– Steps 5,6, & 7 (gather data and refine loop)
• Use model parameters compiled by TRW, SAIC, Raytheon, and USC/CSE– System size (requirements, TPMs, I/F)– Effort drivers (maturity, cohesion, stability)
March 2002 7
COCOMO Suite
COCOMOII
COQUALMOCOPSEMO
CORADMO
COSYSMO
COPROMO
COCOTS
March 2002 8
COSYSMO Operational Concept
COSYSMO
SizeDrivers
CostDrivers
Effort
Duration
Calibration
# Requirements# Interfaces# TPM’s# Scenarios# Modes# Platforms# Algorithms
- 7 Application factors- 8 Team factors
WBS guidedBy EIA 632
COCOMO II-based model
March 2002 9
EIA63233 activities organized into 5 groups:
1. Acquisition and supply2. Technical management
• Planning process• Assessment process• Control process
3. System design• Requirements definition process• Solution definition process
4. Product realization5. Technical evaluation
March 2002 10
Delphi Survey
• 3 Sections:– Scope, Size, Cost
• Used to determine the range for size driver and effort multiplier ratings
• Identify the cost drivers to which effort is most sensitive to
• Reach consensus from systems engineering experts
March 2002 11
7 Size Drivers
1. Number of System Requirements 2. Number of Major Interfaces3. Number of Technical Performance
Measures4. Number of Operational Scenarios5. Number of Modes of Operation6. Number of Different Platforms7. Number of Unique Algorithms
March 2002 12
15 Cost Drivers
1. Requirements understanding 2. Architecture understanding 3. Level of service requirements,
criticality, difficulty 4. Legacy transition complexity 5. COTS assessment complexity 6. Platform difficulty 7. Required business process
reengineering
Application Factors (7)
March 2002 13
15 Cost Drivers (cont…)
1. Number and diversity of stakeholder communities
2. Stakeholder team cohesion
3. Personnel capability
4. Personal experience/continuity
5. Process maturity
6. Multisite coordination
7. Formality of deliverables
8. Tool support
Team Factors (8)
March 2002 14
Delphi Round 123 Surveys returned
Aerospace 2
Galorath 1
Lockheed Martin 8
Raytheon 4
SAIC 6
TRW 1
USC 1
March 2002 15
System Engineering Effort Per EIA Stage
StageSupplier Performance Technical ManagementRequirements DefinitionSolution DefinitionSystems AnalysisRequirements ValidationDesign Solution VerificationEnd Products Validation
Delphi5.2%
13.1%16.6%18.1%19.2%11.3%10.5%
6.6%
Suggested5%
15%15% 20%20%15%
5%5%
3.054.254.544.285.974.586.073.58
Std. Dev.
March 2002 16
Delphi Round 1 Highlights (cont.)
Range of sensitivity for Size Drivers
# A
lgo
rith
ms
# R
eq
uir
emen
ts
# In
terf
aces
# T
PM
’s
# S
cen
ario
s
# M
od
es
# P
latf
orm
s
5.57
Relative
Effort
1
2.232.54
2.212.10
6.48
6
4
2
March 2002 17
Two Most Sensitive Size Drivers
Suggested Rel. Effort
Delphi Respondents EMR
Rel. Effort
Standard Deviation
# Interfaces 4 5.57 1.80
# Algorithms 6 6.48 2.09
March 2002 18
Delphi Round 1 Highlights (cont.)
Range of sensitivity for Cost Drivers (Application Factors)
EMR
1.93
2.812.13
2.432.24
4
2
Req
uir
em
ents
un
d.
Arc
hit
ectu
re u
nd
.
Lev
el o
f se
rvic
e re
qs.
Leg
acy
tra
nsi
tio
n
CO
TS
Pla
tfo
rm d
iffi
cult
y
Bu
s. p
roce
ss r
een
g.
1.741.13
March 2002 19
Delphi Round 1 Highlights (cont.)
Range of sensitivity for Cost Drivers (Team Factors)
1.28
2.461.91 2.161.94
1.25
To
ol
sup
po
rt
Sta
keh
old
er c
om
m.
Sta
keh
old
er c
oh
esio
n
Per
son
nel
cap
ab
ilit
y
Per
son
al
exp
erie
nce
Pro
cess
mat
uri
ty
Mu
ltis
ite
coo
rd.
Fo
rmal
ity
of
del
iv.
1.841.78
EMR4
2
March 2002 20
Suggested EMR
Delphi Respondents EMR
MeanStandard Deviation
Arch. Under. 1.66 2.24 0.83
Reqs. Under. 1.73 2.43 0.70
Pers. Cap. 2.15 2.46 0.66Serv. Req. 2.5 2.81 0.67
Four Most Sensitive Cost Drivers
March 2002 21
Conclusions
Not only do we need to better manage requirements, we also need to manage:
1) # of Interfaces2) # of Algorithms3) Personnel Capability4) Level of service requirements, criticality,
difficulty 5) Level of understanding
“Control the controllables”
March 2002 22
Lessons Learned/Improvements
Lesson 1 – There are lots of people and groupsInterested in more precisely estimating system costs- And they are willing to help do it free.Lesson 2 – Currently, system engineering effort isestimated using activity-based costing heuristicsLesson 3 – When mounting a Delphi, clearly identify what you are trying to do.- Else, system engineers will attack you with a shotgunLesson 4 – We could use help in developing a betterdesigned template for Delphi instruments
March 2002 23
The Next Steps
• Incorporate suggestions from Delphi 1• Write report• Get Masters degree
• Data from completed systems will then be used to statistically confirm or deny initial ratings• Round 2 of Delphi
…in the future
March 2002 24
Special Thanks to:
• Advisors
Dr. Boehm, Dr. Axelband, Don Reifer
• Affiliates
Gary Hafen, Tony Jordano, Chris Miller, Karen Owens, Don Reifer, Garry Roedler, Evin Stump, Gary Thomas, Marilee Wheaton
March 2002 25
Where can I get more info?
• COSYSMO Working Group meeting
Thursday March 14th 12:00 - 5:00
• valerdi.com/cosysmo
March 2002 26
March 2002 27
Delphi Round 1 Participants1 Don Reifer USC2 Gary Thomas Raytheon3 John Rieff Raytheon4 Deke Dunlap Raytheon5 Michael McBride SAIC6 Tony Jordano SAIC7 James Evans LMCO8 Albert Cox TRW9 Abe Santiago Aerospace
10 Craig Hayenga LMCO11 Robert Kaufman SAIC12 Greg Kaszuba LMCO13 Phill Rowell SAIC14 Keith Young LMCO15 Charles Zumba SAIC16 Susan Ruth Aerospace17 Denton Tarbet Galorath18 Don Greenlee SAIC/INCOSE19 Jeffrey Shupp LMCO/INCOSE/Vertex Inc.20 David Lindstrom LMCO21 Garry Roedler LMCO22 Rocky Hudson LMCO23 John McDonald Raytheon