software cost estimation seth bowen samuel lee lance titchkosky
TRANSCRIPT
Software Cost Estimation
Seth Bowen
Samuel Lee
Lance Titchkosky
2
Outline
Introduction
Inputs and Outputs
Methods of Estimation
COCOMO
Conclusion
3
Cost Estimation Is Needed
55% of projects over budget 24 companies that developed large distributed
systems (1994) 53% of projects cost 189% more than
initial estimates Standish Group of 8,380 projects (1994)
4
Cost Estimation An approximate judgment of the costs for a
project Many variables
Often measured in terms of effort (i.e., person months/years)
Different development environments will determine which variables are included in the cost value Management costs Development costs
Training costs Quality assurance
Resources
5
Cost Estimation Affects
Planning and budgeting Requirements prioritization Schedule Resource allocation
Project management Personnel Tasks
6
Cost Estimation During the Software Life Cycle Cost estimation should be done throughout the
software life cycle to allow for refinement Need effective monitoring and control of the
software costs to verify and improve accuracy of estimates At appropriate level of detail Gathering data should not be difficult
Success of a cost estimate method is not necessarily the accuracy of the initial estimates, but rather the rate at which estimates converge to the actual cost
7
Who is the Estimator? Someone responsible for the implementation
Can compare previous projects in organization to current project
Usually experienced Someone from outside the organization
Can provide unbiased estimate Tend to use empirical methods of estimation
Difficulties: Lack of confidence that a model will outperform an
expert Lack of historical data to calibrate the model
8
General Steps and Remarks
Establish Plan What data should we gather Why are we gathering this data What do we hope to accomplish
Do cost estimation for initial requirements Decomposition
Use several methods There is no perfect technique If get wide variances in methods, then should re-
evaluate the information used to make estimates
9
General Steps and Remarks (cont.)
Do re-estimates during life cycle Make any required changes to
development Do a final assessment of cost estimation
at the end of the project
10
Software Cost Estimation Process Definition
A set of techniques and procedures that is used to derive the software cost estimate
Set of inputs to the process and then the process will use these inputs to generate the output
11
Inputs and Outputs to the Estimation Process
Classical view of software estimation process [Vigder/Kark]
12
Inputs and Outputs to the Estimation Process (Cont.)
Actual cost estimation process [Vigder/Kark]
13
Cost Estimation Accuracy
To determine how well a cost estimation model predicts
Assessing model performance Absolute Error = (Epred – Eact)
Percentage Error = (Epred – Eact) / Eact
Mean Magnitude of Relative Error
1n.
Epred Eact Eact
i1
in
i
14
Cost Estimation Methods
Algorithmic (Parametric) model Expert Judgment (Expertise Based) Top – Down Bottom – Up Estimation by Analogy Price to Win Estimation
15
Algorithmic (Parametric) Model
Use of mathematical equations to perform software estimation
Equations are based on theory or historical data Use input such as SLOC, number of functions to
perform and other cost drivers Accuracy of model can be improved by
calibrating the model to the specific environment
16
Algorithmic (Parametric) Model (Cont.) Examples:
COCOMO (COnstructive COst MOdel) Developed by Boehm in 1981 Became one of the most popular and most transparent cost model Mathematical model based on the data from 63 historical software
project COCOMO II
Published in 1995 To address issue on non-sequential and rapid development process
models, reengineering, reuse driven approaches, object oriented approach etc
Has three submodels – application composition, early design and post-architecture
17
Algorithmic (Parametric) Model (Cont.)
Putnam’s software life-cycle model (SLIM) Developed in the late 1970s Based on the Putnam’s analysis of the life-cycle in
terms of a so-called Rayleigh distribution of project personnel level versus time.
Quantitative software management developed three tools : SLIM-Estimate, SLIM-Control and SLIM-Metrics.
18
Algorithmic (Parametric) Model (Cont.) Advantages
Generate repeatable estimations Easy to modify input data Easy to refine and customize formulas Objectively calibrated to experience
Disadvantages Unable to deal with exceptional conditions Some experience and factors can not be quantified Sometimes algorithms may be proprietary
19
Expert Judgment
Capture the knowledge and experience of the practitioners and providing estimates based upon all the projects to which the expert participated.
Examples Delphi
Developed by Rand Corporation in 1940 where participants are involved in two assessment rounds.
Work Breakdown Structure (WBS) A way of organizing project element into a hierarchy that
simplifies the task of budget estimation and control
20
Expert Judgment (Cont.)
Advantages Useful in the absence of quantified, empirical data. Can factor in differences between past project
experiences and requirements of the proposed project Can factor in impacts caused by new technologies,
applications and languages. Disadvantages
Estimate is only as good expert’s opinion Hard to document the factors used by the experts
21
Top - Down
Also called Macro Model Derived from the global properties of the
product and then partitioned into various low level components
Example – Putnam model
22
Top – Down (Cont.)
Advantages Requires minimal project detail Usually faster and easier to implement Focus on system level activities
Disadvantages Tend to overlook low level components No detailed basis
23
Bottom - Up
Cost of each software components is estimated and then combine the results to arrive the total cost for the project
The goal is to construct the estimate of the system from the knowledge accumulated about the small software components and their interactions
An example – COCOMO’s detailed model
24
Bottom – Up (Cont.)
Advantages More stable More detailed Allow each software group to hand an estimate
Disadvantages May overlook system level costs More time consuming
25
Estimation by Analogy
Comparing the proposed project to previously completed similar project in the same application domain
Actual data from the completed projects are extrapolated
Can be used either at system or component level
26
Estimation by Analogy (Cont.)
Advantages Based on actual project data
Disadvantages Impossible if no comparable project had been
tackled in the past. How well does the previous project represent
this one
27
Price to Win Estimation
Price believed necessary to win the contract
Advantages Often rewarded with the contract
Disadvantages Time and money run out before the job is
done
28
COCOMO 81 COCOMO stands for COnstructive
COst MOdel It is an open system First published by
Dr Barry Bohem in 1981 Worked quite well for projects in the
80’s and early 90’s Could estimate results within ~20% of
the actual values 68% of the time
29
COCOMO 81 COCOMO has three different models (each one
increasing with detail and accuracy): Basic, applied early in a project Intermediate, applied after requirements are specified. Advanced, applied after design is complete
COCOMO has three different modes: Organic – “relatively small software teams develop
software in a highly familiar, in-house environment” [Bohem]
Embedded – operate within tight constraints, product is strongly tied to “complex of hardware, software, regulations, and operational procedures” [Bohem]
Semi-detached – intermediate stage somewhere between organic and embedded. Usually up to 300 KDSI
30
COCOMO 81 COCOMO uses two equations to calculate effort in man months
(MM) and the number on months estimated for project (TDEV) MM is based on the number of thousand lines of delivered
instructions/source (KDSI) MM = a(KDSI)b * EAF TDEV = c(MM)d
EAF is the Effort Adjustment Factor derived from the Cost Drivers, EAF for the basic model is 1
The values for a, b, c, and d differ depending on which mode you are using
Mode a b c d
Organic 2.4 1.05 2.5 0.38
Semi-detached 3.0 1.12 2.5 0.35
Embedded 3.6 1.20 2.5 0.32
31
COCOMO 81
A simple example:Project is a flight control system (mission critical) with
310,000 DSI in embedded mode Reliability must be very high (RELY=1.40). So we can
calculate: Effort = 1.40*3.6*(319)1.20 = 5093 MM Schedule = 2.5*(5093)0.32 = 38.4 months Average Staffing = 5093 MM/38.4 months = 133 FSP
32
COCOMO II
Main objectives of COCOMO II: To develop a software cost and schedule
estimation model tuned to the life cycle practices of the 1990’s and 2000’s
To develop software cost database and tool support capabilities for continuous model improvement
From “Cost Models for Future Software Life Cycle Processes: COCOMO 2.0," Annals of Software Engineering, (1995).
33
COCOMO II
Has three different models: The Application Composition Model
Good for projects built using rapid application development tools (GUI-builders etc)
The Early Design Model This model can get rough estimates before the entire
architecture has been decided The Post-Architecture Model
Most detailed model, used after overall architecture has been decided on
34
COCOMO II Differences
The exponent value b in the effort equation is replaced with a variable value based on five scale factors rather then constants
Size of project can be listed as object points, function points or source lines of code (SLOC).
EAF is calculated from seventeen cost drivers better suited for today's methods, COCOMO81 has fifteen
A breakage rating has been added to address volatility of system
35
COCOMO II Calibration
For COCOMO II results to be accurate the model must be calibrated
Calibration requires that all cost driver parameters be adjusted
Requires lots of data, usually more then one company has
The plan was to release calibrations each year but so far only two calibrations have been done (II.1997, II.1998)
Users can submit data from their own projects to be used in future calibrations
36
Importance of Calibration
Proper calibration is very important The original COCOMO II.1997 could
estimate within 20% of the actual values 46% of the time. This was based on 83 data points.
The recalibration for COCOMO II.1998 could estimate within 30% of the actual values 75% of the time. This was based on 161 data points.
37
Is COCOMO the Best?
COCOMO is the most popular method however for any software cost estimation you should really use more then one method
Best to use another method that differs significantly from COCOMO so your project is examined from more then one angle
Even companies that sell COCOMO based products recommend using more then one method. Softstar (creators of Costar) will even provide you with contact information for their competitor’s products
38
COCOMO Conclusions
COCOMO is the most popular software cost estimation method
Easy to do, small estimates can be done by hand
USC has a free graphical version available for download
Many different commercial version based on COCOMO – they supply support and more data, but at a price
39
Conclusions
Project costs are being poorly estimated The accuracy of cost estimation has to be
improved Data collection Use of tools
Use several methods of estimation
40
References Boehm B., Clark B., Horowitz E., Madachy R., Shelby R., Westland C.
(1995). Cost Models for Future Software Life Cycle Processes: COCOMO 2.0, Annals of Software Engineering. http://sunset.usc.edu/research/COCOMOII/Docs/stc.pdf.
Boehm B., Clark B., Horowitz E., Madachy R., Shelby R., Westland C. (1995). An Overview of the COCOMO 2.0 Software Cost Model. http://sunset.usc.edu/research/COCOMOII/Docs/stc.pdf.
Boehm B., Chulani S., Clark B. (1997). Calibration Results of COCOMO II.1997. http://sunset.usc.edu/publications/TECHRPTS/1998/usccse98-502/CalPostArch.pdf.
Boehm B., Chulani S., Clark B. (1997). Calibrating the COCOMO II Post Architecture Model. http://sunset.usc.edu/Research_Group/Sunita/down/calpap.pdf.
Boehm B., Chulani S., Reifer D., The Rosetta Stone: Making COCOMO 81 Files Work With COCOMO II. http://sunset.usc.edu/publications/TECHRPTS/1998/usccse98-516/usccse98-516.pdf.
Chulani, S. (1998). Software Development Cost Estimation Approaches – A Survey. IBM Research.
Humphrey, W.S. (1990). Managing the Software Process. Addison-Wesley Publishing Company, New York, NY.
41
References Hussein, A. (2001). Introduction to Software Process Management.
University of Calgary, Calgary, Canada. http://sern.ucalgary.ca/courses/SENG/621/W01/intro.ppt.
Londeix, B. (1987). Cost Estimation for Software Development. Addison-Wesley Publishing Company, New York, NY.
Pressman, R.S. (2001). Software Engineering: A Practitioner’s Approach. McGraw-Hill Higher Education, New York, NY.
Vigder, M. R. and Kark, A. W. (1994). Software Cost Estimation and Control. Software Engineering Institute for Information Technology. http://wwwsel.iit.nrc.ca/seldocs/cpdocs/NRC37116.pdf.
Wu, L. (1997). The comparison of the Software Cost Estimating Methods. University of Calgary, Calgary, Canada. http://sern.ucalgary.ca/courses/seng/621/W97/wul/seng621_11.html.
Basic COCOMO Software Cost Model. http://www.jsc.nasa.gov/bu2/COCOMO.html.
COCOMO 2, Softstar Systems. http://www.softstarsystems.com/cocomo2.htm.
Answers to Frequently Asked Questions, Softstar Systems. http://www.softstarsystems.com/faq.htm.
42
Questions and Discussion