MTAT.03.244 Software Economics

Software Cost Estimation

Marlon Dumas

marlon.dumas ät ut . ee

1

What do we have until now •  Function points: a relatively objective (though

not fail-proof) way of estimating software size •  We can use this to:

•  Compare software systems in terms of size •  Track achieved progress in a software project

•  So if there are disputes, there is an “objective” way to determine how much to pay for delivered functionality

•  But we don’t know yet anything about costs…

2

For Discussion It is hopeless to accurately estimate software

costs. Most often than not, such estimates are wrong. So why should we bother?

We have 6 months and 10 analysts/developers, so it will take 6 months and 60 person-months. Why bother about estimating the cost?

3

4

Two lessons •  You can be uncertain of something, but you first

need to have something to be uncertain of •  You can change your plan, but you first need to

have something to change

5

What is a Good Estimate? “A good estimate is an estimate that provides a

clear enough view of the project reality to allow the project leadership to make good decisions about how to control the project to hit its targets.”

Steve McConnel Software Estimation: Demystifying the Black Art

6

Good (ARTfUL) Estimates •  Accurate (reasonably) •  Reliable (reasonably) •  Traceable: we should know where the effort will

go into and why? •  Updatable: it should be easy to “refine” with

new data

7

There are lies, dammed lies and statistics.

•  What about a method to estimate software costs from a high-level architecture, that is: •  within 20% of the actual size 50% of the time •  within 30% of the actual size 66% of the time •  Decomposes effort/cost into four project

phases •  Can we use it and how?

8

Cone of Uncertainty

Craig Larman Agile & Iterative Development

9

Effort Estimation Parkinson's Law?

If we have 5 person-years, it will take 5 person-years

Estimation by analogy (nearest neighbour) This project is 20% more complex than the previous one

Expert judgement (multi-source estimation) Wideband Delphi Planning Poker

Parametric (statistical) methods

10

Wide-Band Delphi Ask each team member their estimate

Apply personal experience, Look at completed projects, Extrapolate from modules known to date

Collect and share in a meeting: discuss why/how different people made their estimate

Repeat When stable, Size = (H + 4 X Ave. + L)/6

See: http://www.stellman-greene.com/ch03

11

Fun Variant: Planning Poker 1.  Product owner reads a user story, answers

questions 2.  Simultaneously: each team member pulls a

card with their estimate (e.g. in “story points”, “ideal” days, etc.)

3.  The holders of smallest and the largest estimate give reasons (others discuss too)

4.  Repeat from 2 until convergence 12

Statistical Methods •  Evidence-based scheduling:

www.joelonsoftware.com/items/2007/10/26.html

•  Fitting a function-points-to-effort function using history www.math.vu.nl/~x/ipm/ipm.pdf

•  Parametric cost models •  COCOMO II.2000 (Boehm et al.) •  Costar and Cost Xpert (based on COCOMO II) •  SLIM-Estimate •  Construx Estimate, KnowledgePlan, TruePlanning, ...

13

Six forms of software cost estimation (Caper Jones)

Project

Phase

Activity

Project

Phase

Activity 14

Principle of Parametric Estimation It took me one month to fully develop (end-to-end)

a small software application of 1000 LOC Can I develop an application of 10000 LOC in 10

months? I have four friends with similar experience as

mine, can we develop an application of 10000 LOC in 2 months?

Hints: Brook’s law, Farr & Nanus study

15

Non-Linear Productivity There is overwhelming evidence that, except for

simple projects, development effort goes up exponentially with size, so this is probably wrong: Effort = P x Size

This might be closer to the mark: Effort = A x M x SizeB

where A is a constant derived from historical data, and M is dependent on each project (effort multiplier), and B is dependent on the complexity of the project

16

Approach 1 - Discretize

17

Proceedings 5th Software Measurement European Forum, Milan 2008

129

It is evident that the “base productivity” can be often a value not very significant to

determine the final effort, but in any case you have to start by an initial value. The first effort driver in order to calibrate the “base productivity” is the functional dimension of the projects. Table 1 shows the “base productivity” for Java related to the functional dimension in FP: with the growth of dimension, the productivity decreases by two points; there are five classes of projects. The productivity is expressed in fp/person month.

Table 1

Applying these values to the dimension in FP of a project we obtain a base work effort of

the software development project in Month/person or in Days/person (usually we consider a month of 21 days).

This effort should cover all the activities concerning the software project according to the

RUP (Rational Unified Process) disciplines. For example if there is a project valued 574 FP, from the table above we obtain a “base

productivity” of 16 fp/month and thus a “base work effort” of 35.87 Month/person(574/16) or 753 days/person.

In the case of EMP it is to consider not only the functional dimension of EMP (in Nesma

FP) but also the functional dimension of the application on which the maintenance will be applied to for selecting the base productivity, (for example if we had an EMP for our projects of 574 FP, we will choose a productivity of 16 fp/month, independently of the EMP Nesma functional dimension in FP).

4. The “Productivity Factors”

Every project has its own characteristics, so it is clear that the range of productivity can be

very large! The most critical step in evaluating the effort of the project is to consider all the factors that can influence the productivity. Through the Functional Requirements it is possible to measure the functional dimension in Function Points.

The Technical and Quality requirements should give us information about the factors that influence productivity. But how can we measure them? And, above all, which are these factors?

In 2003 in CSI Piemonte was done an investigation to find them and to evaluate the impact that they have on productivity (growing or decreasing it). To determine the impact of productivity factors it was chosen an approach like COCOMO [6] (the productivity factor is the multiplication of several cost drivers).

Cf. •  ISBSG.org •  Capers Jones database

Source: Function Point: how to transform them in effort? by Gianfranco Lanza

Quick Exercise •  An application for managing construction

equipment rentals has an estimated size of 400 FPs

•  How much will it cost?

18

Approach 2 – Interpolate Nonlinear relationship when exponent > 1

0

2 0 0 0

4 0 0 0

6 0 0 0

8 0 0 0

1 0 0 0 0

1 2 0 0 0

1 4 0 0 0

1 6 0 0 0

0 5 0 0 1 0 0 0

K S L O C

Pe

rso

n M

on

ths B = 1 .2 2 6

B = 1 .0 0

B = 0 .9 1

19

COCOMO - Constructive Cost Model

•  Developed at USC (Barry Boehm et al.) based on a database of 63-161 projects

•  First version of COCOMO (now COCOMO 81) Most recent version COCOMO II.2000

•  Based on statistical model building (fitting actual data to equation)

•  Not very accurate with default database - should be calibrated based on company-specific historical data

20

COCOMO II Designed for an iterative development method

(MBASE) More refined set of cost drivers (6-17) Multiple exponential scale drivers:

PM = a x Sizeb x Π EMi (i = 1 to 6 or 17) where a = 2.94

b = 0.91 + 0.01 x Σ SFj (j = 1 to 5)

21

COCOMO II models COCOMO II incorporates a range of sub-models that produce

increasingly detailed software estimates. Sub-models in COCOMO II:

Application composition model. Used when software is composed from existing parts.

Early design model. Used when requirements are available but design has not yet started (6 cost drivers).

Reuse model. Used to compute the effort of integrating reusable components.

Post-architecture model. Used once the system architecture has been designed and more information about the system is available (17 cost drivers).

From I. Sommerville’s Software Engineering 22

Use of COCOMO II models

From I. Sommerville’s Software Engineering 23

Cost Factors Significant factors of development cost:

scale drivers are sources of exponential effort variation

cost drivers are sources of linear effort variation product, platform, personnel and project attributes effort multipliers associated with cost driver ratings

Defined to be as objective as possible Each factor is rated between very low and very

high per rating guidelines relevant effort multipliers adjust the cost up or down

24

Scale Drivers Precedentedness (PREC)

Degree to which system is new/past experience applies

Development Flexibility (FLEX) Need to conform with specified requirements

Architecture/Risk Resolution (RESL) Degree of design thoroughness and risk elimination

Team Cohesion (TEAM) Need to synchronize stakeholders and minimize conflict

Process Maturity (PMAT) SEI CMM process maturity rating 25

Scale Factors Sum scale factors SFi across all of the factors

to determine a scale exponent, B, using B = .91 + .01 Σ SFi

Scale Factors (Wi) Very Low Low Nominal High Very High Extra High

Precedentedness(PREC)

thoroughlyunprecedented

largelyunprecedented

somewhatunprecedented

generallyfamiliar

largelyfamiliar

throughlyfamiliar

DevelopmentFlexibility (FLEX)

rigorous occasionalrelaxation

somerelaxation

generalconformity

someconformity

generalgoals

Architecture/RiskResolution (RESL)*

little (20%) some (40%) often (60%) generally(75%)

mostly(90%)

full (100%)

Team Cohesion(TEAM)

very difficultinteractions

some difficultinteractions

basicallycooperativeinteractions

largelycooperative

highlycooperative

seamlessinteractions

Process Maturity(PMAT)

Weighted average of “Yes” answers to CMM Maturity Questionnaire

* % significant module interfaces specified, % significant risks eliminated

26

Precedentedness (PREC) and Development Flexibility (FLEX)

Feature Very Low Nominal / High Extra High

PrecedentednessOrganizational understanding of productobjectives

General Considerable Thorough

Experience in working with related softwaresystems

Moderate Considerable Extensive

Concurrent development of associated newhardware and operational procedures

Extensive Moderate Some

Need for innovative data processingarchitectures, algorithms

Considerable Some Minimal

Development FlexibilityNeed for software conformance with pre-established requirements

Full Considerable Basic

Need for software conformance withexternal interface specifications

Full Considerable Basic

Premium on early completion High Medium Low

27

Architecture / Risk Resolution (RESL) Use a subjective weighted average of:

Characteristic Very Low Low Nominal High Very High Extra High

Risk Management Plan identifies all criticalrisk items, establishes milestones forresolving them by PDR.

None Little Some Generally Mostly Fully

Schedule, budget, and internal milestonesthrough PDR compatible with RiskManagement Plan

None Little Some Generally Mostly Fully

Percent of development schedule devotedto establishing architecture, given generalproduct objectives

5 10 17 25 33 40

Percent of required top software architectsavailable to project

20 40 60 80 100 120

Tool support available for resolving riskitems, developing and verifyingarchitectural specs

None Little Some Good Strong Full

Level of uncertainty in Key architecturedrivers: mission, user interface, COTS,hardware, technology, performance.

Extreme Significant Considerable Some Little Very Little

Number and criticality of risk items > 10Critical

5-10Critical

2-4Critical

1Critical

> 5 Non-Critical

< 5 Non-Critical

28

Team Cohesion (TEAM) Use a subjective weighted average of the

characteristics to account for project turbulence and entropy due to difficulties in synchronizing the project's stakeholders.

Stakeholders include users, customers, developers, maintainers, interfacers, and others

Characteristic Very Low Low Nominal High Very High Extra High

Consistency of stakeholderobjectives and cultures

Little Some Basic Considerable Strong Full

Ability, willingness of stakeholders toaccommodate other stakeholders'objectives

Little Some Basic Considerable Strong Full

Experience of stakeholders inoperating as a team

None Little Little Basic Considerable Extensive

Stakeholder teambuilding to achieveshared vision and commitments

None Little Little Basic Considerable Extensive

29

Process Maturity (PMAT) Two methods based on the Software Engineering

Institute's Capability Maturity Model (CMM) Method 1:

Overall Maturity Level (CMM Level 1 through 5)

Method 2: Key Process Areas (see next slide)

30

Key Process Areas Decide the percentage of compliance for each of

the KPAs as determined by a judgment-based averaging across the goals for all 18 Key Process Areas.

Key Process Areas Almost Always(>90%)

Frequently(60-90%)

About Half(40-60%)

Occasionally(10-40%)

Rarely If Ever(<10%)

Does NotApply

Don'tKnow

1 RequirementsManagement

r r r r r r r

2 Software ProjectPlanning

r r r r r r r

3 Software ProjectTracking and Oversight

r r r r r r r

4 Software SubcontractManagement

r r r r r r r

(See COCOMO II Model Definition Manual for remaining details)

31

A company takes on a project in a new domain. The client has not defined the process to be used and has not allowed time for risk analysis. The company has a CMM level 2 rating. Precedenteness - new project – 0.4 Development flexibility - no client involvement - Very high – 0.1 Architecture/risk resolution - No risk analysis - V. Low – 0.5 Team cohesion - new team – nominal – 0.3 Process maturity - some control – nominal – 0.3

Scale factor = 1.17.

Example of Scale Factors

From I. Sommerville’s Software Engineering 32

Cost Drivers (Post-Architectural Model) Product Factors

Reliability (RELY) Data (DATA) Complexity (CPLX) Reusability (RUSE) Documentation (DOCU)

Platform Factors Time constraint (TIME) Storage constraint (STOR) Platform volatility (PVOL)

Personnel factors Analyst capability (ACAP) Program capability (PCAP) Applications experience (APEX) Platform experience (PLEX) Language and tool experience

(LTEX) Personnel continuity (PCON)

Project Factors Software tools (TOOL) Multisite development (SITE) Required schedule (SCED)

33

Example Cost Driver - Required Software Reliability (RELY)

Measures the extent to which the software must perform its intended function over a period of time.

Ask: what is the effect of a software failure? Very Low Low Nominal High Very High Extra High

RELY slightinconvenience

low, easilyrecoverablelosses

moderate,easilyrecoverablelosses

high financialloss

risk to humanlife

34

Example Effort Multiplier Values for RELY

Very Low Low High Very High

Slight Inconvenience

Low, Easily Recoverable

Losses

High Financial Loss Risk to Human

Life

1.15

0.75

0.88

1.39

1.0 Moderate, Easily

Recoverable Losses

Nominal

E.g. a highly reliable system costs 39% more than a nominally reliable system 1.39/1.0=1.39)

or a highly reliable system costs 85% more than a very low reliability system (1.39/.75=1.85)

35

COCOMO II – Schedule Estimation

D = c x Ed x SCED%/100

where c = 3.67

d = 0.33 + 0.2 x [b - 1.01]

SCED% = percentage of required schedule compression

36

Nominal versus Optimal Time

37

Cocomo II Exercise •  See separate handout •  Use the following resources:

•  COCOMO II Data Sheet •  Model Definition Manual •  Online cost Cocomo II calculator (see list of

Cocomo Resources under the course’s “Readings” page)

38

Software Cost vs Price Caution: All of the above is about effort and

schedule estimation From effort and schedule, one can estimate cost

Estimate technical effort cost based on PM x monthy total salary cost

Add licensing costs and overhead cost for administrative support, infrastructure, etc.

But cost ≠ price Price depends on many other factors:

Risk margin, requirements volatility, competitive advantage, market opportunity, need to win a bid… 39

Final Word of Caution COCOMO and similar models are just MODELS COCOMO comes calibrated by a set of projects

that might not reflect a particular project’s context

Should be combined with expert assessment – for example, combine Cocomo with estimates based on the Work Breakdown Structures

Cost estimation should be followed by continuous cost control (more on this next week)

40

Re: Homework 1 Effort and schedule estimation for your system can be done using one of the following methods: •  Using Cocomo II (post-architectural).

•  Explain your choice of cost and scale drivers •  Add a comment after your estimate to discuss how

credible you find the estimate given by Cocomo

•  Wideband Delphi using the method in http://www.stellman-greene.com/ch03

•  Or planning poker over each individual feature •  Provide table of feature and estimate in person-days

41