BE AMERICA’S BEST

Software Technology Support Center

The Problems in Software

Estimating

Dr. Randall JensenSoftware Technology Support Center

Hill AFB, UT

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Outline

Evolution of software estimating models

Software problems

Management is the issue

Data collection concerns

New models?

It’s all about models – “Existence is elsewhere”

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Project Uncertainty Principle

O G D E N A I R L O G I S T I C S C E N T E R

Software model genealogy

NordonIBM 1970

Putnam/SLIMUS Army/GE/QSM 1976

DotyRADC 1977

JS1,2,3CEI 1980

SEER-SEMGAI 1989

Seer II / SageSEI 1995

COCOMOTRW 1981

COCOMO IIUSC 1995

Validation

SeerHAC 1979

PRICE-SPrice 1977

TRUE-SPrice 2003

REVICUSAF 1991

Management

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Sage/SEER-SEM - Normal form

where K = life cycle cost (person years), Td = development time (years)

slocdtee TKCS

3dT

KD

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Common estimate dilemma

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Software problems are not new

Unreliable

Late delivery

Modification costs prohibitive

Impossible to maintain

Inadequate performance

Product exceeds budget costs

1968 CS Conference, Munich, Germany

KEY

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Historic note:

“More software projects have gone awry for lack of calendar time than for all other causes combined…”

F. P. Brooks, Jr., Mythical Man Month

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Common technology claim

If you use (new technology),Productivity will improve by an order of magnitude

AndDefects will reduce to zero

Note: None of the history-based estimating models (tools) support this claim, not even close.

Neither does history.

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

There is always HOPE

1960 1970 1980 1990

100

90

80

70

60

50

Ada

OOD

Structured Analysis

Structured DesignProcess Maturity

PWB

Structured Programming

3rd Generation Languages

PRODUCTIVITY

lppm

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

3 Dimensions of development

PROJECT

PROCESSPEOPLE

1975

1985

1995

1938

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Impact of management on cost

“Poor management can increase software costs more rapidly than any other factor…

Despite this variation, COCOMO does not include a factor for management quality, but instead provides estimates which assume that the project will be well managed [italics mine, RWJ]”

Boehm, B. W., Software Engineering Economics, (Englewood Cliffs, NJ: Prentice-Hall, Inc.), 1981, pg. 486

Note: On a scale of 0 to 10 well managed must rank approximately 3.4?

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Where would you focus effort?

3

11

17

64

TOOLS

PEOPLE

SYSTEMS

MANAGEMENT

Source: G. Weinberg, Quality Software Management, Vol. 3

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Biased research?

Tools People Systems Managers0

10

20

30

40

50

60

70

SEI Papers

Pe

rce

nt

Source: G. Weinberg, Quality Software Management, Vol. 3

O G D E N A I R L O G I S T I C S C E N T E R

Cost driver relative impact

Target ExperienceMultiple Sites

Develop Sys ExperienceMultiple Organizations

Resource LocationDevelopment Tools

Modern PracticesApplication Experience

Practices ExperienceStaff Capability

0 0.5 1 1.5 2 2.5

Relative Development Cost Impact

It takes people to implement processes and use technology.

April 21st, 2009

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

BIG productivity drivers

Analyst capability Management style Motivation Use of team methods Working environment Problem solving skills

Programmer capabilityApplication domain experience Use of modern practicesAutomated tool support

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Capability shift (environment)

Basic Technology Constant

1980 2005

5500 750065006200 85008000

8635(1990)

O G D E N A I R L O G I S T I C S C E N T E R

Historic data observations

Models are made from historic data More data is better Adequate historic data takes time to collect (Lots of time)

Repository vs Database Repository = a place where things may be stored for safekeeping

Data stored without concern for a formal definition of the data placed in the repository

SRDR is defined as a financial report that happens to contain technical information

Database = a collection of data arranged for ease and speed of retrieval

Requires consistency of entries Each data field must satisfy common definition

DCARC is a repository as was SMC database Data definitions?

BE AMERICA’S BEST 18

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Problems?

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

Popular estimating solution

Let’s create a new software model

Models are out of date

Models are inaccurate

Too difficult to use

Oh, really! Are you sure?

O G D E N A I R L O G I S T I C S C E N T E R

BE AMERICA’S BEST

New estimating models?

Lack of confidence in existing tools (current issues) Optimistic estimates Unacceptable estimates

Aging tools (?) Culture is constant New models require validation

New models -- No silver bullets Quality data (if, where, proprietary) New models require validation

New (or better) estimators? Experience Skill Integrity

O G D E N A I R L O G I S T I C S C E N T E R

EstimatingPrediction is very hard, especially when it’s about the

future