we know less than you think (but we do know something)

We Know Less Than You Think

(But We Do Know Something)

Copyright © 2009 Gregory V. Wilson.This presentation may be freely used and distributed with attribution.

Prof. Greg WilsonDept. of Computer Science

University of [email protected]

Computer “Science” (?)

http://www.third-bit.com 00001

Some of computer science is applied mathematics

But what kind of “science” is software engineering?– Most of it isn’t (yet?) as rigorous as

bridge design– Practitioners would object to being

called alchemists, organizational psychologists, or cheerleaders

Show Me

Shaw 1990: “Prospects for a Discipline of Software Engineering”– Called for evidence-based software

engineering– Built on empirical work since the 1970s

by Boehm, Basili, Glass, and othersEmpirical studies now expected by

top journals and conferences


There’s a Lot of Folklore…“The best programmers are up to 28

times more productive than the worst.” (or 40, or 100, or…)– Trace it back: [SAC68] had 12 (self-

taught) subjects[BOE75]: best to worst is a factor of 5

– Borne out by other studies– Consistent with other fields


Non-Linearity

[WOO79]: 25% increase in problem complexity 100% increase in solution complexity– Interactions grow as N2

This can work in your favor– Cut 1/4 of your features, problem

because 1/2 as hard to solve


Glass’s Law

Any new tool or technique initially makes you less productive– Which is why so many senior students

program with Notepad[GLA99]: 5-35% improvement is

the best you can expect from changing tools– Ah, but techniques…


Project Failure

#1 cause: poor estimation– Short iterations cure this in the small,

but don’t help in the large#2 cause: unstable requirements

– But research code often doesn’t have “requirements”

See [VGE91], [COL95], [HOF01]


Boehm’s Curve


[BOE01] and many others

Re-Use

“It takes three times longer to build a reusable component than it does to build something used only once.”

“If more than 25% of a component has to be revised, it’s better to rewrite it from scratch.”

See e.g. [THO97]http://www.third-bit.com 01000

Lines of Code

“Productivity and reliability depend on the length of a program’s text, independent of language level.”

See [PRE00] for a recent experiment

However, machine-independent code has machine-independent performance


“Maintenance makes up 40-80% of the total cost of a software project.”– [BOE75] and many more since

But roughly 60% of “maintenance” is enhancement

And 30% of maintenance time is spent figuring out things work


Maintenance

Perceived Importance versus Understanding of Standard Software Engineering Concepts

52.049.7

79.3

45.4 46.6

59.4

46.1

30.0

46.444.1

81.6

52.4

60.2

45.0

31.1

22.9

5.6 5.6

-2.3-7.0

-13.6

14.4 15.0

7.1

-20

0

20

40

60

80

100

sw

requirements

sw design

sw constructionsw verification

sw testing

sw maintenance

sw product

mgmt.

sw project

mgmt.

Software Engineering Concept

Percent

good/expert understanding important/very important gap

Lack of documentation was the #1 complaint for over 40% of respondents, and in the top three for almost 80%

…but ranked second to last in list of where respondents spend their time


What Hurts?

Emphasis on grids and GPUs is inappropriate and unproductive– Improving skills will yield greater

dividends on shorter timescalesswc.scipy.org for current course

– software-carpentry.org for the next one

– We’ll be webcasting July 13-31


Thoughts


References

[BOE75] Boehm: “The High Cost of Software”[COL95] Cole: “Runaway Projects—Causes and Effects”[GLA99] Glass: “The Realities of Software Technology Payoffs”[HOF01] Hofmann and Lehner: “Requirements Engineering as a Success

Factor in Software Projects”[PRE00] Prechelt: “An Empirical Comparison of Seven Programming

Languages”[SAC68] Sackman et al: “Exploratory Experimental Studies Comparing

Online and Offline Programming Performances”[THO97] Thomas et al: “An Analysis of Errors in a Reuse-Oriented

Development Environment”[VGE91] Van Genuchten: “Why Is Software Late?”[WOO79] Woodfield: “An Experiment on Unit Increase in Problem

Complexity”

Recommended Reading

Doar Practical Development EnvironmentsFeathers Working Effectively with Legacy CodeFogel Producing Open Source SoftwareFord The Productive ProgrammerGlass Facts and Fallacies of Software EngineeringJohnson GUI BloopersKernighan almost anythingKniberg Scrum and XP From the TrenchesMak The Martian PrinciplesNygard Release It!Spinellis Code Reading and Code QualityWeiner Time, Love, Memory

01101http://www.third-bit.com