112 I E E E S O F T W A R E M a y / J u n e 2 0 0 1 0 7 4 0 - 7 4 5 9 / 0 1 / $ 1 0 . 0 0 2 0 0 1 I E E E

This months column is simply a collec-tion of what I consider to be factstruths, if you willabout software en-gineering. Im presenting this softwareengineering laundry list because fartoo many people who call themselves

software engineers, or computerscientists, or programmers, orwhatever nom du jour you prefer,either arent familiar with thesefacts or have forgotten them.

I dont expect you to agree withall these facts; some of them mighteven upset you. Great! Then wecan begin a dialog about whichfacts really are facts and which aremerely figments of my vivid loyalopposition imagination!

Enough preliminaries. Here are the mostfrequently forgotten fundamental facts aboutsoftware engineering. Some are of vital im-portancewe forget them at considerablerisk.

ComplexityC1. For every 10-percent increase in prob-

lem complexity, there is a 100-percent in-crease in the software solutions complexity.Thats not a condition to try to change (eventhough reducing complexity is always desir-able); thats just the way it is. (For one ex-planation of why this is so, see RD2 in thesection Requirements and design.)

PeopleP1. The most important factor in attack-

ing complexity is not the tools and tech-niques that programmers use but rather thequality of the programmers themselves.

P2. Good programmers are up to 30 timesbetter than mediocre programmers, accord-ing to individual differences research. Giventhat their pay is never commensurate, theyare the biggest bargains in the software field.

Tools and techniquesT1. Most software tool and technique im-

provements account for about a 5- to 30-per-cent increase in productivity and quality. Butat one time or another, most of these improve-ments have been claimed by someone to haveorder of magnitude (factor of 10) benefits.Hype is the plague on the house of software.

T2. Learning a new tool or technique ac-tually lowers programmer productivity andproduct quality initially. You achieve theeventual benefit only after overcoming thislearning curve.

T3. Therefore, adopting new tools andtechniques is worthwhile, but only if you (a)realistically view their value and (b) use pa-tience in measuring their benefits.

QualityQ1. Quality is a collection of attributes.

Various people define those attributes differ-

loyal opposition

Frequently Forgotten Fundamental Facts aboutSoftware EngineeringRobert L. Glass

E d i t o r : R o b e r t L . G l a s s C o m p u t i n g T r e n d s r g l a s s @ i n d i a n a . e d u

Continued on p. 110

110 I E E E S O F T W A R E M a y / J u n e 2 0 0 1

LOYAL OPPOSITION

ently, but a commonly accepted col-lection is portability, reliability, effi-ciency, human engineering, testability,understandability, and modifiability.

Q2. Quality is not the same as sat-isfying users, meeting requirements,or meeting cost and schedule targets.However, all these things have an in-teresting relationship: User satisfac-tion = quality product + meets re-quirements + delivered when needed+ appropriate cost.

Q3. Because quality is not simplyreliability, it is about much morethan software defects.

Q4. Trying to improve one qualityattribute often degrades another. Forexample, attempts to improve effi-ciency often degrade modifiability.

ReliabilityRE1. Error detection and removal

accounts for roughly 40 percent of de-velopment costs. Thus it is the mostimportant phase of the developmentlife cycle.

RE2. There are certain kinds ofsoftware errors that most program-mers make frequently. These includeoff-by-one indexing, definition orreference inconsistency, and omittingdeep design details. That is why, forexample, N-version programming,which attempts to create multiple diverse solutions through multipleprogrammers, can never completelyachieve its promise.

RE3. Software that a typical pro-grammer believes to be thoroughlytested has often had only about 55 to60 percent of its logic paths executed.Automated support, such as coverageanalyzers, can raise that to roughly85 to 90 percent. Testing at the 100-percent level is nearly impossible.

RE4. Even if 100-percent test cov-erage (see RE3) were possible, thatcriteria would be insufficient for test-ing. Roughly 35 percent of softwaredefects emerge from missing logicpaths, and another 40 percent arefrom the execution of a unique com-bination of logic paths. They will notbe caught by 100-percent coverage(100-percent coverage can, therefore,

potentially detect only about 25 per-cent of the errors!).

RE5. There is no single best ap-proach to software error removal. Acombination of several approaches,such as inspections and several kindsof testing and fault tolerance, is necessary.

RE6. (corollary to RE5) Softwarewill always contain residual defects,after even the most rigorous error re-moval. The goal is to minimize thenumber and especially the severity ofthose defects.

EfficiencyEF1. Efficiency is more often a

matter of good design than of goodcoding. So, if a project requires effi-ciency, efficiency must be consideredearly in the life cycle.

EF2. High-order language (HOL)code, with appropriate compiler op-timizations, can be made about 90percent as efficient as the comparableassembler code. But that statement ishighly task dependent; some tasksare much harder than others to codeefficiently in HOL.

EF3. There are trade-offs betweensize and time optimization. Often,improving one degrades the other.

MaintenanceM1. Quality and maintenance

have an interesting relationship (seeQ3 and Q4).

M2. Maintenance typically con-sumes about 40 to 80 percent (60percent average) of software costs.Therefore, it is probably the most im-portant life cycle phase.

M3. Enhancement is responsible

for roughly 60 percent of softwaremaintenance costs. Error correctionis roughly 17 percent. So, softwaremaintenance is largely about addingnew capability to old software, notabout fixing it.

M4. The previous two facts con-stitute what you could call the60/60 rule of software.

M5. Most software developmenttasks and software maintenance tasksare the sameexcept for the addi-tional maintenance task of under-standing the existing product. Thistask is the dominant maintenance ac-tivity, consuming roughly 30 percentof maintenance time. So, you couldclaim that maintenance is more diffi-cult than development.

Requirements and designRD1. One of the two most com-

mon causes of runaway projects isunstable requirements. (For the other,see ES1.)

RD2. When a project moves fromrequirements to design, the solutionprocesss complexity causes an explo-sion of derived requirements. Thelist of requirements for the designphase is often 50 times longer thanthe list of original requirements.

RD3. This requirements explosionis partly why it is difficult to imple-ment requirements traceability (trac-ing the original requirements throughthe artifacts of the succeeding life-cycle phases), even though everyoneagrees this is desirable.

RD4. A software problem seldomhas one best design solution. (BillCurtis has said that in a room fullof expert software designers, if anytwo agree, thats a majority!) Thatswhy, for example, trying to providereusable design solutions has so longresisted significant progress.

Reviews and inspectionsRI1. Rigorous reviews commonly

remove up to 90 percent of errorsfrom a software product before thefirst test case is run. (Many researchfindings support this; of course, itsextremely difficult to know whenyouve found 100 percent of a soft-ware products errors!)

Trying to improve one quality attribute

often degrades another. For example,attempts to improve

efficiency often degrade modifiability.

Continued from p. 112

M a y / J u n e 2 0 0 1 I E E E S O F T W A R E 111

LOYAL OPPOSITION

RI2. Rigorous reviews are moreeffective, and more cost effective,than any other error-removal strat-egy, including testing. But they can-not and should not replace testing(see RE5).

RI3. Rigorous reviews are ex-tremely challenging to do well, andmost organizations do not do them,at least not for 100 percent of theirsoftware artifacts.

RI4. Post-delivery reviews are gen-erally acknowledged to be important,both for determining customer satis-faction and for process improvement,but most organizations do not per-form them. By the time such reviewsshould be held (three to 12 monthsafter delivery), potential review par-ticipants have generally scattered toother projects.

ReuseREU1. Reuse-in-the-small (libraries

of subroutines) began nearly 50 yearsago and is a well-solved problem.

REU2. Reuse-in-the-large (compo-nents) remains largely unsolved, eventhough everyone agrees it is impor-tant and desirable.

REU3. Disagreement exists aboutwhy reuse-in-the-large is unsolved, al-though most agree that it is a man-agement, not technology, problem(will, not skill). (Others say that find-ing sufficiently common subproblemsacross programming tasks is difficult.This would make reuse-in-the-large aproblem inherent in the nature ofsoftware and the problems it solves,and thus relatively unsolvable).

REU4. Reuse-in-the-large worksbest in families of related systems,and thus is domain dependent. Thisnarrows its potential applicability.

REU5. Pattern reuse is one solu-tion to the problems inherent in codereuse.

EstimationES1. One of the two most com-

mon causes of runaway projects isoptimistic estimation. (For the other,see RD1.)

ES2. Most software estimates areperformed at the beginning of the lifecycle. This makes sense until we realize

that this occurs before the require-ments phase and thus before the prob-lem is understood. Estimation there-fore usually occurs at the wrong time.

ES3. Most software estimates aremade, according to several researchers,by either upper management or mar-keting, not by the people who willbuild the software or by their man-agers. Therefore, the wrong people aredoing estimation.

ES4. Software estimates are rarelyadjusted as the project proceeds. So,those estimates done at the wrongtime by the wrong people are usuallynot corrected.

ES5. Because estimates are sofaulty, there is little reason to be con-cerned when software projects do

not meet cost or schedule targets. Buteveryone is concerned anyway!

ES6. In one study of a project thatfailed to meet its estimates, the man-agement saw the project as a failure,but the technical participants saw itas the most successful project they

had ever worked on! This illustratesthe disconnect regarding the role ofestimation, and project success, be-tween management and technolo-gists. Given the previous facts, that ishardly surprising.

ES7. Pressure to achieve estima-tion targets is common and tends tocause programmers to skip good soft-ware process. This constitutes an ab-surd result done for an absurd reason.

ResearchRES1. Many software researchers

advocate rather than investigate. Asa result, (a) some advocated conceptsare worth less than their advocatesbelieve and (b) there is a shortage ofevaluative research to help determinethe actual value of new tools andtechniques.

T here, thats my two cents worth ofsoftware engineering fundamentalfacts. What are yours? I expect, ifwe can get a dialog going here, thatthere are a lot of similar facts that Ihave forgottenor am not aware of.Im especially eager to hear what ad-ditional facts you can contribute.

And, of course, I realize that somewill disagree (perhaps even violently!)with some of the facts Ive presented.I want to hear about that as well.

Robert L. Glass is the editor of Elseviers Journal ofSystems and Software and the publisher and editor of The Soft-ware Practitioner newsletter. Contact him at rglass@indiana.edu;hed be pleased to hear from you.

Pressure to achieveestimation targets iscommon and tends to

cause programmers toskip good software

process. This constitutesan absurd result donefor an absurd reason.

Copyright and reprint permission: Copyright 2001 by the Institute of Electrical and Electronics Engi-neers, Inc. All rights reserved. Abstracting is permitted with credit to the source. Libraries are permitted tophotocopy beyond the limits of US copyright law for private use of patrons those post-1977 articles thatcarry a code at the bottom of the first page, provided the per-copy fee indicated in the code is paid throughthe Copyright Clearance Center, 222 Rosewood Dr., Danvers, MA 01923. For copying, reprint, or re-publication permission, write to Copyright and Permissions Dept., IEEE Publications Admin., 445 HoesLn., Piscataway, NJ 08855-1331.

Circulation: IEEE Software (ISSN 0740-7459) is published bimonthly by the IEEE Computer Society. IEEEheadquarters: Three Park Ave., 17th Floor, New York, NY 10016-5997. IEEE Computer Society PublicationsOffice: 10662 Los Vaqueros Cir., PO Box 3014, Los Alamitos, CA 90720-1314; (714) 821-8380; fax (714)821-4010. IEEE Computer Society headquarters: 1730 Massachusetts Ave. NW, Washington, DC 20036-1903. Subscription rates: IEEE Computer Society members get the lowest rates and choice of media option$40/32/52 US print/electronic/combination; go to http://computer.org/subscribe to order and for more infor-mation on other subscription prices. Back issues: $10 for members, $20 for nonmembers. This magazine isavailable on microfiche.

Postmaster: Send undelivered copies and address changes to Circulation Dept., IEEE Software, POBox 3014, Los Alamitos, CA 90720-1314. Periodicals Postage Paid at New York, NY, and at addi-tional mailing offices. Canadian GST #125634188. Canada Post Publications Mail Product (Cana-dian Distribution) Sales Agreement Number 0487805. Printed in the USA.