why software is so bad - chapter 1 - maple bug lists ... software is so bad ..... and what’s being...

Why software is so bad ...... and what’s being done to fix it

By Charles C. MannTECHNOLOGY REVIEW

June 17 - It?s one of the oldest jokes on theInternet, endlessly forwarded from e-mailboxto e-mailbox. A software mogul - usually BillGates, but sometimes another -makes aspeech. ?If the automobile industry haddeveloped like the software industry,? themogul proclaims, ?we would all be driving$25 cars that get 1,000 miles to the gallon.?To which an automobile executive retorts,?Yeah, and if cars were like software, theywould crash twice a day for no reason, andwhen you called for service, they?d tell youto reinstall the engine.?

* Yellow Pages* Auctions atuBid

uBid* PersonalsChannel* ProfessionalTips* Newsletters* Shopping

THE JOKE ENCAPSULATES one of the greatpuzzles of contemporary technology. In anamazingly short time, software has become criticalto almost every aspect of modern life. From bankvaults to city stoplights, from telephone networks toDVD players, from automobile air bags to air trafficcontrol systems, the world around us is regulated bycode. Yet much software simply doesn’t workreliably: ask anyone who has watched a computerscreen flush blue, wiping out hours of effort. All toooften, software engineers say, code is bloated, ugly,inefficient and poorly designed; even whenprograms do function correctly, users find them toohard to understand. Groaning beneath the weight ofbricklike manuals, bookstore shelves across thenation testify to the perduring dysfunctionality ofsoftware. "Software’s simply terrible today," says WattsS. Humphrey, a fellow of Carnegie MellonUniversity’s Software Engineering Institute who haswritten several well-known books on softwarequality. "And it’s getting worse all the time." Goodsoftware, in Humphrey’s view, "is usable, reliable,defect free, cost effective and maintainable. Andsoftware now is none of those things. You can’t takesomething out of the box and know it’s going towork." Over the years, in the view of Edsger W.Dijkstra, an emeritus computer scientist at the

University of Texas at Austin, the average computeruser "has been served so poorly that he expects hissystem to crash all the time, and we witness amassive worldwide distribution of bug-riddensoftware for which we should be deeply ashamed." Jim McCarthy is more generous. The founder,with his wife Michele, of a software quality trainingcompany in Woodinville, WA, McCarthy believesthat "most software products have the necessaryfeatures to be worth buying and using andadopting." But, he allows, "only the extremeusefulness of software lets us tolerate its hugedeficiencies." McCarthy sometimes begins talks athis school with a PowerPoint presentation. The firstslide reads, "Most Software Sucks." GETTING WORSE, NOT BETTER It is difficult to overemphasize the uniquenessof software’s problems. When automotive engineersdiscuss the cars on the market, they don’t say thatvehicles today are no better than they were ten orfifteen years ago. The same is true for aeronauticalengineers: nobody claims that Boeing or Airbusmakes lousy planes. Nor do electrical engineerscomplain that chips and circuitry aren’t improving.As the engineering historian Henry Petroskisuggested in his 1992 book The Evolution of UsefulThings, continual refinement is the usual rule intechnology. Engineers constantly noticeshortcomings in their designs and fix them little bylittle, a process Petroski wryly described as "formfollows failure." As a result, products incrementallyimprove.

In the last 15 yearsalone, softwaredefects havewrecked a satellitelaunch, delayed anairport opening fora year, destroyed aMars mission,killed four Marinesin a helicoptercrash, induced aU.S. Navy ship todestroy a civilian

Software, alas, seems different. One wouldexpect a 45-million-line program like Windows XP,Microsoft’s newest operating system, to have a fewbugs. And software engineering is a newerdiscipline than mechanical or electrical engineering;the first real programs were created only 50 yearsago. But what’s surprising - astonishing, in fact - isthat many software engineers believe that softwarequality is not improving. If anything, they say, it’sgetting worse. It’s as if the cars Detroit produced in2002 were less reliable than those built in 1982. As software becomes increasingly important,the potential impact of bad code will increase tomatch, in the view of Peter G. Neumann, acomputer scientist at SRI International, a private

destroy a civilianairliner, and shutdown ambulancesystems in London,leading to as manyas 30 deaths.

R&D center in Menlo Park, CA. In the last 15 yearsalone, software defects have wrecked a Europeansatellite launch, delayed the opening of the hugelyexpensive Denver airport for a year, destroyed aNASA Mars mission, killed four marines in ahelicopter crash, induced a U.S. Navy ship todestroy a civilian airliner, and shut down ambulancesystems in London, leading to as many as 30 deaths.And because of our growing dependence on the Net,Neumann says, "We’re much worse off than wewere five years ago. The risks are worse and thedefenses are not as good. We’re goingbackwards-and that’s a scary thing." Some software companies are responding tothese criticisms by revamping their procedures;Microsoft, stung by charges that its products arebuggy, is publicly leading the way. Yet problemswith software quality have endured so long, andseem so intractably embedded in software culture,that some coders are beginning to think theunthinkable. To their own amazement, these peoplehave found themselves wondering if the realproblem with software is that not enough lawyersare involved. ‘IT’S TOTAL CHAOS’ Microsoft released Windows XP on Oct. 25,2001. That same day, in what may be a record, thecompany posted 18 megabytes of patches on itsWeb site: bug fixes, compatibility updates, andenhancements. Two patches fixed importantsecurity holes. Or rather, one of them did; the otherpatch didn’t work. Microsoft advised (and stilladvises) users to back up critical files beforeinstalling the patches. Buyers of the home versionof Windows XP, however, discovered that thesystem provided no way to restore these backupfiles if things went awry. As Microsoft’s onlineKnowledge Base blandly explained, the specialbackup floppy disks created by Windows XP Home"do not work with Windows XP Home."

Such slip-ups,critics say, are merelysurface lapses - signsthat the software’sdevelopers were toorushed or too carelessto fix obvious defects.

If you found this storyinteresting, here’s morefrom TechnologyReview:

The real problems liein software’s basicdesign, according toR. A. Downes ofRadsoft, a software

consulting firm. Or rather, its lack of design.Microsoft’s popular Visual Studio programmingsoftware is an example, to Downes’s way ofthinking. Simply placing the cursor over the VisualStudio window, Downes has found, invisiblybarrages the central processing unit with thousandsof unnecessary messages, even though the programis not doing anything. "It’s cataclysmic. ... It’s totalchaos," he complains. The issue, in the view of Dan Wallach, a

computer scientist at Rice University, is not thepointless churning of the processor - after all, henotes, "processing power is cheap." Nor isMicrosoft software especially flawed; critics oftenemploy the company’s products as examples morebecause they are familiar than because they areunusually bad. Instead, in Wallach’s view, theblooming, buzzing confusion in Visual Studio andso many other programs betrays how the techniquesfor writing software have failed to keep up with theexplosive increase in its complexity. Programmers write code in languages such asJava, C and C++, which can be read by humanbeings. Specialized programs known as "compilers"transform this code into the strings of ones andzeroes used by computers. Importantly, compilersrefuse to compile code with obvious problems - theyspit out error messages instead. Until the 1970s,compilers sat on large mainframes that were oftenbooked days or weeks in advance. Not wantingerrors to cause delay, coders - who in the early daystended to be trained as mathematicians or physicists- stayed late in their offices exhaustively checkingtheir work. Writing software was much like writingscientific papers. Rigor, documentation andpeer-review vetting were the custom. OVERWHELMED BY COMPLEXITY But as computers became widespread, attitudeschanged. Instead of meticulously planning code,programmers stayed up in caffeinated all-nighthacking sessions, constantly bouncing results off thecompiler. Again and again, the compiler would spit

Review: * Home page * Latest issue * Two free trial issues

back error messages; the programmers would fix themistakes one by one until the software compiledproperly. "The attitude today is that you can writeany sloppy piece of code and the compiler will rundiagnostics," says SRI’s Neumann. "If it doesn’t spitout an error message, it must be done correctly,right?" As programs grew in size and complexity,however, the limits of this "code and fix" approachbecame evident. On average, professional codersmake 100 to 150 errors in every thousand lines ofcode they write, according to a multiyear study of13,000 programs by Humphrey of Carnegie Mellon.Using Humphrey’s figures, the business operatingsystem Windows NT 4, with its 16 million lines ofcode, would thus have been written with about twomillion mistakes. Most would have been too smallto have any effect, but some - many thousands -would have caused serious problems.

Advertisement Naturally, Microsoft exhaustively tested NT 4before release, but "in almost any phase of testsyou’ll find less than half the defects," Humphreysays. If Microsoft had gone through four rounds oftesting, an expensive and time-consumingprocedure, the company would have found at most15 out of 16 bugs. "That’s going to leave you withsomething like five defects per thousand lines ofcode," Humphrey says. "Which is very low" - butthe software would still have as many as 80,000errors. Software engineers know that their code isoften riddled with lacunae, and they have long beensearching for new technologies to prevent them. Tomanage increasingly distended projects likeWindows, for example, they have developed avariety of techniques, of which perhaps the bestknown is component-based design. Just as housesare built with standardized two-by-fours andelectrical fittings, component-based programs arebuilt out of modular, interchangeable elements: anexample is the nearly identical menu bar atop everyWindows or Macintosh program. Such standardizedcomponents, according to Wallach, are not onlygood engineering practice, they are "the only wayyou can make something the size of MicrosoftOffice work at all." Microsoft, he says, was anearly, aggressive promoter of this approach - "it’sthe single best engineering decision they ever

made." INADEQUATE PLANNING CITED Unfortunately, critics say, the components areoften glued together with no real central plan-as ifcontractors tried to erect large structures with noblueprints. Incredibly, Humphrey says, the designfor large software projects is sometimes "nothingbut a couple bubbles on the back of an envelope."Worse, for marketing reasons companies wire asmany features as possible into new software,counteracting the benefits of modular construction.The most widespread example is Windows itself,which Bill Gates testified in an April session of theMicrosoft antitrust trial simply would not functionif customers removed individual components suchas browsers, file managers or e-mail programs."That’s an incredible claim," says Neumann. "Itmeans there’s no structure or architecture or rhymeor reason in the way they’ve built those systems,other than to make them as bundled as possible, sothat if you remove any part it will all fail."

The inadequatedesign in the finalproducts, critics argue,reflects inadequateplanning in theprocess of creatingthem. According to astudy by the StandishGroup, a consultingfirm in WestYarmouth, MA, U.S.commercial softwareprojects are so poorlyplanned and managedthat in 2000 almost aquarter were canceledoutright, creating no

final product. The canceled projects cost firms $67billion; overruns on other projects racked up another$21 billion. But because "code and fix" leads tosuch extensive, costly rounds of testing, evensuccessful projects can be wildly inefficient.Incredibly, software projects often devote 80percent of their budgets to repairing flaws theythemselves produced - a figure that does not includethe even more costly process of furnishing product

Tools and Toys

* High-tech scanners look forlies

* Review: ’Grand Theft Auto3’

* USB 2.0 takes on FireWire* Webcast royalty rates

halved* MSN, Verizon in broadband

alliance* Toshiba to offer iPod twin

for PCs* Integration key to chip

advances

support and developing patches for problems foundafter release. "System testing goes on for almost half theprocess," Humphrey says. And even when "theyfinally get it to work, there’s still no design." Inconsequence, the software can’t be updated orimproved with any assurance that the updates orimprovements won’t introduce major faults. "That’sthe way software is designed and built everywhere -it’s that way in spaceships, for God’s sake." IS SOFTWARE A SPECIAL CASE? The potential risks of bad software were grimlyillustrated between 1985 and 1987, when acomputer-controlled radiation therapy machinemanufactured by the government-backed AtomicEnergy of Canada massively overdosed patients inthe United States and Canada, killing at least three.In an exhaustive examination, Nancy Leveson, nowan MIT computer scientist, assigned much of theblame to the manufacturer’s inadequatesoftware-engineering practices. Because theprogram used to set radiation intensity was notdesigned or tested carefully, simple typing errorstriggered lethal blasts. Despite this tragic experience, similar machinesrunning software made by Multidata SystemsInternational, of St. Louis, massively overdosedpatients in Panama in 2000 and 2001, leading toeight more deaths. A team from the InternationalAtomic Energy Agency attributed the deaths to "theentering of data" in a way programmers had notanticipated. As Leveson notes, simple data-entryerrors should not have lethal consequences. So thisfailure, too, may be due to inadequate software.

?It?s like a carmanufacturersaying, ?This yearwe?re going tomake a rocket shipinstead of a car.?Of course they?llhave problems.? - CHARLES H.CONNELLformer principal engineer of LotusDevelopment

Programming experts tend to agree that suchdisasters are distressingly common. Consider theMars Climate Orbiter and the Polar Lander, bothdestroyed in 1999 by familiar, readily preventedcoding errors. But some argue that software simplycannot be judged, measured and improved in thesame way as other engineering products. "It’s just afact that there are things that other engineers can dothat we can’t do," says Shari Pfleeger, a seniorresearcher at the Rand think tank in Washington,DC, and author of the 1998 volume SoftwareEngineering: Theory and Practice. If a bridgesurvives a 500-kilogram weight and a

50,000-kilogram weight, Pfleeger notes, engineerscan assume that it will bear all the values between.With software, she says, "I can’t make thatassumption-I can’t interpolate." Moreover, software makers labor underextraordinary demands. Ford and General Motorshave been manufacturing the same product - afour-wheeled box with an internal-combustionengine - for decades. In consequence, says CharlesH. Connell, former principal engineer of LotusDevelopment (now part of IBM), they have beenable to improve their products incrementally. Butsoftware companies are constantly asked to createproducts - Web browsers in the early 1990s, newcell phone interfaces today - unlike anything seenbefore. "It’s like a car manufacturer saying, ‘Thisyear we’re going to make a rocket ship instead of acar,’" Connell says. "Of course they’ll haveproblems." "The classic dilemma in software is that peoplecontinually want more and more and more stuff,"says Nathan Myhrvold, former chief technologyofficer of Microsoft. Unfortunately, he notes, theconstant demand for novelty means that software isalways "in the bleeding-edge phase," when productsare inherently less reliable. In 1983, he says,Microsoft Word had only 27,000 lines of code."Trouble is, it didn’t do very much" - whichcustomers today wouldn’t accept. If Microsoft hadnot kept pumping up Word with new features, theproduct would no longer exist. "Users are tremendously non-self-aware,"Myhrvold adds. At Microsoft, he says, corporatecustomers often demanded that the companysimultaneously add new features and stop addingnew features. "Literally, I’ve heard it in a singlebreath, a single sentence. ‘We’re not sure why weshould upgrade to this new release - it has all thisstuff we don’t want - and when are you going to putin these three things?’ And you say, ‘Whaaat?’"Myhrvold’s sardonic summary: "Software sucksbecause users demand it to." HIGHER STANDARDS In January, Bill Gates issued a call to Microsoftemployees to make "reliable and secure" computingtheir "highest priority." In what the company billedas one of its most important initiatives in years,

Gates demanded that Microsoft "dramaticallyreduce" the number of defects in its products. Amonth later, the company took the unprecedentedstep of suspending all new code writing for almosttwo months. Instead, it gathered togetherprogrammers, a thousand at a time, for masstraining sessions on reliability and security. Usinghuge screens in a giant auditorium, companyexecutives displayed embarrassing snippets offlawed code produced by those in the audience. Gates’s initiative was apparently inspired by theblast of criticism that engulfed Microsoft in July2001 when a buffer overflow - a long-familiar typeof error - in its Internet Information ServicesWeb-server software let the Code Red wormvictimize thousands of its corporate clients. (In abuffer overflow, a program receives more data thanexpected - as if one filled in the space for a zip codewith a 50-digit number. In a computer, the extrainformation will spill into adjacent parts of memory,corrupting or overwriting the data there, unless it iscarefully blocked.) Two months later, the Nimdaworm exploited other flaws in the software to attackthousands more machines.

Battered by suchexperiences, softwaredevelopers arebecoming moreattentive to quality.Even as Gates wasrallying his troops,think tanks like theKestrel Institute, ofPalo Alto, CA, weredeveloping"correct-by-construction"programming tool kitsthat almost forcecoders to write reliableprograms (see "First

Aid for Faulty Code" ). At Microsoft itself,according to Amitabh Srivastava, head of the firm’sProgrammer Productivity Research Center, codersare working with new, "higher-level" languages likeC# that don’t permit certain errors. And in May,Microsoft cofounded the $30 million SustainableComputing Consortium - based at Carnegie Mellon- with NASA and 16 other firms to promote

On the Frontier

* Tiny surveillance planetakes flight

* Androids take the field inRoboCup

* Molding a smaller, fasterchip

* Engineers develop the ’toothphone’

* IBM ’punch cards’ setstorage record

* Internet navigators thinksmall

standardized ways to measure and improve softwaredependability. Quality control efforts can pay offhandsomely: in helping Lockheed Martin revampthe software in its C130J aircraft, Praxis CriticalSystems, of Bath, England, used such methods tocut development costs by 80 percent whileproducing software that passed stringent FederalAviation Administration exams with "very fewerrors." Critics welcome calls for excellence like thosefrom Kestrel and Microsoft but argue that they willcome to naught unless commercial softwaredevelopers abandon many of their ingrainedpractices. "The mindset of the industry is to treatquality as secondary," says Cem Kaner, a computerscientist and lawyer at the Florida Institute ofTechnology. Before releasing products, companiesroutinely hold "bug deferral meetings" to decidewhich defects to fix immediately, which to fix laterby forcing customers to download patches or buyupgrades, and which to forget about entirely. "Otherindustries get sued when they ignore knowndefects," Kaner says. "In software, it’s standardpractice. That’s why you don’t buy version 1.0 of aprogram." Exasperatingly, software vendors deliverbuggy, badly designed products withincomprehensible help files - and then charge highfees for the inevitable customer service calls. In thisway, amazingly, firms profit from poor engineeringpractices. When engineers inside a software companychoose to ignore a serious flaw, there are usuallyplenty of reviewers, pundits, hackers and otheroutsiders who will point it out. This is a good thing;as Petroski wrote in The Evolution of UsefulThings, "a technologically savvy and understandingpublic is the best check on errant design."Unfortunately, companies increasingly try todiscourage such public discussion. The fine print inmany software licenses forbids publishingbenchmark tests. When PC Magazine tried in 1999to run a head-to-head comparison of Oracle andMicrosoft databases, Oracle used the license termsto block it - even though the magazine had gone outof its way to assure a fair test by asking both firmsto help it set up their software. To purchaseNetwork Associates’ popular McAfee VirusScansoftware, customers must promise not to publish

reviews without prior consent from NetworkAssociates - a condition so onerous that the State ofNew York sued the firm in February for creating an"illegal ... restrictive covenant" that "chills freespeech." (At press time, no trial date had been set.) <!-- var sUA =navigator.appName.toLowerCase(); if(sUA.indexOf("webtv") == -1) {

IS LITIGATION THE ANSWER? Even a few members of thesoftware-is-different school believe that someprogramming practices must be reformed. "Wedon’t learn from our mistakes," says Rand’sPfleeger.

?There is nowell-definedmechanism [in thesoftware industry]for investigatingfailures and nomechanism forensuring thatpeople read aboutthem.? - SHARI PFLEEGERsenior researcher, Rand

In 1996, for example, the French Ariane 5rocket catastrophically failed, exploding just 40seconds after liftoff on its maiden voyage. Its $500million satellite payload was a total loss. Accordingto the subsequent committee of inquiry, the accidentwas due to "systematic software design error" -more precisely, a buffer overflow. In mostengineering fields, Pfleeger says, such disasterstrigger industrywide reforms, as the collapse of theWorld Trade Center seems likely to do forfireproofing in construction. But in software, "thereis no well-defined mechanism for investigatingfailures and no mechanism for ensuring that peopleread about them." If the French coders had beendrilled, like other engineers, in the history of theirown discipline, the Ariane fiasco might have beenavoided. One way or another, some computer scientistspredict, software culture will change. To thesurprise of many observers, the industry is relativelyfree of product liability lawsuits. The "I Love You"virus, for instance, spread largely because Microsoft- against the vehement warnings of security experts- designed Outlook to run programs in e-mailattachments easily. According to ComputerEconomics, a consulting group in Carlsbad, CA, thetotal cost of this decision was $8.75 billion. "It’samazing that there wasn’t a blizzard of lawsuits,"Wallach says. Software firms have been able to avoid productliability litigation partly because software licensesforce customers into arbitration, often on

unfavorable terms, and partly because such lawsuitswould be highly technical, which means thatplaintiffs would need to hire costly experts to buildtheir cases. Nonetheless, critics predict, the lawsuitswill eventually come. And when the costs oflitigation go up enough, companies will bemotivated to bulletproof their code. The downsideof quality enforcement through class actionlawsuits, of course, is that groundless litigation canextort undeserved settlements. But as Wallach says,"it just might be a bad idea whose time has come." In fact, a growing number of software engineersbelieve that computers have become so essential todaily life that society will eventually be unwilling tokeep giving software firms a free legal pass. "It’seither going to be a big product liability suit, or thegovernment will come in and regulate the industry,"says Jeffrey Voas, chief scientist of Cigital Labs, asoftware-testing firm in Dulles, VA. "Something’sgoing to give. It won’t be pretty, but oncecompanies have a gun to their head, they’ll figureout a way to improve their code." Copyright © 2002 Technology Review, Inc. AllRights Reserved.

Latest reports from Technology Review

High-tech scanners look for lies Review: ’Grand Theft Auto 3’ Big Brother at the office USB 2.0 takes on FireWire Summer solstice and the science of seasons MSNBC Cover Page

Blue chips continue tumbling Wall Street’s woes threaten economy Rite Aid execs indicted for fraud Martha Stewart broker put on leave R.J. Reynolds slapped with damages MSNBC Cover Page

why software is so bad - chapter 1 - maple bug lists ... software is so bad ..... and what’s being...

Documents