december 3, 2009coms w41561 coms w4156: advanced software engineering prof. gail kaiser...
TRANSCRIPT
December 3, 2009 COMS W4156 1
COMS W4156: Advanced Software Engineering
Prof. Gail Kaiser
http://bank.cs.columbia.edu/classes/cs4156/
December 3, 2009 COMS W4156 2
Topics covered in this lecture
• Mythical Man Month
• No Silver Bullet
December 3, 2009 COMS W4156 3
Mythical Man Month
December 3, 2009 COMS W4156 4
The Mythical Man-Month
Fred Brooks, 1975
December 3, 2009 COMS W4156 5
Background• Fred Brooks began managing IBM’s OS/360 software
development effort in 1964• Brooks’ previous experience was in hardware design• OS/360 was (probably) the largest software system attempted
to date at that time• OS/360 “was late, took more memory than was planned,
costs were several times the estimate, and it did not perform very well until several releases after the first”
• Planned for release in 1965 (low-end) and 1966 (high-end), both finished a year late, so another temporary OS was cobbled together so hardware could be used
• OS.360 retired in 1972 (see http://ldworen.net/fun/os360obit.html)
December 3, 2009 COMS W4156 6
Background
• The MMM book results from analyzing the OS/360 experience, which was quite different from the 360 hardware effort
• MMM was first published in 1975, 20th anniversary edition published in 1995
• Turing Award in 1999• Brooks is now a Professor at University of North
Carolina, Chapel Hill, with research interests in graphics, user interfaces and virtual worlds
• http://www.cs.unc.edu/~brooks/
December 3, 2009 COMS W4156 7
The Tar Pit• Developing large software systems is “sticky” - the more you fight it, the deeper you
sink• Projects may emerge from the tar pit with running systems, but miss goals,
schedules and budgets• “No one thing seems to cause the difficulty – any particular paw can be pulled
away. But the accumulation of simultaneous and interacting factors brings slower and slower motion”
• Analogy meant to convey that it is hard to discern the nature of the problem(s) facing software development
December 3, 2009 COMS W4156 8
Program to Product
Program Programming System
Programming Product
Programming Systems Product
December 3, 2009 COMS W4156 9
Program to Product• A product (more useful than a program):
– can be run, tested, repaired by anyone – usable in many environments on many sets of data. – must be tested and documented
• To be a component in a programming system (collection of interacting programs like an OS): – input and output must conform in syntax and semantics to
defined interfaces – must operate within resource budget – must be tested with other components to check integration
(interactions grow exponentially)
December 3, 2009 COMS W4156 10
What makes programming fun?
• Sheer joy of creation
• Pleasure of creating something useful for self and other people
• Creating (and solving) puzzles
• Life-long learning
• Working in a tractable medium - software is extremely malleable
December 3, 2009 COMS W4156 11
What’s not so fun about programming?
• You have to be perfect!
• You are rarely in complete control of the project
• You have to depend on other people
• Design is fun; debugging is just work
• Testing takes too long!
• The program may be obsolete by the time its finished
December 3, 2009 COMS W4156 12
Why are software projects late?
• Estimating techniques are poorly developed• Estimating techniques confuse effort with
progress• Since we are uncertain of our estimates, we
don’t stick to them• Progress is poorly monitored• When slippage is recognized, we add people
(“like adding gasoline to a fire”)
December 3, 2009 COMS W4156 13
Optimism
• “All programmers are optimists”– “All will go well” with the project – thus we don’t plan for
slippage– Each task has a nonzero probability of failure or slippage -
probability that all will go well is near zero • One reason for optimism is the nature of creativity
– Idea, implementation and interaction– The medium of creation constrains our ideas – in software
the medium is extremely malleable, thus we expect few problems in implementation
December 3, 2009 COMS W4156 14
The Mythical Man-Month
• Cost does indeed vary as the product of the number of people and the number of months
• Progress does not!• The unit of man-month [person-month] implies that
people and months are interchangeable– This is only true when a task can be partitioned among
many workers with no communication among them– When a task is sequential, more effort has no effect on the
schedule – many tasks in software engineering have sequential constraints
December 3, 2009 COMS W4156 15
MMM
• Most tasks require communication among workers
• Communication consists of– Training– Sharing information (intercommunication)
• Training affects effort at worst linearly• Intercommunication adds n(n-1)/2 to effort if
each worker must communicate with every other worker
December 3, 2009 COMS W4156 16
Intercommunication effort
2 workers 1 path
3 workers 3 paths
4 6
5 10
6 15
7 21
December 3, 2009 COMS W4156 17
Comparison graphs
Adding more people then lengthens, not shortens, the schedule
months
No communication With communicationpeople
December 3, 2009 COMS W4156 18
Scheduling• Brooks’ rule of thumb
– 1/3 planning– 1/6 coding– 1/4 component test– 1/4 system test
• In looking at other projects, Brooks found that few planned for 50% testing
• But most actually spent 50% of their time testing (writing test harnesses can be almost as much work as or sometimes more than writing the actual code)
• Many projects were on schedule until testing began
December 3, 2009 COMS W4156 19
Development Team
• How should the development team be arranged?
• The problem: good programmers are much better than poor programmers– Typically 10 times better in productivity– Typically 5 times better in terms of program
elegance and resource consumption
December 3, 2009 COMS W4156 20
The dilemma of team size
• Consider a 200-person project with 25 experienced managers (experienced at programming, not necessarily at managing)
• Productivity differences argue for firing the 175 workers and use the 25 managers as the team
• OS/360 had over 1000 people working on it
December 3, 2009 COMS W4156 21
Two needs to be reconciled
• For efficiency and conceptual integrity a small team is preferred
• To tackle large systems considerable personnel resources are needed
• One solution: Harlan Mill’s Surgical Team approach – one person performs the work (with a co-pilot), all others perform support tasks
December 3, 2009 COMS W4156 22
Harlan Mills’ Surgical Team
• Surgeon – chief programmer
• Co-pilot – like surgeon but less experienced (training, insurance)
• Administrator – relieves surgeon of administrative tasks
• Editor – proof-reads and copy-edits documentation
• 2 Secretaries – support administrator and editor
• Program clerk – tracks versions
• Toolsmith – develops tools, utilities for surgeon
• Tester• Language lawyer (shared
among multiple projects)
December 3, 2009 COMS W4156 23
How is this different?
• Normally, work is divided equally among team members – now only surgeon and copilot divide the work
• Normally each person has equal say – now surgeon is absolute authority
• Note communication paths are reduced– Normally 10 people → 45 paths– Surgical Team → 15 paths
• Many roles on team now automated (no one hires 9 people to support each chief programmer)
December 3, 2009 COMS W4156 24
How does this scale?
• Reconsider the 200 person team – communication paths →19,900!
• Create 20 ten-person surgical teams
• Now only 20 surgeons must work together – 20 people → 190 paths
• The key problem is ensuring conceptual integrity of the design
December 3, 2009 COMS W4156 25
Conceptual Integrity
• Brooks’ analogy: Cathedrals
• Many cathedrals (e.g., Worms Cathedral) consist of contrasting design ideas
• But Reims Cathedral was the result of 8 generations of builders repressing their own ideas and desires to build a cathedral that embodies the key design elements of the original architect
December 3, 2009 COMS W4156 26
Worms vs. Reims
December 3, 2009 COMS W4156 27
Conceptual Integrity
• With respect to software, design by too many people results in conceptual disunity of a system, making the program hard to understand and use
• Better to leave functionality out of a system if it causes the conceptual integrity of the design to break
• Ease of use is enhanced only if the functionality provides more power than it takes to learn (and remember) how to use it in the first place
December 3, 2009 COMS W4156 28
Software Architects as Aristocrats
• Conceptual integrity requires that the design be the product of one mind
• The architect (or surgeon) has ultimate authority - and ultimate responsibility
• Does this imply too much power for architects?– Architect sets the structure of the system– Developers can then be creative in how system is
implemented
December 3, 2009 COMS W4156 29
The Second-System Effect
• An engineer is careful in designing his/her first system – he/she realizes that they are working in uncharted territory
• But in the second system, the engineer has some experience and wants to throw everything into the design
• Symptoms:– Functional embellishment – to an unnecessary degree– Optimizations to obsolete functionality
• OS/360 linker had a sophisticated program overlay functionality, but the architecture no longer depended on overlays – resulting in unnecessarily slow linkage
December 3, 2009 COMS W4156 30
How to avoid it?
• Employ extra self-discipline– Avoid functional ornamentation– Be aware of changes in assumptions– Strive for conceptual integrity
• How do managers avoid it?– Insist on a senior architect with more than two
systems under his/her belt
December 3, 2009 COMS W4156 31
Communicating design decisions
• Written specifications - “The Manual”– Answers questions– Conceptual integrity– Demands high precision
• Telephone log – make sure to capture all design decisions [today: IM, email, wiki, blog, etc.]
• Product test – external test group keeps implementation honest
December 3, 2009 COMS W4156 32
Formal Definitions
• Natural language is not precise• Notations help express precise semantics –
however, natural language is often needed to “explain” the meaning to the uninitiated
• What about using an implementation as the formal definition?– Advantages: precise specification– Disadvantages: Over-prescription, potential for inelegance,
may be modified– Inconsistencies between multiple implementations can
identify problems in the specs – with only one implementation its easier to change the manual
December 3, 2009 COMS W4156 33
Why did the Tower of Babel Fail?
• Communication (lack of it) made it impossible to coordinate
• How do you communicate in large project teams? – informal, meetings, workbook
• Workbook– Structure placed on project’s documentation– Technical prose lives a long time, best to get it structured
formally from the beginning, also helps with distribution of information
December 3, 2009 COMS W4156 34
Tower of Babel[Genesis 11:1-9] 1 And the whole earth was of one
language, and of one speech. 2 And it came to pass, as they journeyed from the east, that they found a plain in the land of Shinar; and they dwelt there. 3 And they said one to another, Come, let us make brick, and burn them thoroughly. And they had brick for stone, and slime had they for mortar. 4 And they said, Come, let us build us a city and a tower, whose top may reach unto heaven; and let us make us a name, lest we be scattered abroad upon the face of the whole earth. 5 And the Lord came down to see the city and the tower, which the children builded. 6 And the Lord said, "If as one people speaking the same language they have begun to do this, then nothing they plan to do will be impossible for them. 7 Come, let us go down, and there confound their language, that they may not understand one another's speech. 8 So the Lord scattered them abroad from thence upon the face of all the earth: and they left off to build the city. 9 Therefore is the name of it called Babel (confusion); because the Lord did there confound the language of all the earth: and from thence did the Lord scatter them abroad upon the face of all the earth
December 3, 2009 COMS W4156 35
Reducing communication paths
• Communication needs are reduced by– Division of labor– Specialization of function
• A tree structure often results from applying this principle– However this serves power structures better than
communication (since communication between siblings often needed)
– So communication structure is often a network
December 3, 2009 COMS W4156 36
Organizational structure
• Brooks outlines– Mission, producer, director, schedule, division of
labor, interfaces between the parts• The (then) novel suggestions are the producer
and the director• Producer manages project and obtains
resources (product manager)• Director manages technical detail (program
manager)
December 3, 2009 COMS W4156 37
Calling the shot
• Estimates for programming in the small don’t scale
• Need to add planning, documentation, testing, system integration and training in large projects
• Effort vs. program size increases exponentially
December 3, 2009 COMS W4156 38
Plan to throw one away
• You will anyway• Consider chemical engineers: scaling a laboratory
result up to actual (and practical) use requires a pilot step – e.g., desalting water 10,000 gallons/day first, then 2,000,000
• Software projects typically plan to deliver the first thing they build to customers– Typically hard to use, buggy, inefficient, etc.– Experience shows you will discard a lot of the first
implementation anyway (wrt version 2)
December 3, 2009 COMS W4156 39
Rapid prototypes
• Help gain early feedback• Intended from start to be thrown away• Management question:
– Plan to build a system to throwaway• Experience gained, feedback can be applied
– Plan to build a throwaway - that is delivered to the customer
• User is aggravated and demands support
• Brooks focused on planning for change
December 3, 2009 COMS W4156 40
Change
• Causes:– Both the actual need and the user’s perception of that
need will change as programs are built, tested, used– Other factors – hardware, assumptions, environment
• Handling:– Modularization– Precise and complete interfaces
• Standard calling sequences• Complete documentation
– High-level languages– Configuration management
December 3, 2009 COMS W4156 41
Organizational issues
• Culture must be conducive to documenting decisions, otherwise nothing gets documented
• Other points to consider– Job titles– Keeping senior people trained
December 3, 2009 COMS W4156 42
Maintenance
• Two steps forward and one step back– Lifecycle of bugs– Fixing a bug has a chance of adding another, lots
of regression testing needed• One step forward and one step back
– Maintenance is an entropy-increasing process– As maintenance proceeds, system is less
structured than before, conceptual integrity degrades (foreshadows refactoring)
December 3, 2009 COMS W4156 43
Hatching a catastrophe
• A project gets to be a year late one day at a time
• Major calamities are “easy” to handle – whole team pulls together and solves it
• Day to day slippage is harder to recognize
December 3, 2009 COMS W4156 44
How to keep on track?
1. Have a schedule– Overestimates come steadily down as the activity proceeds– Underestimates do not change until scheduled time draws near
2. Have checkable milestones– Not “coding complete”– But “specifications signed by architects”– Or “debugged component passes all tests”
3. Track the critical path – who is waiting on whom to finish what
4. Address the “status disclosure problem”– Managers must distinguish between action meetings and status
meetings: if inappropriate action taken in response to a status report, it discourages honest status reports
December 3, 2009 COMS W4156 45
Summary of MMM
• Adding more people to a late project makes it later
• Reduce communication paths
• Design by small number of top-notch people, but avoid second system effect
• Plan throwaway
• Track schedule
December 3, 2009 COMS W4156 46
No Silver Bullet
December 3, 2009 COMS W4156 47
No Silver Bullet – Essence and Accident in Software Engineering
Fred Brooks, 1986
December 3, 2009 COMS W4156 48
No Silver Bullet
“There is no single development, in either technology or management technique, which by itself promises even one order-of-magnitude improvement within a decade in productivity, in reliability, in simplicity”
December 3, 2009 COMS W4156 49
Why a Silver Bullet?
• http://www.fvza.org/wmyths.html
December 3, 2009 COMS W4156 50
Past Gains in Software Productivity
• Came from removing artificial barriers such as severe hardware constraints, awkward programming languages, lack of machine time
• Unless at least 9/10ths of what software engineers do today is still devoted to the accidental (incidental), then shrinking all the accidental activities to zero time will not give an order of magnitude improvement
December 3, 2009 COMS W4156 51
Essence vs. Accident• Essential:
– The difficulties inherent in the nature of the software– The fashioning of the complex conceptual structures that
compose the abstract software entity– The specification, design and testing of these conceptual
constructs• Accidental (or incidental):
– Those difficulties that attend software’s production but that are not inherent
– The labor of representing these abstract entities in programming languages and the mapping of these onto machine languages within space and speed constraints
– Testing the fidelity of the representation
December 3, 2009 COMS W4156 52
Past Silver Bullets(from Brooks)
• High-level languages (vs. bits, registers, opcodes)
• Time-sharing (vs. batch programming)
• Unified programming environments (e.g., Unix)
All addressed accident, not essence
December 3, 2009 COMS W4156 53
Past Silver Bullets(mentioned by others)
• Stored programs• Keyboard and mouse• Subroutine and return stack• Assemblers, compilers and interpreters• Multitasking• Files and file system hierarchies• Relational databases• Video display (vs. teletype) and wysiwyg editing• Data networks and modems
December 3, 2009 COMS W4156 54
No Silver Bullet – Precise Definition
“There is no single development, in either technology or management technique, which by itself promises even one order-of-magnitude improvement within a decade in productivity, in reliability, in simplicity”
Silver Bullet defined as an individual technology or practice that would make 10x improvement within 10 years
December 3, 2009 COMS W4156 55
Comparison with Hardware
• The anomaly is not that software progress is so slow, but that computer hardware progress is so fast
• No other technology since civilization began has seen six orders of magnitude price-performance gain in 30 years [written in 1986]
• In no other technology can one choose to take the gain in either improved performance or in reduced costs
• We cannot expect ever to see two-fold gains in software productivity every two years
December 3, 2009 COMS W4156 56
Four Issues of Essential Difficulty
• Like physical hardware limits (e.g., speed of light, heat dissipation), there are software engineering problems that will never be solved
• Inherent difficulties of software production 1. complexity 2. conformity 3. changeability 4. invisibility
December 3, 2009 COMS W4156 57
Complexity
• Software is far more complex than any other human construct
• Software is far more complex than hardware – 32 bit word in HW – 232 possible states– If variable x gets stored there, get those same
states + states everywhere in code that is dependent on it!
December 3, 2009 COMS W4156 58
Complexity
• No two parts are alike (above statement level):– Very large number of states– Scaling up not merely a repetition of the same
elements in larger sizes, but an increase in the number of different elements
– Elements interact with each other in some non-linear fashion, so the complexity of the whole increases more than linearly
December 3, 2009 COMS W4156 59
Complexity
• Complexity is an essential property: Descriptions of a software entity that abstract away its complexity often abstract away its essence
• Increased complexity causes increased communication difficulty - leading to product flaws, cost overruns, schedule delays, difficulty of extending programs
December 3, 2009 COMS W4156 60
Complexity• From the complexity comes the difficulty of
enumerating, much less understanding, all the possible states of the program - and from that comes unreliability
• From the complexity of the functions comes the difficulty of invoking the functions - which makes programs hard to use
• Increased complexity of structure causes increased difficulty in extending programs to new functions - without creating side effects
• From complexity of structure comes the unvisualized states that constitute security trapdoors
December 3, 2009 COMS W4156 61
Complexity
• Management, communication and personnel turnover exacerbate all of these:– Difficult to overview, understand whole product,
impeding conceptual integrity– Hard to find and control all the loose ends– Errors in specification as well as code – How can you estimate without understanding? – Maintenance is a nightmare
December 3, 2009 COMS W4156 62
Conformity
• To the many (arbitrary) human institutions and systems to which it interfaces
• Software is embedded in a cultural matrix of applications, users, laws and machines
December 3, 2009 COMS W4156 63
Conformity
• Software conceived as most conformable • Interface with an existing (physical) system
– Example: factory built & asked to create software to control
– Software must conform to the factory • Interface with a new (physical) system
– Misperception that software is easier to conform – Again, software will be forced to conform
December 3, 2009 COMS W4156 64
Changeability
• Software changes very easily, but ramifications serious
• Manufactured things infrequently changed after manufacture – instead superseded by later models– Product recalls rare – but software often updated– Software easier to change than hardware
December 3, 2009 COMS W4156 65
Changeability
• Pressure to change – Reality changes – Useful software will encourage new requests – Long lifetime (~15 yrs) vs. hardware (~4 yrs) – Software changes viewed as “free”
• Buildings are in fact changed– But change understood by all to be time-
consuming and expensive (and messy)
December 3, 2009 COMS W4156 66
Invisibility• Software is invisible and unvisualizable
– The reality of software is not inherently embedded in geometric space
– Compare to the floor plan of a building• When we diagram software, we find many super-
imposed structures: – flow of control, – flow of data, – patterns of dependency, – time sequence,– …
• These graphs usually not planar, not hierarchical
December 3, 2009 COMS W4156 67
Invisibility
• There is no single good structure – No way to represent a complete product or overview – Complete views (e.g., code) incomprehensible – Partial views misleading
• Geometric visualization is an important tool for dealing with conceptual complexity
• But even with visualization, hard to communicate to – Other software professionals – Users & clients
December 3, 2009 COMS W4156 68
Past Breakthroughs Solved Accidents
• High level languages enhance/ease representation, improve vocabulary and how to think about problems
• Time sharing/multitasking - quick turnaround benefits, immediacy, don’t lose context
• Unified programming environments - help us better manage conceptual constructs, use programs together, but not figure out what they should be
• These and other past breakthroughs addressed accidents, not essence
December 3, 2009 COMS W4156 69
Hopes for the Silver[in 1986]
• Have only made small improvements (not 10x) • Workstations (PCs) – just speeds up compilers,
eventually compilation time will go to “zero”• Ada - retraining of programmers in modern design
and modularization techniques• OO – abstract data types plus hierarchical interfaces
and refined subtypes (can achieve x10 productivity only if 90% of the overall effort is type specification)
December 3, 2009 COMS W4156 70
More Hopes for the Silver [still 1986]
• AI – sliding meaning (use of computers to solve problems that previously could only be solved by applying human intelligence)
• Expert Systems – rule base plus inference engine, benefit is separation of application complexity from the program itself, still need good knowledge base (in principle, novice programmers could benefit from the accumulated wisdom of experts)
• Automatic programming - very domain dependent, selects among known approaches to limit problem
December 3, 2009 COMS W4156 71
More Hopes for the Silver [still 1986]
• Graphical programming - when software is hard to visualize, how can this help?
• Program verification – could work on code viz. the specification, but also needs to work on eliminating errors upstream
• Environments and tools - hierarchical file systems, uniform file formats, generalized tools, language based editors, integrated database systems to track details
December 3, 2009 COMS W4156 72
Promising Attacks[in 1986]
• Attacks on the formulation of complex conceptual structures: – Buy vs. build - hard part is requirements,
specifications and design, not implementation, and many real problems can be solved by non-programmers using spreadsheets
– Iterative requirements refinement and rapid prototyping – clients do not know what they want
– Incremental development - grow, not build
December 3, 2009 COMS W4156 73
Promising Attacks[in 1986]
• Cultivate great designers:– The software art centers on people– Better software engineering training
– Career mentors
– Career development paths
December 3, 2009 COMS W4156 74
Summary of No Silver Bullet
• Software is the most complex entity constructed/modified by humans
• Most past productivity gains came from addressing “accidental” rather than “essential” aspects of software development
• Most proposed (as of 1986) approaches to productivity gains also address “accidental”
• Promising attacks (as of 1986) include large scale reuse, iterative and incremental development, cultivating great designers
December 3, 2009 COMS W4156 75
New Silver Bullets? [after 1986]
• UML• Patterns• Frameworks• Scripting languages• Open source• World wide web
Silver Bullet defined as an individual technology or practice that would make 10x improvement within 10 years
December 3, 2009 COMS W4156 76
Final Notes
December 3, 2009 COMS W4156 77
Next Assignment
• Demo “week” Monday December 7 – Monday December 14
• Some teams will give demos in class on Tuesday December 8 and Thursday December 10
• Otherwise schedule with your TA
December 3, 2009 COMS W4156 78
Upcoming Deadlines
• Demo “week” Monday December 7 – Monday December 14
• Second iteration final report due Tuesday December 15
• Final Individual Assessment posted Thursday December 10
• FIA due Friday December 18
December 3, 2009 COMS W4156 79
COMS W4156: Advanced Software Engineering
Prof. Gail Kaiser
http://bank.cs.columbia.edu/classes/cs4156/