Copyright 2003-2013, Dennis J. Frailey
Software Engineering Best Practices – Things You Seldom Learn Much About in School
Dennis J. FraileyACM Distinguished Speaker
This talk discusses several practices that make a big difference for real
software development projects
Copyright 2003-2013, Dennis J. Frailey
The Distinguished Speakers Program is made possible by
For additional information, please visit http://dsp.acm.org/
3 Copyright 2003-2013, Dennis J. Frailey 3
Objective
To present some basic techniques used in professional software development projects– Some of these are not widely covered in
academic computing programs
4 Copyright 2003-2013, Dennis J. Frailey 4
Would You Buy This Car? The price is 50% more than what you
agreed to pay Delivered to the dealer 3 months late Not the right color Some of the options you selected aren’t
there (“we’ll install them later”) The speakers and the CD player are
incompatible, so you can only use the CD player through headphones
Although “new”, it has 5,000 miles on it from the extensive testing that was done
Despite that testing, it still won’t start if it’s below freezing and it has a number of other problems
5 Copyright 2003-2013, Dennis J. Frailey 5
Frequent Problems for Product Developers on Real Software Projects
Nobody knows what work must be done– Do we need to provide an
installation guide?– Is there a warranty on this software?
and what should we do about it if there is?
– In what format should we deliver the code?
– Do we have to provide training for the end users?
6 Copyright 2003-2013, Dennis J. Frailey 6
Frequent Problems on Real Software Projects (continued)
Vague understanding of the big picture– Has the customer used a computer before?– How does the software fit into the overall
customer environment? Are the networks and computers suitable for the
software?– How will the customer use the software?
7 Copyright 2003-2013, Dennis J. Frailey 7
Air Traffic Control Replacement System
The New System The Old System
8 Copyright 2003-2013, Dennis J. Frailey 8
Frequent Problems on Real Software Projects (continued)
Unclear understanding of how much has been accomplished and how much is left to do– “How much longer will it take?”– “How much more money is
required?”
9 Copyright 2003-2013, Dennis J. Frailey 9
Frequent Problems on Real Software Projects (continued)
Ignorance about risks– What are our biggest potential
project problems?– Do we have any idea what the
real risks are?– Do we know which ones to
worry about the most?– Can we mitigate the risks?
(make them less likely or lower their potential impact)
– Do we have a contingency plan? (what to do if the risks happen?
10 Copyright 2003-2013, Dennis J. Frailey 10
Frequent Problems on Real Software Projects (continued)
Confusion about different versions of software– Which version of the data base goes
with the latest version of the user interface?
– How come it does one thing in the customer’s site but something else in our testing lab?
– Does the design match the code? If not, which one is right?
11 Copyright 2003-2013, Dennis J. Frailey 11
Frequent Problems on Real Software Projects (continued)
Haphazard approaches to quality– “We didn’t have time to test it
thoroughly”– “We tested it for weeks but it still
had lots of bugs”
12 Copyright 2003-2013, Dennis J. Frailey 12
Best Practices for Managing These Problems
The Work Breakdown Structure– Documenting and organizing the work to be
done Estimating and Tracking
– So you know where you are Systems Engineering
– Designing the big system before designing the software
Risk Management Configuration Management Quality Assurance
13 Copyright 2003-2013, Dennis J. Frailey 13
The Work Breakdown StructureDefinition: A work breakdown structure is a
hierarchical list of the work activities required to complete a project.
This includes tasks for: - Software development - Management of software development - Support of software development - Any other activities required to meet
customer requirements, such as:– creation of documents, – training programs, – tool development or acquisition, – travel, etc.
Parser CodeGenerator
FileSystem
Run TimeSystem
UserInterface
ManageSoftware
Development
Build “C”Compiler
Build TestSuite
WriteDocumentation
WriteInstallationSoftware
Software for“C” Compiler
14 Copyright 2003-2013, Dennis J. Frailey 14
Sample Work Breakdown Structure for Taking a Course
1. Take a Course on Programming in Java1.1 Do Assignment 1 – Simple Program1.2 Do Assignment 2 – Program with Arrays1.3 Take Midterm Exam1.4 Do Assignment 3 – Program with Pointers1.5 Do Assignment 4 – Program with I/O1.6 Take Final Exam
These are the tasks you must complete in order to finish the course
15 Copyright 2003-2013, Dennis J. Frailey 15
Sample Work Breakdown StructureHierarchy Allows Additional Detail
1. Take a Course on Programming in Java1.1 Do Assignment 1 – Simple Program 1.1.1 Study Chapters 1 and 2 of textbook 1.1.2 Write a Simple Java Program 1.2 Do Assignment 2 – Program with Arrays 1.2.1 Study Chapters 3 and 4 1.2.2 Review Graded Assignment 1 1.2.3 Write Java Program with Arrays1.3 Take Midterm Exam 1.3.1 Study Chapters 1-5 1.3.2 Meet with study group to review 1.3.3 Go to exam room at designated time
16 Copyright 2003-2013, Dennis J. Frailey 16
Sample WBS for a Software Project (top levels only)
1.0 Software for Payroll system 1.1 Build the Payroll software 1.1.1 Build a User Interface 1.1.2 Build a File System 1.1.3 Build a Data Base 1.1.4 Build a Table of Rules 1.1.5 Build a Check Printing System 1.2 Build the Test Suite for the Software 1.2.x (detailed steps for building test suite) … 1.3 Write Documentation 1.4 Write Installation Software 1.5 Manage the Above
17 Copyright 2003-2013, Dennis J. Frailey 17
The WBS Is ...
A “table of contents” for the project.
18 Copyright 2003-2013, Dennis J. Frailey 18
Top Level Role of WBS
SourceDocuments
(Statement of Work,
Requirements,contract,
test criteria, etc,)
HistoricalRecords(at end
of project)
WBS
CostEstimate
(proposal &/project start)
CostTracking(during
execution)
19 Copyright 2003-2013, Dennis J. Frailey 19
Best Practices for Managing These Problems
The Work Breakdown Structure– Documenting and organizing the work to be
done Estimating and Tracking
– So you know where you are Systems Engineering
– Designing the big system before designing the software
Risk Management Configuration Management Quality Assurance
20 Copyright 2003-2013, Dennis J. Frailey 20
Using the WBS to Estimate
WBS #
Task Estimated Hours
Estimated Completion
1.1 Do Assignment 1 15 9/221.1.1 Read Chapters 1, 2 3 9/151.1.2 Write Simple Java Pgm 12 9/221.2 Do Assignment 2 21 9/301.2.1 Read Chapters 3, 4 4 9/231.2.2 Review Graded Assign. 1 1 9/231.2.3 Write Java Program with
Arrays16 9/30
21 Copyright 2003-2013, Dennis J. Frailey 21
Using the WBS to TrackWBS
#Task Estimated
HoursActual Hours
Estimated Completion
Actual
1.1 Do Assignment 1 15 12 9/22 9/251.1.1 Read Chapters
1, 2 3 2 9/15 9/22
1.1.2 Write Simple Java Pgm
12 10 9/22 9/25
1.2 Do Assignment 2 21 9/301.2.1 Read Chapters
3, 4 4 9/23
1.2.2 Review Graded Assign. 1
1 9/23
1.2.3 Write Java Pgm with Arrays
16 9/30
When the project is over, you have a detailed record of how much time and effort were required.
22 Copyright 2003-2013, Dennis J. Frailey 22
Graph to Show ProgressI M Exhausted, CSE 7315, Fall 2004
BCWS/ACWP/BCWP Line Chart
0
20
40
60
80
100
120
140
160
180
Wk 1 Wk 2 Wk 3 Wk 4 Wk 5 Wk 6 Wk 7 Wk 8 Wk 9 Wk 10 Wk 11 Wk 12 Wk 13 Wk 14 Wk 15
Week Number
Hou
rs
BCWSACWPBCWP
23 Copyright 2003-2013, Dennis J. Frailey 23
Using the WBS for Historical Records
Previous WBS
New WBS
Facts about New
Project
Previous WBSPrevious WBSPrevious WBS
Old ProjectsYou can make more accurate estimates for the next project.
24 Copyright 2003-2013, Dennis J. Frailey 24
Best Practices for Managing These Problems
The Work Breakdown Structure– Documenting and organizing the work to be
done Estimating and Tracking
– So you know where you are Systems Engineering
– Designing the big system before designing the software
Risk Management Configuration Management Quality Assurance
25 Copyright 2003-2013, Dennis J. Frailey 25
Systems EngineeringA Sample System
Display PrinterKeyboard
Processorand
MemorySystem
SoftwarePowerSupply
Computer
SystemUnit
26 Copyright 2003-2013, Dennis J. Frailey 26
Systems Engineering - Step 1 - Analyze the Requirements of the Total System
Determine what the customer really wants– >> Analysis of the problem and the
need Produce verifiable specifications
(specifications may be in the form of simulations of expected behavior, use cases, specification documents, system requirements models, or other things)
Confirm the specifications are right by conferring with the customer
27 Copyright 2003-2013, Dennis J. Frailey 27
Systems Engineering - Step 2 - Design the System
Determine the architecture of the system – >> Synthesis, not analysis– Done by Systems engineer and/or chief
architect Determine the major parts (functional
decomposition into sub-systems) Determine the interfaces between the
subsystems
28 Copyright 2003-2013, Dennis J. Frailey 28
Systems Engineering - Step 3 - Allocate the Requirements to the Components of the System
Determine what each part will do Assign requirements to each part based on what it is supposed to do
This is how you get some of the software requirements when software is part of a larger system
29 Copyright 2003-2013, Dennis J. Frailey 29
Typical Requirements Flowdown
Transmission
System Analysis& Design
Allocated Requirements
Automobile
Drive Train ChassisEngine
100 pounds
torque ...
50 pounds 2000 pounds
500 pounds250 hp
...
Original Requirements
3000 pounds0-60mph in 9 sec
30 Copyright 2003-2013, Dennis J. Frailey 30
When the Product is FinishedVerify that it Satisfies the Requirements
Requirements
Development Testing or Other Verification
Methods
Verification Methods derived from requirements
When the requirements (specifications) are verifiable by
testing, you are much more likely to have effective tests
31 Copyright 2003-2013, Dennis J. Frailey 31
Ideal Requirements (some but not all desirable characteristics)
Cohesive (each requirement addresses only one thing)
Complete (each requirement is fully stated; all requirements are known and allocated)
Sufficient (every required feature is specified)
Consistent (no requirements contradict each other)
Unambiguous (only one possible interpretation)
Verifiable (you can tell when they are met)
32 Copyright 2003-2013, Dennis J. Frailey 32
Actual Requirements
Some requirements will be:– TBD (unknown - to be
determined)– Wrong– Missing– Vague or unclear
Some requirements will– Contradict others– Not be readily
verified– Change in mid-
stream Technology changes Customer
understanding of problem changes
Problem changes
33 Copyright 2003-2013, Dennis J. Frailey 33
Risk Management for Imperfect Requirements
Define a lifecycle/process that is adaptable – it will readily accommodate requirements changes
Assign someone to be responsible to manage requirements changes– including involvement of software staff
Select a lifecycle based on level of expected requirements stability
Get ranges for “TBD” (to be determined) requirements
Work to eliminate wrong and missing requirements
34 Copyright 2003-2013, Dennis J. Frailey 34
Interface Requirements How does your software interface with
other parts of the system? This is just as important as other
requirements Make sure the interface requirements are
– Documented– Under Control
If possible, requirements should be – Contained in (or controlled by) a SINGLE
document or other control point
35 Copyright 2003-2013, Dennis J. Frailey 35
Who is Establishing the Requirements? All Stakeholders!
End user -- will use the software Champion -- will pay for it Marketing -- will sell it Accountants and Lawyers -- will protect
but may not understand the product or the process
Political factors– Such as engineers -- who “know better” than the
customer
36 Copyright 2003-2013, Dennis J. Frailey 36
“V” Model of Software DevelopmentA General Model of Software Development
Requirements Analysis
Architecture Design
Detailed Design
Implementation
Unit Test
Integration Test
Acceptance Test
Test Cases Based on Requirements
Test Cases Based on Architecture
Test Cases Based on Design
Test Planning Begins with Requirements!
This model underlies almost all others
37 Copyright 2003-2013, Dennis J. Frailey 37
Are we Asking the Right Question?
The usual question:– How can we verify the software (make sure we
have satisfied the requirements)?
A better question:– How can we define requirements that are readily
verified? The requirement can be verified through one of
these basic methods: inspection, demonstration, test (instrumented) or analysis (to include validated modeling &
simulation).
38 Copyright 2003-2013, Dennis J. Frailey 38
Best Practices for Managing These Problems
The Work Breakdown Structure– Documenting and organizing the work to be
done Estimating and Tracking
– So you know where you are Systems Engineering
– Designing the big system before designing the software
Risk Management Configuration Management Quality Assurance
39 Copyright 2003-2013, Dennis J. Frailey 39
Risk Management Risks are:
– things that can go wrong
If it has already happened, or is certain to happen, it is an issue, not a risk!– You should be discussing your action plan
for managing the problem
Copyright 2003-2013, Dennis J. Frailey
Risk Management Phases1) Risk Assessment
– The things you do before you start the project to identify
– What can go wrong?– How it can hurt you?– How likely is it to happen?– Why it may happen?– How you can mitigate (reduce) the risk?
2) Risk Control– The things you do during the project
– Preventing the unexpected– Responding when you cannot prevent
41 Copyright 2003-2013, Dennis J. Frailey 41
Risk Assessment ChartRisk Evidence How
LikelyImpact Priority Mitigation
Late project due to employee turnover
Unplanned turnover rate has been running high
7 7 49 Employee bonuses if they stay to end of project
Bottleneck at testing due to changing require-ments
Past experience with this customer
8 6 48 Iterative development Customer workshops
Etc. Etc. Etc. Etc. Etc. Etc.
42 Copyright 2003-2013, Dennis J. Frailey 42
Risk Control
A) Risk Monitoring- Watching to see if risks happen– Metrics to warn you of problems– Periodic reviews
B) Risk Abatement- Counteracting risks- Taking contingency actions as needed
43 Copyright 2003-2013, Dennis J. Frailey 43
Risk Control ChartRisk Monitoring Counteracting /
Contingency Ongoing
MitigationLate project due to employee turnover
Measure turnover rate and project schedule
Get management help in identifying alternate employees
Periodic employee mini-bonuses
Bottleneck at QA due to changing require-ments
Track requirements changes and QA backlog
Requirements freeze; Multiple releases
Feature prioritization
Etc. Etc. Etc. Etc.
44 Copyright 2003-2013, Dennis J. Frailey 44
Risk Management is Ongoing
Risk Management (planning, analysis)
Product Development
Data and reports to
identify and monitor risks
Actions to mitigate or abate risks
45 Copyright 2003-2013, Dennis J. Frailey 45
Example of Risk Monitoring:Requirements Stability Monitoring
Requirements Stability
020406080
100120140160
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
TotalTBDChanges
CDRPDR
46 Copyright 2003-2013, Dennis J. Frailey 46
Setting Thresholds
Expected Range of Values
Abnormally High Values
Abnormally Low Values
47 Copyright 2003-2013, Dennis J. Frailey 47
Thresholds Help You Monitor Risks
Thresholds tell you when to act– Beware of the “contented frog” problem
– Historical experience is a good basis to judge when things are getting out of hand
It’s only a little hotter
than yesterday!
48 Copyright 2003-2013, Dennis J. Frailey 48
Best Practices for Managing These Problems
The Work Breakdown Structure– Documenting and organizing the work to be
done Estimating and Tracking
– So you know where you are Systems Engineering
– Designing the big system before designing the software
Risk Management Configuration Management Quality Assurance
49 Copyright 2003-2013, Dennis J. Frailey 49
What Is Software Configuration Management?
Definition:
Software Configuration Management is the process
of identifying, organizing and managing modifications to
software
50 Copyright 2003-2013, Dennis J. Frailey 50
Purpose ofConfiguration Management
The purpose of configuration management is to maintain integrity of the product by controlling change to the
product
51 Copyright 2003-2013, Dennis J. Frailey 51
All components that make up or relate to a version of a product– The proper version of each:
What Is The Configuration?
� document� compiler used to develop� development tool� data set used for
simulating� special equipment� operating system
� test code� test data� library� procedur
e� hardware� etc.
� code module� component� data file� macro� simulator� specification� requirement
If any of the above is wrong, the product lacks integrity and may not work
52 Copyright 2003-2013, Dennis J. Frailey 52
What is Integrity?
Having the right parts that all fit together to form a correct product.
Requirements
User’s Guide
Main, rel 2.3Sub1, rel 1.7Sub2, rel 3.1...
Object code
53 Copyright 2003-2013, Dennis J. Frailey 53
Why is SW Configuration Management Needed?
Because software is flexible, changeable, and intangible
It is too easy to lose integrity of software because of these factors
Unmanaged change is very possibly the largest single cause of failure to deliver
systems on time and within budget.NASA SCM Guidebook, August, 1995
54 Copyright 2003-2013, Dennis J. Frailey 54
Examples of Loss of Integrity Performance 2 years ago cannot be
duplicated with the current release Source code will not compile without
errors Compiled code will not execute with the
same results as the released code It worked in the factory but not in the
field All records of a particular feature are
gone, and the programmer/designer left the company
55 Copyright 2003-2013, Dennis J. Frailey 55
More Examples of Loss of Integrity
Compiled code has different object code or code size from released code
A bug that we fixed once before has reappeared
Code does not match the design specification
A feature that we added and fully tested has suddenly stopped working
We cannot figure out which version of each module is needed to reproduce a problem we found in the field
56 Copyright 2003-2013, Dennis J. Frailey 56
Configuration Management Has Five Major Functions
1) Configuration Identification– A unique name for every version of every part– A record of which versions of the parts are combined
in what way to make a given release/version of the product [known as a Version Description Document]
2) Configuration Control– Rules for making changes to software
So two people don’t change it at the same time3) Change Authorization
– Deciding who is allowed to make changes and when?4) Status accounting
– Knowing where to find the current version of each component?
5) Configuration Authentication– Verifying that everything is right
57 Copyright 2003-2013, Dennis J. Frailey 57
Best Practices for Managing These Problems
The Work Breakdown Structure– Documenting and organizing the work to be
done Estimating and Tracking
– So you know where you are Systems Engineering
– Designing the big system before designing the software
Risk Management Configuration Management Quality Assurance
58 Copyright 2003-2013, Dennis J. Frailey 58
Software Quality AssurancePurposes:
– To prevent defective products from being shipped to the customer
– To arbitrate disputes when there are debates about whether the product is “good enough”
– To provide managers and developers with visibility into the process being followed and the products being built So they can manage the outcome
59 Copyright 2003-2013, Dennis J. Frailey 59
Quality Assurance Looks at the Entire Process
Requirements
Development Testing and Inspection
PassFail
Standards ofQuality
Process and Design Standards
Tests derived from requirements
60 Copyright 2003-2013, Dennis J. Frailey 60
Software Quality AssuranceTypical Practices:
– Inspections– Reviews– Audits– Communication– Measurements– Independent confirmation of compliance
Standards, requirements, procedures
61 Copyright 2003-2013, Dennis J. Frailey 61
Professional Software Engineering Is ...
Building Software Systems that People can Rely On
“The application of a systematic, disciplined, quantifiable approach to the development, operation and maintenance of software; that is, the application of engineering to software.”
IEEE Standard Glossary of Software Engineering Terminology
62 Copyright 2003-2013, Dennis J. Frailey 62
Where Are The Computing Jobs?
Applications of Computers & Electronics to Problems
in Other Disciplines
Research JobsSystems Jobs
(tool development,
operatingsystems, etc.)
All of the
Other Jobs
63 Copyright 2003-2013, Dennis J. Frailey 63
What Kinds of Jobs does a Computing Graduate Do?
Write Software or Build Hardware Fix Bugs in Other Peoples’
Software/Hardware Test Software or Hardware Design Software or Hardware Understand Customers and Document Their
Requirements Design Systems to Solve Customer
Problems Control Changes to Software or Hardware Assure Quality in Software or Hardware Plan Software or Hardware Projects Manage Software or Hardware Projects
Career Progression
64 Copyright 2003-2013, Dennis J. Frailey 64
Spectrum of Computing Jobs
Theory, Principles, Innovation
Application, Deployment,
Configuration
Computer Hardware
Organization; Information
Systems
Application Technologies
Software Methods &
Technologies
Systems Infrastructure
(Source: Computing Curriculum 2005, sponsored by ACM and IEEE Computer Society)
65 Copyright 2003-2013, Dennis J. Frailey 65
Spectrum of Computing Jobs
Theory, Principles, Innovation
Application, Deployment,
Configuration
Computer Hardware
Organization; Information
Systems
Application Technologies
Software Methods &
Technologies
Systems Infrastructure Computer Engineer
Build Hardware that Works
66 Copyright 2003-2013, Dennis J. Frailey 66
Spectrum of Computing Jobs
Theory, Principles, Innovation
Application, Deployment,
Configuration
Computer Hardware
Organization; Information
Systems
Application Technologies
Software Methods &
Technologies
Systems Infrastructure
Software Engineer
Build Software that You Can Depend On
67 Copyright 2003-2013, Dennis J. Frailey 67
Spectrum of Computing Jobs
Theory, Principles, Innovation
Application, Deployment,
Configuration
Computer Hardware
Organization; Information
Systems
Application Technologies
Software Methods &
Technologies
Systems Infrastructure
Computer Scientist
Find New Ways to Solve Computing
Problems
68 Copyright 2003-2013, Dennis J. Frailey 68
Spectrum of Computing Jobs
Theory, Principles, Innovation
Application, Deployment,
Configuration
Computer Hardware
Organization; Information
Systems
Application Technologies
Software Methods &
Technologies
Systems Infrastructure
Information Systems
Design Computer Systems that Solve Organization Problems
69 Copyright 2003-2013, Dennis J. Frailey 69
Spectrum of Computing Jobs
Theory, Principles, Innovation
Application, Deployment,
Configuration
Computer Hardware
Organization; Information
Systems
Application Technologies
Software Methods &
Technologies
Systems Infrastructure
Information Technology
Build Computer Systems that
Solve Organizational
Problems
70 Copyright 2003-2013, Dennis J. Frailey 70
Some Things are Learned On the Job
Foundation SkillsMathematics Science Communication
Computing SkillsRequirements Design ProgrammingTesting Management Planning
Application Skills(depends on
application area)
Job Experience
College Education
71 Copyright 2003-2013, Dennis J. Frailey 71
The Changes in Computing 1950’s 1960’s 1970’s 1980’s 1990’s 2000’s 2010’s
Hardware Technology
Vacuum Tubes
Transistors
I ntegrated Circuits
LSI VLSI ULSI Nano-systems
Programming Languages
Binary Assembly
Fortran Cobol
Pascal Algol Ada C Lisp
C++ GUI J ava
C# PhP XML
F#
Computing Paradigm
1 user Mainf rame
Batch Time Sharing
Personal Computer
LAN, WEB .NET ???
Operating System
none 1 user multi user multi user linked
networked Web, Open
source
Cloud
Data Base Methods
none Linear (tapes)
Hier-archical
Relational Object Oriented
X Query ???
Sof tware Design
pad and pencil
Flow Charts
Structured Design
Data Flow Object Oriented
RAD, XP, RUP
???
Copyright 2003-2013, Dennis J. Frailey
About ACM
ACM, the Association for Computing Machinery is the world’s largest educational and scientific computing society, uniting educators, researchers and professionals to inspire dialogue, share resources and address the field’s
challenges.
ACM strengthens the computing profession’s collective voice through strong leadership, promotion of the highest standards, and recognition of technical
excellence.
ACM supports the professional growth of its members by providing opportunities for life-long learning, career development, and professional
networking.
With over 100,000 members from over 100 countries, ACM works to advance computing as a science and a profession. www.acm.org
73 Copyright 2003-2013, Dennis J. Frailey 73
Questions?