software engineering practice - configuration management
DESCRIPTION
TRANSCRIPT
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 1
Configuration management
McGill ECSE 428Software Engineering Practice
Radu Negulescu
Winter 2003
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 2
About this course module
A typical software project may generate thousands of source files and pieces of documentation, and there are many opportunities for introducing inconsistencies between these artifacts:
• Teamwork, parallel development and QA
• Change requests
• Bug fixes
• Schedule pressures
Here we discuss how to effectively organize the artifacts of a software project.
Recommended reading:
• Jalote ch. 10
• Bruegge & Dutoit 10.2,10.3,10.4.1,10.4.3,10.4.4
• McConnell Rapid Dev. pp. 338-341, 403-414
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 3
What is configuration management
B&D: The disciplines and techniques of initiating, evaluating and controlling change to software products during and after development.
• In other words, staying on top of change
Known by many names:
• CM
• SCM - sometimes pronounced “scum”
• Version control
• Source control (especially in relation to code)
• Document control
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 4
Development problems addressed
Typical problems of software projects due to lack of tracking:
• Communication blocks
• InconsistenciesAccidental deletesWrong releasesOverlapping updates
• Lost issues (and rework)Lost bug reportsLost reasons for change
• Poor predictabilityAbsence of baseline data
• Awkward integration
• High cost of human errorDifficult “undo”
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 5
Challenges of configuration management
Challenges:
• Number of change requests (100s, 1000s, ...)
• Rapid growth of configurations: builds components variants
• Superlinear growth of team coordination overheadNumber of communication paths required
• Dangers of bureaucracy
• ...
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 6
Configurations
Item: artifact under configuration management
• Possibly nested aggregate
• Possibly overlapping with other aggregates
Configuration: state of an item
Variants: configurations of the same item that are intended to coexist
Beware:
• Terminology varies, although needs and solutions are similar
• Focus on the concepts
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 7
Configurations
Version: augmented specification
• Additional features
• Major or minor, depending on the amount of features
Release: same spec, enhanced implementation
• Bug fixes
• Refactored architecture
Build: instance of a system, integrated
• For testing
• Release candidatesShown to customer for evaluationBecome a release once accepted
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 8
Typical CM functions
Retrieve a configuration / undo a change
• E.g. what was our best build?
• … before we messed up the event model?
Life cycle control
• Static access restrictionsE.g. a code module can only be signed out by the “owner” (Dev.)E.g. a bug report can only be put in “closed” state by QAE.g. each new feature must be impact analyzed before it is scheduled for developmentE.g. only PL and CC create releases
• Dynamic access restrictionsE.g. no simultaneous updates
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 9
Typical CM functions
Automatic builds
• Invoke the compiler tools on the updated files only
Concurrent updates
• Branching then merging
• Reconciliation of changes
Tracing rationale
• E.g. why did we need a new event model anyway?
• Who issued a request for change?
Propagating change
• E.g. bug fix for deployed version then new version
• E.g. multiple platforms, languages, user privileges
Statistics
• E.g. determine percentage of testing vs. development delays
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 10
Retrieving a configuration
What we want from a mechanism for retrieving configurations:
• Label configurations in a way that orients the users
• Store configurations efficientlyThere can be many of them...
• Be tailored to the actual flow of configurations
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 11
How configurations are generated
Three main sources of change
• Shifting requirementsRework previous code
• Bug fixesBatches from QA sessionsContinuous flow from debugging
• Internal issues, such as: New vendor/new technologyPerformance tuningRefactoring
Not synchronized to the main development process
• Need for change may occur at any time
• Thus, you need some buffering
Reqflow
Issueflow
Bugflow
Develop.process
Cust
om
er
QA
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 12
Labeling of configurations
File names are vital
• Orient the developers and customers
• Gets interesting when dealing with 100s of files… or even 10s of files
Releases
• 2.5: version 2, release 5
• 2.3.5: version 2.3, release 5
Builds
• RN-2002-03-08-16:01:53
• Internal use onlyNot in the release file nameCC keeps record of release number
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 13
Example: JAR files
Distinguish release candidates sent to customer
• E.g.: do not send multiple System.jar files to customerBetter: FlatCart-Release4.3-…
Include the purpose of the file in an envelope nameE.g.: FlatCart-Release4.3-ForAcceptanceCandidate2.zip
Can use normal release names in the contained filesE.g.: FlatCart4.3.jar (JAR file)E.g.: FlatCart4.3.exe (self-extracting archive)
In Java 2, JAR files can be annotated with version info
• Version (“specification-version” attribute)
• Release or build (“package-version” attribute)
What if the system includes multiple components with different version & release numbers?
• The system can have yet another version scheme
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 14
Example: branch labeling
[After Bruegge & Dutoit]
revised
revised
Main
revised
Branch derived
merged
released
MUE.1.1:Releas
MUE.1.2:Releas
MUE.1.3:Releas MUE.1.2.1.1:Releas
MUE.1.2.1.2:Releas
MUE.2.0:Releas
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 15
Storage of configurations
Basic operations supported
• Get latest configuration: most frequent
• Insert a new configuration: frequent
• Get previous configurations: on a need-to basisRollback (undo changes)Locate insertion of a defect
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 16
Storage of configurations
Reverse delta
• Complete “baseline” version: latest
• DeltasSee diff tool
• Retrieval: compute previous versions from deltas
• E.g. VSS
Storing branches and variants
• A new base can be created for each branch on splitting from trunkNot many branches sharing won’t save much space
Storing aggregates
• Store cross-references only consistency, space (preferred)
• Store copies of all included components ready access
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 17
Item life cycles
Enforcing item life cycles
• Proper state transitions
• E.g. program modulesThe system is ready for acceptance testing when all modules are OK(Next page)
• E.g. triage of new feature requestsImpact analysis: which modules will be affected?Prioritization, cost estimationDevelopment, testingAcceptance
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 18
Item life cycles
Passed Module 1 Private
Ready for unit test
Baselined / released
Done
Failed
Passed Module 2 Private
Ready for unit test
Done
Failed
...
System/acceptance testing
Passed
Failed
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 19
Life cycle control
Objectives
• Facilitate access by developers
• Permit collection of management stats
• Allow easy traceability
Main mechanisms
• Directories
• Item database
• Document logs
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 20
Directories
Storage
• One directory for each module state
• State transitions: move among directories
• May use read/write permissions for directories
• Use CM tools for access restrictions in common areas
• Example: Infosys’ WAR project
Enforcement
• Directory read/write privileges
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 21
Item database
Storage
• State attributes for each item
• Some tools restrict which users may set which attributes
Enforcement
• Attribute settings
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 22
Document logs
Storage
• Use module headers
• Insert date of change, author
• Specify distribution list
Enforcement
• Self-discipline
• Configuration auditsA reason to maintain redundant info...
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 23
Dynamic rights
Mutual exclusion of updates
• Check-in/check-out
• Except for deliberate branching/merging
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 24
Example: WinCVS
www.wincvs.org
Work areas
• Repository
• Working directory
Check out
• CVS admin → Checkout module
• Specify module and working directory
Check in
• Update the working directory
• Commit to the repository
Related tools
• On-line CVS: www.freepository.org
• CVS
Workspace 1Workspace 2
Master directory
Software repository
Check-in Check-out
Baseline
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 25
Example: WinCVS
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 26
Example: document control
[After Jalote]
States = directories
• Under development← Initial stateAuthor: Done → Under review
• Under reviewReviewer: Defects found → Under developmentReviewer: No defects → Baselined
• BaselinedCCB: CR authorized → Under development
Under dev Underreview Baselined
[author]
[reviewer]
[reviewer]
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 27
Example: document control
Distribution listControlled
ProprietaryLimitedUnlimited
UncontrolledTo be destroyed
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 28
Automatic builds
Builds are frequent
• Some processes support daily buildsE.g. McConnell ch. 18Highly effective schedule reduction
• Other processes support continuous refactoringXP compensates de-emphasis of global design
Compilation can take a lot of time
• Idea: recompile only the updates
• Careful not to compile the wrong ones
Redundancy is hard to maintain
• Executables are easily created⇒ executables are not normally controlled
• Hence the term “source control”
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 29
Example: the make utility
Make
• A tool that controls generation of executables
• Handles multiple source files
• Recompiles if executable is older than a source
• Looks up change dates
• Not tied to a particular language
• Can even be used to format documents
Makefile
• Tells make how to recompile
• A text file
• Two sections: variables and rules
• Variables come first, but are optional
More info (GNU make)http://www.gnu.org/software/make/make.htmlhttp://www.gnu.org/manual/make/html_mono/make.html
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 30
Makefile rules
E.g. [from GNU]edith : main.o kbd.o command.o display.o \
insert.o search.o files.o utils.o
cc -o edith main.o kbd.o command.o display.o \
insert.o search.o files.o utils.o
main.o : main.c defs.h
cc -c main.c
kbd.o : kbd.c defs.h command.h
cc -c kbd.c
command.o : command.c defs.h command.h
cc -c command.c
display.o : display.c defs.h buffer.h
cc -c display.c
insert.o : insert.c defs.h buffer.h
cc -c insert.c
search.o : search.c defs.h buffer.h
cc -c search.c
files.o : files.c defs.h buffer.h command.h
cc -c files.c
utils.o : utils.c defs.h
cc -c utils.c
clean :
rm edith main.o kbd.o command.o display.o \
insert.o search.o files.o utils.o
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 31
Makefile rules
Callmake Program
make
make clean
General formattarget: prerequisite1 prerequisite2 ...
<tab> command1
<tab> command2
...
And here's a very useful one:clean:rm *.o core ./tmp/*
Deletes all object files in the current directoryany core dumpall files in ./tmp
// Looks up a file named “makefile”
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 32
Makefile variables
Exampleobjects = main.o kbd.o command.o display.o \
insert.o search.o files.o utils.o
edith : $(objects)
cc -o edith $(objects)
main.o : main.c defs.h
cc -c main.c
kbd.o : kbd.c defs.h command.h
cc -c kbd.c
command.o : command.c defs.h command.h
cc -c command.c
display.o : display.c defs.h buffer.h
cc -c display.c
insert.o : insert.c defs.h buffer.h
cc -c insert.c
search.o : search.c defs.h buffer.h
cc -c search.c
files.o : files.c defs.h buffer.h command.h
cc -c files.c
utils.o : utils.c defs.h
cc -c utils.c
clean : rm edith $(objects)
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 33
Concurrent updates
Branch: concurrent development path requiring independent configuration management.
Merge: reconcile different branches.
merging point
UI branch
main trunk
comms branch
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 34
Concurrent updates
Support for change reconciliation
• View changes side-by-sideDecide which changes to adoptMight be tedious for divergent versionsSupported for text files, not binary (e.g. VSS)
• Merge the most recent changesExample: Word for windows “Merge Documents”
Tools → Merge Documents → open documentAccept or reject changes(Tools→Track changes→ ...)
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 35
Example: feature interaction
Functionality partitioning
• First release: F1, F2
• Second release: F3, F4
F3 is easy
• M4, M5 can be a separate increment
• Versioning of M4, M5 independent of M1, M2, M3
F4 needs a separate CM branch
• Create branch: copy M2, M4, M5; create M6
Do not check out M2, M4, M5 at this timeOther developers keep working on M2, M4, M5
• Implement F4
• Reintegrate branch to main build (trunk)
Check out M2, M4, M5Use delegated permission
Merge changesRegression tests of F4
Check in M2, M4, M5Regression tests of F1, F2, F3(and F4 to be sure)
• If reintegration fails, one version is dumped…
M1 M2 M3 M4 M5 M6 F1 X X X F2 X X F3 X X F4 X X X X
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 36
Change propagation & traceability
Forward change requests to all related modules
Retrieve reason for each modification
• So the modification can be undone!
Traceability databases
• Propagate changes along dependency relationsE.g. among variantsE.g. from SRS to DD to code to TP
• Dependency matrix
• Dependency tree
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 37
Change propagation & traceability
Example: derived modules
• After the current version is deployedThe current version is in a maintenance phase.Work may begin on a new version.
• If multiple versions are supportedBug fixes may occur for any supported versionBug reports, changes, etc. must propagate to the other versions
Keep module derivation relationships across multiple supported system versions.
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 38
The CM plan
CM item structure
• Naming scheme
• Versioning scheme
Release
Baseline
Physical setting
Configuration control process
• Staff responsibilities
Configuration audits
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 39
CM item structure
Can be more or less elaborate
• Depends on size of project, usage of artifacts, and change flow
Same idea as in modular design
• Things that are likely to change together should be kept together
• Things that are unlikely to change together should be separate
• The hierarchy of storage should reflect the likelihood of change
Modularity
• Subsystems
Nesting
• E.g. per increment
Overlapping
• E.g. per increment & per artifact type
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 40
Example: client-server system
[From B&D]
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 41
Responsibilities
Example:
• Configuration controllerCreate executablesBaseline code and docs...
• DevelopersCheck out, check in at specific times
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 42
Configuration auditing
Danger: state related mistakes
• E.g. move a non-tested module into the release directory
Use several mechanism for life-cycle support
• E.g.: directory structure + “master table”
• E.g.: directory structure + revision lists in module headers
Status monitoring
• Consistency of the redundant state mechanismsE.g. directory placement same as master table entry
• Consistency of the revision list with the artifact life cycle
Audits
• Check whether proper process is being followed
• Check the baseline and release states
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 43
Change control board
The change control board (CCB)
• Analyze changes
• “Triage” changes
• Bundle changes
Representatives from each party
• Development, QA, user documentationArtifacts owners
• Customer support, marketing, mgmt
Danger: go overboard...
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 44
CASE example: the Rational Suite
The full Rational Enterprise Suite is installed in our lab.
Team integration tools
• RequisitePro: organize, prioritize, track, control requirements
• ClearQuest: manage every type of changeImplements artifact lifecycleTalks to a database
• SoDA: automated documentationExtracts info and puts it in a Word template
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 45
Development
• Rose: draw UML diagramsChecks consistencyImplements code frameworks from the diagram: it writes class and method declarations and some docs, and you fill in the body of the methods
• Purify: run-time errors and memory leaksNeeded big time in C/C++Theoretically, not needed in Java (JVM-managed garbage collection)Practically, Java programs can exhibit object pool overflows, JVM malfunctions, heap reallocations, etc. -> need memory leak tools
• Visual PureCoverage: expose testing gapsIt instruments the code to register execution
• Visual Quantify: Profiles the application to check performance bottlenecks
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 46
CASE example: the Rational Suite
Quality assurance
• Robot: record and playback test scriptsPoint-and-click operations are saved into a scriptGraphical object captureVerification points: test only certain properties of certain objects. This allows changes in the UI design without changing all the test casesThe scripting language is called SQA-Basic, and it is very similar to VB except for a couple of statements. See if you can run SQA-Basic scripts in VB
• TestFactoryAutomatic test generation:“Best test”: a random run through the code that works out as much of it as possibleSource code coverage analysis: what code was exercised by the test cases
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 47
CASE example: the Rational Suite
Integration among these tools
• Rational Administrator
Integration with other tools
• MS Project: activities can be linked to requirements in the RequisitePro database
• MS Office: many of the Rational tools use Access for a RDBMS, and Word for their documentation interface
• MS VisualStudio: VB, VC++, ...
• Java
McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 48
Discussion
Thank you!
Any questions?