19/04/23 Dr Andy Brooks 1

MSc Software MaintenanceMSc Viðhald hugbúnaðar

Fyrirlestrar 45 og 46Essential Documentation

19/04/23 Dr Andy Brooks 2

Case StudyDæmisaga

ReferenceA Study of the Documentation Essential to Software

Maintenance, Sergio Cozzetti B. de Souza, Nicolas Anquetil, and Káthia M. De Oliveria.The 23rd International Conference on Design of Communication, SIGDOC´05, pp 68-75, 2005 ©ACM

1. Introduction• Documentation continues to be “renowned for its

absence” despite the emphasis placed by educators.• Agile software development actually questions the

importance of documentation.– “Combined with the preference for face-to-face communication,

agile methods usually produce less written documentation than other methods.”

• http://en.wikipedia.org/wiki/Agile_software_development

• There is pressure to re-document legacy software.– to support ongoing maintenance or re-engineering

• All this raises the question: “What documentation would be most useful to software maintenance?”

19/04/23 Dr Andy Brooks 3

Examining the Agile Manifestohttp://www.ambysoft.com/essays/agileManifesto.html

19/04/23 Dr Andy Brooks 4

1. Introduction

“Working software over comprehensive documentation. 

When you ask a user whether they would want a fifty page document describing what you intend to build or the actual software itself, what do you think they’ll pick?  My guess is that 99 times out of 100 they’ll choose working software.  If that is the case, doesn’t it make more sense to work in such a manner that you produce software quickly and often, giving your users what they prefer? 

Furthermore, I suspect that users will have a significantly easier time understanding any software that you produce than complex technical diagrams describing its internal workings or describing an abstraction of its usage, don’t you?  Documentation has its place, written properly it is a valuable guide for people’s understanding of how and why a system is built and how to work with the system.  However, never forget that the primary goal of software development is to create software, not documents – otherwise it would be called documentation development wouldn’t it?”

Examining the Agile Manifestohttp://www.ambysoft.com/essays/agileManifesto.html

http://en.wikipedia.org/wiki/Agile_software_development

19/04/23 Dr Andy Brooks 5

1. Introduction

“Continuous attention to technical excellence and good design enhances agility. 

It's much easier to understand, maintain, and evolve high-quality source code than it is to work with low-quality code.  Therefore, agilists know that they need to start with good code, to keep it good via refactoring, and take a test-driven approach so that they know at all times that their software works.  We also adopt and follow development guidelines, in particular coding guidelines and sometimes even modeling guidelines.”

• Specific Agile tools and techniques include continuous integration, automated unit testing, pair programming, test driven development, design patterns, and code refactoring.

• Ongoing code refactoring in Agile software development means that (preventative) maintenance is a normal state of affairs.

1. Introduction

• Agile software development, however, does not adress what happens over extended periods of time “when a development team is sure to disperse with the knowledge it has of the implementation details”, so documentation is still a “highly relevant artifact”.– teams eventually break up and take their knowledge with them– there is a need for communication across time between the

development team and the maintenance team.

• Re-documenting legacy software or software re-documentation is about trying to make up for past deficiencies. This is, however, a costly activity whose expense is difficult to justify to customers.

19/04/23 Dr Andy Brooks 6

3.1 Some documentation problems

• non-existent or poor quality or out of date– Last Updated July 17 2002.

• too much (!)– volumes of documentation are easier ignored than read

• difficult to access– tables, text, and diagrams across multiple file types

• lack of programmer interest in documentation– do you find it boring writing JavaDoc? I know I do...

• http://java.sun.com/j2se/javadoc/

• difficult to produce standardised documentation– how many developers know all of UML well? I know I don’t...

• http://www.uml.org/

19/04/23 Dr Andy Brooks 7

JPEGTIFFPNGGIFBMP...

F. A. Cioch and M. Palazzolo. A documentation suite for maintenance programmers. In Proceedings of the 1996 International Conference on Software Maintenance (ICSM’96), pages 286–95. IEEE,1996.

• Documentation is designed specifically for new staff joining a software maintenance team.

• Staff begin as newcomers and progress through the stages of student and intern until they become experts.

• Newcomers receive short contextual overviews: project mission, product review, and module role in product. – The overviews are like the high-level information that is found in

marketing presentations or briefings to management.

• Students study what modules do, how they work, and how they interact with other code and the external environment. Architecture diagrams and design stories are used.

19/04/23 Dr Andy Brooks 8

3.2 Documentation for maintenance related work

F. A. Cioch and M. Palazzolo. A documentation suite for maintenance programmers. In Proceedings of the 1996 International Conference on Software Maintenance (ICSM’96), pages 286–95. IEEE,1996.

• Interns undergo practical training. Task-oriented manuals are used to explain how code should be organised, compiled, executed and tested. Interns are not exposed to complete standards documents (e.g. The configuration management standards) as this may “trigger the feeling of information load”.

• Experts make use of traditional documents produced during software development such as requirements and design specifications.– Traditional documents are said to provide too much information

to newcomers, students, and interns.

19/04/23 Dr Andy Brooks 9

3.2 Documentation for maintenance related work

Scott W. Amblerhttp://www.agilemodeling.com/essays/agileDocumentation.htm

• Ambler recognises the need in Agile software development to provide various kinds of documentation for software maintainers: design decisions, project

overview, and system documentation.• Decision decisions is a summary of critical decisions

regarding design and architecture made by the team throughout development.– Ambler advises to focus on decisions:

• which are not obvious

• which had other reasonable alternatives

19/04/23 Dr Andy Brooks 10

3.2 Documentation for maintenance related work

Scott W. Amblerhttp://www.agilemodeling.com/essays/agileDocumentation.htm

• Project overview is a summary of:– system vision– tools and technologies used– build details– procedures for the backing up of data– references to the source code and available documents

• System documentation is an overview of the technical architecture, the business architecture, and high-level requirements. (A detailed technical architecture may be provided.)

– “System documentation is said to provide truck insurance, the assurance that if the development team leaves, or gets hit by a truck, that critical information about the project is left behind.” 

19/04/23 Dr Andy Brooks 11

3.2 Documentation for maintenance related work

Andy says: Ambler´s essay on Agile documentation is worth a read.

A. Forward and T. C. Lethbridge. The relevance of software documentation, tools and technologies: a survey. Proceedings of the 2002 ACM symposium on Document engineering (DocEng’02) ,pages 26–33, 2002. ©ACM

• 48 respondents (mainly managers and developers)

• Question 4: How often is documentation updated when changes occur in a software system?

1 = ‘never made’ … 5 ‘updates are made within a few days’

19/04/23 Dr Andy Brooks 12

3.2 Documentation for maintenance related work

Table 2 ©ACMmode/tíðasta gildi

A. Forward and T. C. Lethbridge. The relevance of software documentation, tools and technologies: a survey. Proceedings of the 2002 ACM symposium on Document engineering (DocEng’02) ,pages 26–33, 2002. ©ACM

• Question 6: How often do you consult available software documentation?

1 ‘never’ ... 5 ‘always’• Across all 48 respondents, the most consulted document

was Specifications (average score 3.85).• Across 12 respondents identifying one role as that of

manager, the most consulted document was Requirements (average score 3.60).

• Across 17 respondents identifying as a developer, but not manager, the most consulted document was Architectural (average score 4.33).

19/04/23 Dr Andy Brooks 13

3.2 Documentation for maintenance related work

Questionnaire survey one

• The questionnaire could be answered on paper or over the internet.– Voluntarily and anonymously.

• Convenience (or accidental) sampling was used.– One of the authors used his list of contacts.

• 76 software maintainers “from various parts of Brazil” answered the questionnaire in the second half of 2004.

• Two part questionnaire: respondent details and ratings of importance of various types of documentation.

19/04/23 Dr Andy Brooks 14

convenience sample/hentugleikaúrtakdrop-out/brottfall

Critical commentary from Andy

• For external validity , a random sample from a population is required. In this study, a convenience sample was used which can introduce bias. – Asking a professional society to distribute or publicise the

questionnaire is better, but does not in itself guarantee a lack of bias. For example, only those who have had problems with documentation might answer the questionnaire.

• We are not informed how many of the contacts declined to answer the questionnaire. This is the drop-out or non-response rate which can be used to help judge the results of a survey.– If you obtain only 10 replies from a random sample of 1,000 then

it is likely some sort of bias is present.

19/04/23 Dr Andy Brooks 15

drop out/brottfallbias/skekkja

Can the results be generalized?

Respondent details

• Position: – manager, analyst or programmer

• Years of experience in maintenance:– 1-3, 3-5, 5-10, >10 years

• Number of systems maintained:– 1-5, 6-10,11-20, >20 systems

• Known approaches:– structured analysis, object-oriented

19/04/23 Dr Andy Brooks 16

Questionnaire survey one

“In Brazil, all software engineers usually know a technique called structured analysis.”

Ratings

• “Based on your practical experience, indicate what importance each documentation artifact has in the activity of understanding a software to be maintained”. – 1 = “no importance”, – 2 = “little importance”– 3 = “important”, – 4 = “very important”– don´t know

• There were 34 documentation artificats in total, arranged in the following categories: requirement elicitation, analysis, design, coding, test, and transition.

19/04/23 Dr Andy Brooks 17

Questionnaire survey one original questionnaire was in portugese

Documentation artefacts

• Requirement elicitation– Structured analysis: (1) requirements list, (2) context diagram,

(3) requirement description– Object-oriented: (4) vision document, (5) use case diagram– Structured and OO: (6) conceptual data model, (7) glossary

• Analysis– Structured analysis: (8) functions derived from the requirements,

(9) hierarchical function diagram, (10) data flow diagram.– Object-oriented: (11) use cases specifications, (12) class

diagram, (13) activity diagram, (14) sequence diagram, (15) state diagram

– Structured and OO: (16) non functional prototype, (17) logical data diagram (MER), (18) data dictionary

19/04/23 Dr Andy Brooks 18

Questionnaire survey one MER likely ERM Entity Relationship Model

Documentation artefacts

• Design:– Structured analysis: (19) architectural model, (20) general

transaction diagram, (21) components specification.– Object-oriented: (22) collaboration diagram, (23) components

diagram, (24) distribution diagram.– Structured and OO: (25) physical data model, (26) functional

prototype

• Coding– Structured and OO: (27) comments in source code, (28) source

code

19/04/23 Dr Andy Brooks 19

Questionnaire survey one

Documentation artefacts

• Test: – Structured and OO: (29) unitary test plan, (30) system test plan,

(31) acceptance test plan

• Transition– Structured and OO: (32) data migration plan, (33) transition plan,

(34) user manual

19/04/23 Dr Andy Brooks 20

Questionnaire survey one

Questionnaire survey two

• The questionnaire was answered only on paper to allow the same person to provide answers for up to 8 different maintenance projects. – Voluntarily and anonymously.

• Convenience (or accidental) sampling was used.– No effort was made to have all the first survey participants answer the second

survey. • “some maintainers actually answered both surveys, others only the first or only the second”

• 52 software maintainers “from various parts of Brazil” provided answers for 237 maintenance projects.

• Two part questionnaire: respondent and system details and actual usage of various types of documentation.

19/04/23 Dr Andy Brooks 21

convenience sample/hentugleikaúrtakdrop-out/brottfall

Respondent and system details

• Years of experience in maintenance:– 1-3, 3-5, 5-10, >10 years

• Months of experience with the system maintained:– 1-3, 3-6, 6-12, >12 months

• Application domain:– banking, education, ...

• Size of the maintenance team:– 1 person, 1 team, several teams

• System age:– <1 year, 1-2 years, 3-5 years, >5 years

19/04/23 Dr Andy Brooks 22

Questionnaire survey two

Respondent and system details• System maturity:

– recently developed, mostly corrective maintenance, growing – corrective and evolutive maintenance, stablised – few corrective and mostly evolutive maintenance, phasing out – new product being developed

• Documentation quality:– complete? –yes/no, up-to-date? -yes/no, readable? -yes/no

• Approach:– structured analysis, object-oriented

• Maintenance type realized during the survey:– corrective, evolutive

• Defined maintenance process?– yes,no

19/04/23 Dr Andy Brooks 23

Questionnaire survey two

Documentation artefacts

• The documentation artefacts were as before, but rather than one long list, questionnaires were arranged either for structured analysis (24 documentation artefacts) or object-oriented (25 documentation artefacts.)

19/04/23 Dr Andy Brooks 24

Questionnaire survey two

structured analysis (24) object-oriented (25)

Table 1 ©ACM

19/04/23 Dr Andy Brooks 25

structured analysis, 70 repliesQuestionnaire survey one

Table 1 ©ACM

19/04/23 Dr Andy Brooks 26

object-oriented, 54 repliesQuestionnaire survey one

Survey one results• Across both structural analysis and object-oriented, the

two most important artificats are the source code and comments. – “This result was expected.”

• Across both structural analysis and object-oriented, the following were ranked highly:– data models (circles ●)– requirements (squares ■)– acceptance test plan (diamond ♦)

• Across both structural analysis and objection-oriented, unit and system test plans are ranked relatively well.

• “A big surprise” was the low rankings of general views of the system (triangle ►).

19/04/23 Dr Andy Brooks 27

19/04/23 Dr Andy Brooks 28

Table 2 ©ACMstructured analysis, 142 repliesQuestionnaire survey two

19/04/23 Dr Andy Brooks 29

Table 2 ©ACMobject-oriented, 95 repliesQuestionnaire survey two

Survey two results• Across both structural analysis and object-oriented, the

source code and comments are ranked very highly.– 1st and 3rd.

• Across both structural analysis and object-oriented, data models (circles ●) “lost the importance they had in the first survey”.

• Across both structural analysis and object-oriented, the test plans are well used.

• The low rankings of general views of the system “seems partially confirmed “ (triangle ►).

19/04/23 Dr Andy Brooks 30

System maturity

• Total replies = 51– 3 (6%) recently developed– 32 (64%) growing– 14 (27%) stabilized– 2 (4%) phase out

• The authors did not try to analyse survey responses by system maturity.– Systems that are stabilized or being phased out are

unlikely to undergo architectural changes. Are general views less important for these systems?

19/04/23 Dr Andy Brooks 31

Survey two results

Documentation quality• Completeness: total=51

– complete = 14 (27%), – not complete = 37 (73%).

• Actuality: total=51– up-to-date = 17 (33%)– out-of- date = 34 (67%).

• Readability: total=51– readable = 41 (80%)– unreadable = 10 (20%)

19/04/23 Dr Andy Brooks 32

Survey two results

“The quality of the documentation is as could be expected: generally incomplete and out-of-date.”

Maintenance type

• Total replies = 237– 64 (27%) corrective– 125 (53%) evolutive– 48 (20%) both

• The authors did not try to analyse survey responses by maintenance type.– Is the scope of corrective maintenance usually so

small (e.g. fixing a method) that a maintainer does not need to make use of general views of the system?

19/04/23 Dr Andy Brooks 33

Survey two results

6. Conclusion• “The surveys confirmed that source code and comments

are the most important artifact to understand a system to be maintained.”

• “Data models and requirement descriptions were other important artifacts.”– test plans too were also relatively important

• “Surprisingly, and contrary to what we found in the literature, architectural models and other general views of the system are not very important.”– To explain this, the authors’ suggest that such models providing

general views may be used once, then never consulted again.

• “Further research is needed...”

19/04/23 Dr Andy Brooks 34

Critical commentary from Andy

• The scope of a single maintenance request can vary from a single statement to the system in its entirety. An explicit model of maintenance requests is, however, absent in this work.

• If the scope is a single statement, a maintainer will need the source code and comments.

• If the scope is the entire system, then the maintainer is also likely to need models providing general views.

• The two surveys may have simply indirectly measured the frequency distribution of maintenance request types.

• General views are obviously very important when making changes to the system architecture.

19/04/23 Dr Andy Brooks 35

19/04/23 Dr Andy Brooks 36

http://yourdon.com/strucanalysis/wiki/index.php?title=Introduction

The context diagram shows the flow of data to and from external entities.

Figure 0 shows the decomposition of the system into main sub-processes.

Figure 3 shows the decomposition of sub-process 3.

If a maintenance request resulted in the need for a new flow of data from a new external entity, the maintainer is likely to want to view the context diagram.

Structured AnalysisData Flow Diagramming