mining software repositories to assist developers and support managers ahmed e. hassan software...
TRANSCRIPT
Mining Software RepositoriesMining Software Repositoriesto Assist Developersto Assist Developers
and Support Managers and Support Managers
Ahmed E. Hassan Ahmed E. Hassan
SoftWare Architecture Group (SWAG)SoftWare Architecture Group (SWAG)
University Of WaterlooUniversity Of Waterloo
[email protected]@alumni.uwaterloo.ca
2
Software Repositories
• Track the evolution of a software project: – source control systems store changes to the code
– defect tracking systems follow the resolution of defects
– archived personnel communications record rationale for decisions throughout the life of a project
• Used primarily for historical record supporting activities such as: checking bug status, or retrieving old code
3
Usage of Software Repositories Beyond Record Keeping
• Prior Research:– SEL - NASA [Basilli, et. al]:
• Fault analysis (Fortran 84)
– 5ESS - Lucent [Eick, Graves, Mockus, Perry, et. al]:• code decay, fault and effort predictions, visualizations, process
– EMERALD - Nortel [Hudepohl, Khoshgoftaar, et. al]– Telephony - Nokia [Gall et. al]:
• reveal hidden dependencies
– Open Source [Chen et. al]: • index code for searching
• Personal Industrial Experience• Open Source Phenomena: easy access to rich repositories
for large projects
4
Survey on Usage of Source Control Repositories in Industry
• Developers:– More likely to record the change rationale than list
implementation alternatives or limitations– Use change msgs to understand legacy code and fix bugs
• Senior developers use messages to:– Track the progress of the project– Monitor the quality of the code (specific developers)– Ensure code ownership boundaries
• Change messages in open source projects are as descriptive as ones in industrial systems
CVS Mailings
Bugzilla
CVS Mailings
Bugzilla
HistoricalInformation
Extraction
Bug Prediction
Resource Allocation
Software Understanding
Change Propagation
CVS Mailings
Bugzilla
HistoricalInformation
Extraction
Bug Prediction
Resource Allocation
Software Understanding
Change Propagation
8
Extracting Information from Source Control Repositories
• CVS tracks changes at the line level, instead an evolutionary extractor tracks:– Addition, removal, or modification of code entities
(functions, variables & macros):• function_1 is added/removed/modified
– Addition and removal of dependencies:• function_1 no longer calls function_2
• Attaches additional CVS change information: developer name, co-changed entities, change reason
9
Complexity of Evolutionary Extraction
main() { int a; /*call help*/ helpInfo(); }
helpInfo() { errorString!} main() { int a; /*call help*/ helpInfo(); }
helpInfo() { int b;} main() { int a; /*call help*/ helpInfo(); }
V1:Undefined func.(Link Error)
V2:Syntax error
V3:Valid code
10
Evolutionary Extractors: Challenges and Complexity
• Unstable & Evolving Code
• Scalability and Robustness– Large amount of data/snapshots– Legacy systems are hard to parse
• Accuracy– Entities with similar names throughout time
Assisting Software Developers
12
Understanding the Software Architecture
File System
MemoryManager
NetworkInterface
ProcessScheduler
Inter-ProcessCommunication
Legend: subsystem depends on
File System
MemoryManager
NetworkInterface
ProcessScheduler
Inter-ProcessCommunication
Legend: subsystem depends on unexpecteddependency
Conceptual(proposed)
Concrete (reality)
Why? Why? Who?Who?
When? When? Where?Where?
13
Dependency Investigation Questions (W4 Approach)
• Which low-level code entity is responsible for a dependency?– Network (SendData) Scheduler (PrintToLog)
• Who added/removed the dependency?– Junior vs. senior/experienced developer
• When was the dependency modified?– Late night / Just before release
• Why was the dependency added/removed?– The rationale!
14
Source StickyNotes
• Static dependencies give a static view of the system – not enough detail!
• Need to extend static dependencies, but how?– Ask developers to fill StickyNotes for each
change– Use software repositories to build these notes
automatically
15
Case Study – NetBSD(VM Subsystem)
HardwareTrans.
Kernel FaultHandler
Pager
FileSystemVirtual Addr.
Maint.VM Policy
Subsystem
Depend
DivergenceHardware
Trans.
Kernel FaultHandler
Pager
FileSystemVirtual Addr.
Maint.VM Policy
Convergence
Subsystem
Why? Why? Who?Who?
When? When? Where?Where?
Conceptual(proposed)
Concrete (reality)
16
Unexpected Dependencies
• Eight unexpected dependencies• All except two dependencies existed since the first day:
– Virtual Address Maintenance Pager
– Pager Hardware Translations
Which? vm_map_entry_create (in src/sys/vm/Attic/vm_map.c) depends on pager_map (in /src/sys/uvm/uvm_pager.c)
Who? cgd
When? 1993/04/09 15:54:59 Revision 1.2 of src/sys/vm/Attic/vm_map.c
Why?
from sean eric fagan: it seems to keep the vm system from deadlocking the system when it runs out of swap + physical memory. prevents the system from giving the last page(s) to anything but the referenced "processes" (especially important is the pager process, which should never have to wait for a free page).
17
Understanding Software Using StickyNotes
• Traditional dependency graphs and program understanding models usually do not use historical information
• In many open source projects, CVS comments are used for:– Communicating new features– Explaining design decisions– Narrating the progress of a project
Supporting Managers
19
Code Development Process
• Central part in making software – Code changes to:– Implement/Enhance features– Repair faults
• Acts as an early warning:– Does a complex code development process produce complex
software?
• Can we:– measure its complexity?– monitor the evolution of its complexity?– predict problems (delayed releases or faults)?
20
Development Process Chaos
A chaotic code development process negatively affects its outcome, the software system, such as the occurrence of faults
Domain
Requirements
Team(size/structure)
SchedulePressure
Code/DesignDevelopment
Process
“Complexity is the business we are in and complexity is what limits us”. Fred Brooks, The Mythical Man-Month
21
Complexity of Development Process
• “To implement feature A we modified one file”
• “To implement feature B we modified too many files – I don’t even remember half of them!”
• Too much information to recallDecay of grasp of what is going in a complex system
22
• Measures the amount of information in a message• Shannon Entropy for a random prob distrib. P
pk = 1:– If all pk = 1/n (equal prob.) maximal entropy
– If exists pi = 1 minimal entropy
• Min. number of bits – Best compression
Information Theory
Hn(P) = - pk log2(pk)
23
0.5
0.3
0.1
0.1A
D
C
B
Time
File
period(for e.g. week)
Info Theory in Dev Process
24
Evolution of Entropy
Time
Entropy
0.5
1
1.5
2
A
D
C
B
period 1 period2 period 3 period 4
0.5
0.3
0.1
0.1A
D
C
B
Time
File
period(for e.g. week)
25
Adaptive System Sizing
• System size changes over time • Standardized Shannon Entropy H:
• Many files are rarely modified (eg. util or libs):– A large n reduces the entropy– Use an active working set instead of n = # of files
H(P) = (log2 n)-1 * Hn(P) = - (log2 n)-1 * pk log2(pk)
26
Postgres – Development Chaos
0.7
0.75
0.8
0.85
0.9
0.95
1
320 330 340 350 360 370 380 390 400
Time (months since 1970)
Trad. Entropy
Adaptive Entropy
Release Dates
6.0 6.3 6.4 6.5 7.0 7.1 7.26.21.09 6.1
“This release marks a major step in the development team's mastery of the source code we inherited from Berkeley. You will see we are now easily adding major features, thanks to the increasing size and experience of our world-wide development team” 6.5 release notes
27
Case Study 2: KDE
0.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
330 340 350 360 370 380 390 400
Time (months since 1970)
En
tro
py
Trad Entropy
Adaptive Entropy
KDE 2.0 was also a very ambitious project, and the nature of the changes delayed the project for a whole year”, Miguel de Icaza, founder of GNOME and Ximian
28
Mathematical Validation Details
• Studied repositories of six open source software systems (OpenBSD, FreeBSD, NetBSD, Postgres, KDE, KOffice)
• Built statistical regression models using the 2nd and 3rd years:– Faults
– Changes
– Entropy metric
data to predict faults in 4th and 5th years
29
Results of Mathematical Validation
• Prior faults better fault predictor than prior changes (~50% improvement in prediction error)
• Entropy metric better fault predictor than:– Changes (~13-40% improvement)– Faults (~15-38% improvement)
• Results statistically significant at (95%) using a paired T-test
30
Development Chaos Summary
• A new perspective on the complexity of software – focus on process instead of code
• Entropy metrics used to study the evolution of complexity and predict faults
• Results verified using 6 open source projects
31
Conclusion
• Software repositories contain a wealth of valuable information about the evolution of projects
• Presented approaches and techniques to:– Recover data from source control repositories
– Assist developers in understanding code structure
– Support managers in predicting the occurrence of faults
• MSR likely to take a central role in supporting software development practices and research
33
Thesis Contributions
• Evolutionary Software Extractors (C-REX)• For Developers:
– Source StickyNotes– Development Replay Approach
• For Managers:– Top Ten List– Software Development Chaos
• All contributions were validated through:– case studies using open source systems, and– a survey of software practitioners
34
Research Overview
• Develop approaches and techniques to :– Part I : Extract Information from Repositories– Part II : Assist Developers in
• Understanding the current code structure
• Changing code and propagating changes
– Part III: Support Managers in• Allocating limited testing resources
• Predicting faults
35
Part II: Assisting Developers
• Source StickyNotes:– Assist in architecture understanding
– Attach historical information to the dependency graph
• Development Replay:– Reenacts historical changes using proposed tools and
strategies to assess their benefits
– Permits us to perform empirical based estimation of the benefits of tools and strategies, e.g. change propagation
36
Part III: Supporting Managers
• Top Ten List: – Assists managers in allocating limited resources
• Development Process Chaos: – Develop a complexity metric based on the
process followed by the developers to produce the code instead of on the code or the requirements
37
38
Research Hypothesis
“Software repositories contain a wealth of valuable information about the evolution of a software project. By mining such historical information, we can develop techniques and approaches to support software developers and managers in their endeavors to build and maintain complex software systems”