softwarereconstruction - chapter 20 - software architecture in practice
TRANSCRIPT
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 1
L. Bass, P. Clements, R. Kazman: „Software Architecture in Practice“
Chapter 20: Architecture Reconstruction and Conformance
Maciej Lukaszewski
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 2
1. Introduction
2. Architecture Reconstruction Process.
2.1. Raw view extraction
2.2. Database Construction
2.3. View Fusion
2.4. Finding Violations.
3. Guidelines
4. Summary
Agenda
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 3
Introduction: why reconstruction?
What if system already exists?
Documentation lost?
Documentation no longer synchronized?
The answer is: reverse engineering!
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 4
Architecture reconstruction:
„...techniques that allow an analyst to build, maintain, and understand a representation of an existing architecture...“
Two purposes:
- to document an architecture
- to ensure conformance between as-built and as-designed architecture
Introduction II
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 5
Reconstruction, but how?
- using automated and semiautomated extraction tools
- probing the original design intent of the architect
→ combination of both techniques
→ in practise: reconstruction not fully automated
Introduction II
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 6
First: workbench!
Phases:
1. Raw view extraction
2. Database construction
3. View fusion and manipulation.
4. Architecture analysis: finding violations
Architecture Reconstruction Process
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 7
„...analyzing a system's existing design and implementation artifacts to construct one or more models of it.“
Goals:
- extracting a target set of architectural views to support documentation effort
- answers on „what components are affected if I rewrite component A?“, „How to refactor layering to remove cyclic dependencies“
→ elements of interest (files, functions, variables) and their relationships come from source artifacts (code, header files, build files)
Raw view extraction
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 8
Source element
Relation Target element
Description
File includes File C preprocessor #include of one file by another
file contains Function Definition of a funcion in a file
A typical list of the elements and several relationships among them.
Raw view extraction II
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 9
Two categories of information obtained:
- static – observing only system artifacts
- dynamic – observing how the system runs
Goal: fusing both to create more accurate system views
Raw view extraction III
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 10
What if system changes at runtime?
→ capturing runtime configuration
Why capturing?
- dynamic elements
- late binding
- static reverse engineering not possible (peer-2-peer, SOA, cloud)
Raw view extraction IV
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 12
Last, but not least → reduce irrelevant information
Example: operating-system specific libraries
Raw view extraction V
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 13
- storing extracted information in database (large amount of data, manipulations of data error-prone)
- when using a workbench (set of tools) choosing a database and deciding on internal representations of views required
- often extracted data from the code too complex → white noise
Database construction
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 14
• Raw views may be too specific to aid in architectural understanding.
Accurate, but not helpful
Database construction II
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 15
How to filter information?
Manipulating views (e.g. hiding methods inside class definitions)
Showing abstractions (e.g. identifing distinct layers)
Database construction III
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 16
Views which have been extracted are manipulated to create fused views
e.g. static call view fused with dynamic call view to provide more accurate information about the part of system
Creating a fused view = creating a hypothesis about the architecture and a visualization of it
- views can be interpreted, further refined, or rejected
- no universal completion criteria for this process
View fusion
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 17
● Tool for static code review written in Java, support of other programming languages planed
● „Bridging the gap between the architects and developers“
● More http://www.hello2morrow.com/products/sonarj
Short excursion: SonarJ (Sonargraph for Java)
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 18
Short excursion: SonarJ (Sonargraph for Java)
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 19
SonarJ extracts facts from a set of source code files and lets you define a set of layers and vertical slices through those layers.
View fusion: example of a fused view
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 20
View fusion phase gave us a lot of hypotheses– those inform of architectural elements (layers),
constraints, relationships
– Hypotheses need to be tested in order to check conformance with architect's intention
– Two major possibilities for maintaining conformance between code and architecture:
- conformance by construction
- conformance by analysis
Architecture analysis: finding violations
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 21
Previous figure with added relationships and constraints, which have been added by an architect to reflect design intent.
Architecture analysis II: example
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 22
Black Arc: SonarJ has found a violation. The arc between the
Service layer and JUnit.
Architecture analysis III
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 24
● Have a goal and a set of objectives or questions in mind before undertaking an architecture reconstruction project.
● Obtain some representation of the system before beginning the detailed reconstruction process.
● Identifying layers is a good place to start● In some cases: disregard the existing documentation● Tools can not do all the work! Involvement of people familiar
with the system required.
Guidelines
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 25
● Architecture reconstruction ensures that a system is built the way it was designed and that it evolves in a way that is consistent with developer's intentions.
● Reconstruction needed when:
- documentation lost or out-of-date
- checking conformance
- starting the reengineering process to a new desired architecture
Summary
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 26
● The software architecture reconstruction process comprises the following phases:
- raw view extraction
- database construction
- view fusion
- architecture analysis
Summary II
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 27
1. What inferences can you make about a system's module structures from examining a set of behavioral traces gathered dynamically?
Discussion
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 28
2. Suppose you hypothesize that a system makes use of particular tactics to achieve a particular quality attribute. Fill in the columns of the table below to show how you would go about verifying your hypothesis.
Discussion
Tactics for.. Interviewing experts on the system
Analyzing structure of source code files
Static analysis of source code
Dynamic analysis of system's execution
Availability
Interoperability
Modifiability
Performance
Security
Testability
Usability
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 2
1. Introduction
2. Architecture Reconstruction Process.
2.1. Raw view extraction
2.2. Database Construction
2.3. View Fusion
2.4. Finding Violations.
3. Guidelines
4. Summary
Agenda
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 3
Introduction: why reconstruction?
What if system already exists?
Documentation lost?
Documentation no longer synchronized?
The answer is: reverse engineering!
Imaginary situation: → so far discussed architectural decisions to achieve goals
and requirements, but: → you come to a company with some complex solutions
already working and the architecture not recorded by actual developers?
→ system already exists, but no documentation describing original processes
→ documentation available, but no longer up-to-date due to series of changes
How to maintain sucha system and test quiality attributes?
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 4
Architecture reconstruction:
„...techniques that allow an analyst to build, maintain, and understand a representation of an existing architecture...“
Two purposes:
- to document an architecture
- to ensure conformance between as-built and as-designed architecture
Introduction II
In architecture reconstruction, the as-built architecture of an implemented system is reverse engineered from existing system artifacts
- the opposite to forward engineering: elements mapped to specific implementation elements (functions, classes, files, objects)
- typically its a very tool-intensive activity
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 5
Reconstruction, but how?
- using automated and semiautomated extraction tools
- probing the original design intent of the architect
→ combination of both techniques
→ in practise: reconstruction not fully automated
Introduction II
In practise: not fully automatic, rather an interpretive, interactive and iterative process involving many
activities and skills of experts and architects
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 6
First: workbench!
Phases:
1. Raw view extraction
2. Database construction
3. View fusion and manipulation.
4. Architecture analysis: finding violations
Architecture Reconstruction Process
→ requires skillfull application of tools, not possible with a single tool , e.x. implementations written in different scripting languages, no tool speaks every language
→ workbench: tool set – should be open (easy to integrate new tools), and provide an integration framework whereby new tools dont impact existing data and tools
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 7
„...analyzing a system's existing design and implementation artifacts to construct one or more models of it.“
Goals:
- extracting a target set of architectural views to support documentation effort
- answers on „what components are affected if I rewrite component A?“, „How to refactor layering to remove cyclic dependencies“
→ elements of interest (files, functions, variables) and their relationships come from source artifacts (code, header files, build files)
Raw view extraction
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 8
Source element
Relation Target element
Description
File includes File C preprocessor #include of one file by another
file contains Function Definition of a funcion in a file
A typical list of the elements and several relationships among them.
Raw view extraction II
→ Typical list of the elements and relationships that might be extracted
→ each relationship gives different information about the system
1. calls – helps to build a call graph2. includes – set of dependencies3. access_read access_write – how data is usedEtc.
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 9
Two categories of information obtained:
- static – observing only system artifacts
- dynamic – observing how the system runs
Goal: fusing both to create more accurate system views
Raw view extraction III
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 10
What if system changes at runtime?
→ capturing runtime configuration
Why capturing?
- dynamic elements
- late binding
- static reverse engineering not possible (peer-2-peer, SOA, cloud)
Raw view extraction IV
So, What is this „dynamic“-thing all about?
Why capturing dynamic elements?
- dynamic elements- It may be possible that system changes at runtime. In that case the runtime configuration should be captured and used when carrying out the reconstruction. Example: configuration file read in at startup and connections are made or elements executed.
- late binding – mechanism where a method being called upon an object is looked up by name at runtime ex. Polymorphism, plug-ins, pointers.
- peer-2-peer, SOA and cloud computing only dynamically, because precise topology of a system may not be determined until runtime. It depends on availability, loading , so cannot be reverse engineered using static tools. Therefore tools that generate dynamic info about the system.
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 12
Last, but not least → reduce irrelevant information
Example: operating-system specific libraries
Raw view extraction V
Before loading a raw view into the database → remove irrelevant information
For example: while building or using libraries that are OS-specific, you may be only interested in a specific OS (e.g. Linux) and discard the libraries for other platforms.
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 13
- storing extracted information in database (large amount of data, manipulations of data error-prone)
- when using a workbench (set of tools) choosing a database and deciding on internal representations of views required
- often extracted data from the code too complex → white noise
Database construction
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 14
• Raw views may be too specific to aid in architectural understanding.
Accurate, but not helpful
Database construction II
White noise: when extracted data from a code consists of classes and methods and inclusions and calling relations. Each element plotted on a grid, and each relation drawn as a line. Accurate view, but provides no insight into structures →
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 15
How to filter information?
Manipulating views (e.g. hiding methods inside class definitions)
Showing abstractions (e.g. identifing distinct layers)
Database construction III
Because of white noise → therefore manipulating such raw views to collapse information (ex. Hiding methods inside class definitions)
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 16
Views which have been extracted are manipulated to create fused views
e.g. static call view fused with dynamic call view to provide more accurate information about the part of system
Creating a fused view = creating a hypothesis about the architecture and a visualization of it
- views can be interpreted, further refined, or rejected
- no universal completion criteria for this process
View fusion
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 17
● Tool for static code review written in Java, support of other programming languages planed
● „Bridging the gap between the architects and developers“
● More http://www.hello2morrow.com/products/sonarj
Short excursion: SonarJ (Sonargraph for Java)
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 18
Short excursion: SonarJ (Sonargraph for Java)
→ Debt index: aggregated metrics to explain the effort to make system „clean“, „how difficult to remove all warnings and violations“
→ then lines of code etc.→ cyclicity, 30,98% of packages involved in cyclic dependencies.
By clicking on one, visualisation of cycle group will be shown→ below „how many spots to cut“ - to remove to package cycles→ other interesting metric: ACD (average component dependency)
shows how many files are going to be affected if I change this project
→ in warning box, general warnings e.g. 1 cycle group or code duplications (by clicking on it, system directly jumps to duplicated lines)
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 19
SonarJ extracts facts from a set of source code files and lets you define a set of layers and vertical slices through those layers.
View fusion: example of a fused view
Early results from the tool SonarJ.
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 20
View fusion phase gave us a lot of hypotheses– those inform of architectural elements (layers),
constraints, relationships
– Hypotheses need to be tested in order to check conformance with architect's intention
– Two major possibilities for maintaining conformance between code and architecture:
- conformance by construction
- conformance by analysis
Architecture analysis: finding violations
- by construction: automatically generation a substantial part of the system based on an architectural specification. Highly desirable, because tools guarantee conformance. Can be unfortunetely only applied when engineers can employ specific architecture-based development tools, languages. For systems requiring implementation from outside, this approach does not apply
- by analysis – technique to analyze system information to mark nonconforming elements, so they can be fixed: brought to conformance.
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 21
Previous figure with added relationships and constraints, which have been added by an architect to reflect design intent.
Architecture analysis II: example
In this example, the architect has indicated the relationships between the layers.
Using these relationships and constraints, a tool such as SonarJ is able to automatically detect and report violations of the layering in the software.
What does the figure tell us?
Data layer (row 2) can access, and hence depends on, the DSI layer.
Data layer may not access, and has no dependencies on, Domain, Service, or Controller (rows 1, 3, and 5).
The JUnit component in the “External” component is defined to be inaccessible.
No portion of the application should depend upon JUnit, because this should only be used by test code.
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 22
Black Arc: SonarJ has found a violation. The arc between the
Service layer and JUnit.
Architecture analysis III
This violation is found by SonarJ by searching through its database, applying the user-defined patterns, and finding violations of those patterns.
The highlighted arc indicates that this is an illegal dependency and architectural violation.
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 24
● Have a goal and a set of objectives or questions in mind before undertaking an architecture reconstruction project.
● Obtain some representation of the system before beginning the detailed reconstruction process.
● Identifying layers is a good place to start● In some cases: disregard the existing documentation● Tools can not do all the work! Involvement of people familiar
with the system required.
Guidelines
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 25
● Architecture reconstruction ensures that a system is built the way it was designed and that it evolves in a way that is consistent with developer's intentions.
● Reconstruction needed when:
- documentation lost or out-of-date
- checking conformance
- starting the reengineering process to a new desired architecture
Summary
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 26
● The software architecture reconstruction process comprises the following phases:
- raw view extraction
- database construction
- view fusion
- architecture analysis
Summary II
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 27
1. What inferences can you make about a system's module structures from examining a set of behavioral traces gathered dynamically?
Discussion
→dynamic analysis is performed by executing programs on a real or virtual processor
→ to be effective, sufficient test inputs needed; different techniques to ensure that a slice of set of possible behaviors has been observed e.g. code coverage (higher coverage, less bugs)
→ several steps: preparing input, running a test program, gathering parameters, analyzing the output
→ dynamic analysis is executed by passing a set of data to the input of the program : efficiency of test depends on quantity and quality of data
→ Metrics provided: - ressources consumed – time of program execution on whole or for
different modules, the number of external queries (e.g. to the database), the number of memory being used, other ressources
- cyclomatic complexity, degree of code coverage with tests, program metrics (on performance, reaction time, response)
- program errors (divisions by zero), null pointer dereferencing, memory leaks, race conditions (output depends on sequence or timing of other event)
- vulnerabilities in the program- other advantages: dynamic analysis detects dependencies that is not
possible for static, e.g. dynamic dependencies, polimorphism
Remember: even 100% coverage doesnt mean there are no errors at all! Dynamic analysis doesnt reveal logic errors and secondly, testing utility is not free of errors too.
Maciej Lukaszewski | WS 13/14Chapter 20: Architecture Reconstruction and Conformance| Folie 28
2. Suppose you hypothesize that a system makes use of particular tactics to achieve a particular quality attribute. Fill in the columns of the table below to show how you would go about verifying your hypothesis.
Discussion
Tactics for.. Interviewing experts on the system
Analyzing structure of source code files
Static analysis of source code
Dynamic analysis of system's execution
Availability
Interoperability
Modifiability
Performance
Security
Testability
Usability
→ Interviewing: availability („what are the SLA? What time is the system meant to be
aviailable? What's the minimum availiable time? What's the maximum downtime? What's the counted % of availability?“)
Performance/Usability: user behavior analysis, surveyInteroperability (work with other systems): does the system adhere to
interface standards? Is there an easy way to convert into different standard?
→ Structure of code: availability/performance („Bad smells“ which or duplicated code, which lead to delays)
Security (security checks and detecting security vulnerabilities using tools) Interoperability: known standard used? Same format? Same codification of
data?
→ Dynamic analysis of system's execution: availability (response time, downtime/delays due to late binding, causing availibility problems, runtime measurements)
Performance: ressources consumed (how much time needed to execute a certain module?)
Testability: memory leaks
→ static and dynamic analysis: tools designed specifically for finding security flaws e.g. sql-injection, spoofing and other security vulnerabilities. Leading tools e.g. Fortify or Coverity.
Top security vulnerabilities e.g. unvalidated parameters, broken access control or sessions, insecure use of cryptography.