software architecture erosion and modernization
DESCRIPTION
Software Architecture Erosion, Modernization and what you can do about itTRANSCRIPT
Page: 1
Bernhard Merkle Frederic Madiot
R&D Software Products Manager
SICK AG, Germany OBEO, France
Contact us on linkedin.com or xing.com
Stop the Software
Architecture Erosion
Page: 2
Agenda
– Software Architecture
– Detect Architectural Erosion
– Eclipse + Open Source Projects
– Model-Driven Reverse-Engineering
Page: 6
Architecture, Erosion
Page: 7
Software-Architecture: Definitions
IEEE 1471-2000:
The fundamental organization of a system,
embodied in its components,
their relationship to each other and the
environment,
and the principles governing its design and
evolution.
Page: 9
Architectural Erosion
“Sometimes the developers manage to
maintain this purity of design through the
initial development and into the first release.
More often something goes wrong.
The software starts to rot like a
piece of bad meat”.
Uncle Bob: “Agile Software Development”
Page: 10
Architectural Erosion
Why should we care ?
– In (lots of) Projects, Architecture decay happens
– We are not alone, as we‘ve some good representatives… ;-)
There is/was a plan
Page: 11
X-raying Software…
Page: 12
Findbugs: the first years
0.7.2
(03/2004)
0.8.6
(10/2004)
Page: 13
Findbugs: first erosion
0.8.7 (05/2005)
Page: 14
Tangle increase…
0.8.8
(05/2005)1.0.0
(06/2006)
Page: 15
Tangle increase…
1.3.0
(07/2007)
Page: 16
ONE BIG Tangle…
1.3.8 (03/2009)
Page: 17
Page: 20
Modeling Subsystems:
Page: 21
Fixing Architectural Violations
Page: 22
Fixing Architectural Violations
Page: 23
Fixing Architectural Violations
Page: 24
Fixing Architectural Violations
Page: 25
Fixing Architectural Violations
Page: 26
Fixing Architectural Violations
Page: 27
Fixing Architectural Violations
Page: 28
Tools for Architecture-Analysis
– SotoArc SonarJ
– Bauhaus Lattix
– Structure101 Klocwork, Coverity
– http://code.google.com/p/architecturerules/
– ODASA, CodeCrawler, Codecity, SeeSoft, XRadar, XDepend, SonarSource, Kalistick, Sqoring, …
– Eclipse-Based: MoDisco (+Modernization of Applications)
http://se-radio.net/
episode-115-architecture-analysis
Page: 29
Basic Approaches (Dependencies)
– makedepend, jdepend
– Eclipse (Java Build Path)
– OSGI (Dependencies)
Page: 30
Basic Approaches
PDE Dependency Visualization
Page: 31
Missing in basic approaches:
Architecture Analysis (Deviation)
Drill-Down + Aggregation
Displaying results
Monitoring changes, trends
Rating of Architecture
� Requirements for Architectural Analysis Tools
Page: 32
Architecture Analysis (deviation)
Should-
Architecture
Requirements
Architecture-
Design
Comparison “Diff-”
ArchitectureActions
Extraction Is-
Architecture
Existing Code
•Violations
•Conformance
Page: 33
Displaying results:
Page: 34
How to replace
key people ?
How to maintain
confidence with users?How to develop
a market ?
How to augment
the development
and support load ?
Open-SourceProject
Architecture: critical for OS-projects
Marketplace
Page: 35
Openess limits Erosion
� Developers expose their reputation
� Names are associated to the architecture
� Community can provide feedback
� Warnings, Recommendations, ...
Page: 36
Risks of Erosion in FOSS
� Contributors from several organizations
� Different cultures, processes, tools, …
� Lower pressure from management
� Indirect Business
� Hazardous funding
� Difficulties to calculate costs and benefits
Page: 37
Eclipse: Architectural Analysis
Page: 38
Eclipse: 10 Years Legacy System
Eclipse Helios = 33M lines of code
Page: 39
Eclipse Architecture
Page: 40
Eclipse Architecture
Page: 41
Eclipse Architecture
Page: 42
E3.4: Plattf:Ant ���� JDT:*
Page: 43
E3.4: Plattf:Ant ���� JDT:UI
Page: 44
Plattform: CVS ���� Workb (internal)
Page: 45
Team-UI����UI-workbench (internal)
https://bugs.eclipse.org/bugs/show_bug.cgi?id=90582
Page: 46
Antipatterns
Page: 47
Dependent BaseClass
– Type:
• Design Problem
– Problem:
• one of more Methods shall implement different
behavior, depending on the type, passed in
– Context:
• make “extensible” systems, frameworks
– Forces:
• Programming languages offer, instanceof/typeid funcs.
– Antipattern:
• Methods of the baseclass, depend on derived classes, e.g. accessing
their members, doing switch/case depending on type information
Page: 48
AntiPatterns / Bad Smells:
Metrics/1000 Classes Eclipse JDK
Dependent Baseclass: 1 16
Multiple Interface Inher. 4 18
Abstractable Methods 80 60
Abstractable Attributes 45 170
Unused Classes 50 150
Unused Methods 950 2500
Unused Attributes 30 20
Page: 49
Code ClonesCode
Clones
Page: 50
Code Clones
– identical Files
• E2.0 JDT,CDT
– jdt\debug\internal\ui\dialogfields\ListDialogField.java
– cdt\debug\internal\ui\dialogfields\ListDialogField.java
• E3.4 CDT: identical packages
– cdt\debug\internal\ui\dialogfields,
cdt\debug\mi\internal\ui\dialogfields
– variation of former identical Files
• E3.4 JDT,CDT
– jdt\debug\internal\ui\dialogfields\ListDialogField.java
– cdt\debug\internal\ui\dialogfields\ListDialogField.java
Page: 51
Rating Eclipse Architecture
Minor Erosions, Code Duplication
– large codebase,
– OSGI helps a lot
– API Police,
– Guidelines
– upcoming PDE tools
– Processes and Tools
Page: 52
•Community Practices
• Engage your user community
•Software Development Practices
• Unit Test, Unit Test, Unit Test
• Continuous Integration
• Integrate in small steps
Architecture Council Recommend.
•Architectural Recommendations
• Be asynchronous
• Think API
• Long-running operations should be
cancelable
• Separate policy and mechanism
• Keep simple things simple
• Create Unittests early
• Minimize plug-in dependencies
• Be aware of the deployment context
• Package coherence
• Putting only related things together
http://wiki.eclipse.org/Architecture_Council/Top_Ten_Project_Development_Practiceshttp://wiki.eclipse.org/Architecture_Council/Top_Ten_Recommendations
Page: 53
Yearly Simultaneous Releases Rules
� Projects should leverage only published APIs of
dependencies
� Version number ends with « qualifier »
� Source code must use ICU4J classes
� The project must contain an « about.html » file
� Packages name should start with the plug-in ID
� Plug-in must not contain JARs files
� Plug-in should contain only one
« message.properties » and « Message.java » files
� Etc…
http://www.eclipse.org/indigo/planning/EclipseSimultaneousRelease.php
Page: 54
New Eclipse projects …
driving to new architectures
Components that are more reusable + customizable� Service-oriented programming model
� GUI represented by a model and configurable with CSS
� Enabling Javascript app to be executed by Eclipse runtime
� Framework to build SWT app declarativeley
Ajax-enabled web applications by using a subset of RCP APIs� Eclipse development model
� Plug-ins & Eclipse workbench extension points
� Widget toolkit using SWT API
Browser-based tools to develop for the web, in the web� Client: loosely coupled components written in JavaScript
� Server: services exposed via REST-oriented HTTP APIs
Page: 55
Manage architectural transition?
Architecture A Architecture B
?
Page: 56
Evolution to e4
Backward Compatibility Layer
• Eclipse 3.x plug-ins run on e4
Challenges
• Not clean APIs usage?
� Refactoring
• Using e4 development model?
� Migration
• Single sourcing?
�Forward Compat. Layer(Session: Singlesourcing for Eclipse 4.x and Eclipse 3.x)
Page: 57
Evolution to RAP
Single sourcing with RAP is not automatic
• API differences
• Missing extension points
• Application startup and Activator scope
• API Differences
• Field validation
• SWT ressources
• Singletons and Scopes
• Jobs and background threads
• Internationalization and localization
(http://eclipsesource.com/fileadmin/doc/2009_product/single_sourcing_guide.pdf)
� Refactoring
Page: 58
How to modernizeexisting Eclipse plug-ins?
Page: 59
Architectural Modernization Process
Legacy
Software
System
Modernized
Software
System
Audit
Reference
Tests
Modernization
Strategy
Transformation
& Integration
Non-Regression
Tests
Transformation
Tooling
Page: 60
Anatomy of an Eclipse 3.x Plug-in
MANIFEST.MF
plugin.xml
Source code
build.properties
plugin.properties
.project
.classpath
files
folders
Eclipse Plug-in
Hetero
geneity!
Page: 61
Model-Driven
Reverse-Engineering
of Eclipse plug-ins
Page: 62
Software artifacts :
- source code- configuration files
- tests- database- …
Existing
Software SystemDiscover
Models
use Models to represent and manipulate
artifacts of existing systems
Understand
Viewpoints
Transform
New
Software System
MoDisco
http://www.eclipse.org/MoDisco/
Page: 63
Supported Technologies
Infrastructure
MoDisco Architecture
Java �Metamodel�Discoverer�Generator�Transfo. to KDM
XML�Metamodel�Discoverer�Generator
Model Browser
Navigation through complex models
Customization& Extensibility
Definition ofspecific Viewpoints
DiscoveryManager
Plug and orchestratetransformations
Eclipse Modeling projects
JSP�Metamodel�Discoverer�Generator
OMG/ADMStandards
Pivot
Metamodels(SMM & KDM)
EclipsePlugin
�Metamodel�Discoverer
Page: 64
Using EMF to describe a Plug-in
MANIFEST.MF
plugin.xml
Source code
build.properties
plugin.properties
.project
.classpath
files
folders
Eclipse Plug-in
Project’s structure
(KDMSource)
.project (XML)
.classpath (XML)
manifest
build.properties
(KDMCore)
plugin(eclipseplugin)
Java source code
(Java)
plugin.properties
(KDMCore)
extensions (XML)Homogeneity
!
Page: 65
Using EMF to describe a Plug-in
Page: 66
What can you do with
the EMF model
of a plug-in ?
Page: 67
EMF Modelof an Eclipse plug-in
Leverage Eclipse Modeling components
Generate code
(Acceleo, Xpand, Jet)
Generate code
(Acceleo, Xpand, Jet)
Transform
(ATL, QVTo)
Transform
(ATL, QVTo)
Compare
(EMF Compare)
Compare
(EMF Compare)
Inspect
(MoDisco Browser, EEF)
Inspect
(MoDisco Browser, EEF)
Vizualize
(GMF, Graphiti)
Vizualize
(GMF, Graphiti)
Query
(EMF Query)
Query
(EMF Query)
Page: 68
Architectural Modernization Process
• Quality Analysis
• Volumetry
• Cartography
Legacy
Software
System
Legacy
Software
System
Modernized
Software
System
Modernized
Software
System
Audit
Reference
Tests
Reference
Tests
Modernization
Strategy
Modernization
StrategyTransformation
& Integration
Transformation
& Integration
Non-Regression
Tests
Non-Regression
Tests
Transformation
Tooling
Transformation
Tooling
Page: 69
Validation
Page: 70
Architecture Visualization
GMF diagram
created with ObeoDesigner
• purple: EMF Model
• green: UI
Page: 71
Architectural Modernization Process
Legacy
Software
System
Legacy
Software
System
Modernized
Software
System
Modernized
Software
System
AuditAudit
Reference
Tests
Reference
Tests
Modernization
Strategy
Modernization
StrategyTransformation
& Integration
Transformation
& Integration
Non-Regression
Tests
Non-Regression
Tests
Transformation
Tooling
• Parsing
• Transformation rules
• Re-generation
Page: 72
Refactoring & Migration
Transformation Rules
What to change + How to change
• Renaming
• Changing code structure
(inheritance, attributes, methods, etc)
• Replacing method calls
• Changing instructions order
• Etc
Transformed
Plug-in
ExistingPlug-in
Model of the existing
Plug-in
Model of the migrated
Plug-in
Page: 73
Architectural Modernization Process
Legacy
Software
System
Legacy
Software
System
Modernized
Software
System
Modernized
Software
System
AuditAudit
Reference
Tests
Modernization
Strategy
Modernization
StrategyTransformation
& Integration
Transformation
& Integration
Non-Regression
Tests
Non-Regression
Tests
Transformation
Tooling
Transformation
Tooling
• Tests coverage analysis
Page: 74
Test coverage analysis
Page: 75
Architectural Modernization Process
Legacy
Software
System
Legacy
Software
System
Modernized
Software
System
Modernized
Software
System
AuditAudit
Reference
Tests
Reference
Tests
Modernization
Strategy
Modernization
Strategy
Transformation
& Integration
Non-Regression
Tests
Non-Regression
Tests
Transformation
Tooling
Transformation
Tooling
• Automatic transformation
• Manual transformation
• Build
Page: 76
Build
B3 Model(build configuration)
Plug-ins to build
Model of plugins to build
Team information(CVS, SVN, etc)
Update sites
(p2)
Model of
update sites content
Build Strategies
IDM++ Research Project (ANR) -> Sept 2011
constraints solver
Page: 77
Summary
Stop the Software Architecture Erosion ?
Analysis is Required
– Evaluate the architectural situation
– Take the right decisions at the right time
Modernization is Required
– Correct erosion consequences
– Go with architectural Evolutions
=> Continuous Analysis + Modernization
Page: 78
Thank you !