detecting and quantifying architectural debt

Yuanfang Cai & Rick Kazman

May 2017 ICSE Technical Briefing

Detecting and Quantifying Architectural Debt:

Theory and Practice

1© Yuanfang Cai, Rick Kazman, 2017

The Gap between Theory and Practice

Cunningham’s

Debt Metaphor [Cunningham 1992]

Conway’s Law [Conway 1968 ]

Parnas’s Theory [Parnas 1972]

Baldwin and Clark’s

Design Rule Theory

[Baldwin and Clark 1999 ]

2 © Kazman, Cai 2017

The Gap between Theory and Practice

Manager Developer

We need to

refactor !

What is the

We have

technical debt!

How much is

the debt?

Our objective

Towards a Quantifiable Software Architecture Health Monitoring

System, based on principled theoretic foundations

The Concept of Technical Debt

Software complexity accumulates slowly as part of normal

development activities.

Because it accumulates slowly, most developers are unaware that it

is occurring: they are focused on their immediate tasks.

However, quick-and-dirty changes, over time, lead to problematic

structures in a code base.

The impact of design decisions on software maintenance and

evolution have not been well studied or understood.

Technical Debt vs. Real Debt

Debt in Life Technical Debt

Where are the debts?

How much extra maintenance cost?

How fast does penalty accumulate over time (interest rate)?

Cost: 1,500$

Interest: 14.24%

Cost: 20,000$

Interest: 4.46%

Cost: 1000,000$

Interest: 4.75%

Design Rule Theory [Baldwin and Clark 1999]

Design Rules create Modules

Modules creates values in the form of options

Investing in modularization (refactoring) = Buying

options

Design Rule Theory

Design Rules and True Modules

Modular Operator: Splitting Design rules split the rest of the system into true modules

True modules only depend on design rules, but not on each other

A and B-C are proto-modules A and B-C become true modules

Splittingby adding new design rules

The key idea of decoupling is to insert design rules that

(1) create true modules, so that

(2) true modules only depend on design rules, but not on each other.

Design Rule Theory

Modular Operator: Substitution

A true module can be substituted with a new version

True modules can be improved, changed, or even replaced, without

influencing other parts of the system

Module-wise improvement can be accomplished independently and in

parallel, as along as design rules remain unchanged

Design Rule Theory

Modules create options

A designer has the right, but not the obligation to improve a module (similar

to real options)

The value of the system increases as the value of each module increases

independently

The value of a system with n modules:

V = S0 + NOV1 + NOV2 +…+ NOVn

Design Rule Theory

The value of the system:

The Net Option Value of each module

Where:

• NOVi: Net option value of module i

• i: Technical potential of module i

• n: The complexity of module i

• Q(ki): The probability of getting the highest value of module i after k experiments

• Ci(ni)ki : The cost of k experiment of module i of complexity n

• Zi: The cost of revising the modules depending on module i

V = S0 + NOV1 + NOV2 +…+ NOVn

NOVi = maxki {ini1/2 Q(ki) – Ci(ni)ki –Zi}

The essence of the theory

The more independent modules are there, the higher the value of the

system

The fewer dependencies a module has, the higher the value

The option value of a module increases and then decreases as the

complexity of the module increases

The more active (technical potential) a module is, the higher its option

value.

V = S0 + NOV1 + NOV2 +…+ NOVn ;

NOVi = maxki {ini1/2 Q(ki) – Ci(ni)ki –Zi}

Design Rule Theory

From Theory to Principles

The visualization and quantification of critical design principles

Information hiding

Hierarchical structure

SOLID principles

Single responsibility

Open for extension, closed for modification

Liskov substitution principle

Interface segregation principle

Dependency Inversion principle

The features of well-modularized systems explained in DRT and DSM

Information Hiding Explained

Key concepts from Parnas:

Information hiding: hiding changeable decisions behind stable

interfaces

Modules: defined as independent task assignments

Explained in Design Rule Theory and DSM

Stable interfaces-> design rules

Modules-> the source of option value

Baldwin and Clark explained information hiding in design rule theory

Visualization using DSMs

X Y Z A D G J B E H K C F I L M

X - Computer .

Y - Corpus X . X

Z - User X .

A - In Type .

D - Circ Type .

G - Alph Type .

J - Out Type .

B -In Data X X . X X

E - Circ Data X X X . X

H - Alph Data X X X X .

K - Out Data X X .

C - In Alg X X X X .

F - Circ Alg X X X X X .

I - Alph Alg X X X X X X X .

L - Out Alg X X X X X X .

M - Master X X X X X .

Sequential Design in DSMs [Sullivan et al. 2001]

Sequential Design

Each module is modeled as 3

elements:

Function signature

Data structure

Detailed implementation

Design Rules

Function signature and data

structure

Modules:

Detailed implementation

X - Computer .

Y - Corpus X . X

Z - User X .

A - In Type .

D - Circ Type .

G - Alph Type .

J - Out Type .

K - Out Data X X .

Sequential Design in DSMs [Sullivan et al. 2001]

Environment Parameters

Modeling the dimensions that

are likely to change

Extracted from documentation

or added by architects

Changes start from

requirements

Same environment parameters

applied to both designs

X Y Z N A D G J O P B C E F H I K L M

X - Computer .

Y - Corpus X . X

Z - User X .

N - Line Type .

A - In Type .

D - Circ Type .

G - Alph Type .

J - Out Type .

O - Line Data X X X . X

P - Line Alg X X X X .

B - Input Data X X X . X

C - Input Alg X X X X X .

E - Circ Data X X X X . X

H - Alph Data X X X X . X

I - Alph Alg X X X X X X .

K - Out Data X X X . X

L - Out Alg X X X X X .

M - Master X X X X X X .

Information Hiding Design in DSMs [Sullivan et al. 2001]

Information hiding:

changeable environmental

parameters do not affect

design rules

X - Computer .

Y - Corpus X . X

Z - User X .

A - In Type .

D - Circ Type .

G - Alph Type .

J - Out Type .

K - Out Data X X .

X Y Z N A D G J O P B C E F H I K L M

X - Computer .

Y - Corpus X . X

Z - User X .

N - Line Type .

A - In Type .

D - Circ Type .

G - Alph Type .

J - Out Type .

O - Line Data X X X . X

P - Line Alg X X X X .

B - Input Data X X X . X

C - Input Alg X X X X X .

E - Circ Data X X X X . X

H - Alph Data X X X X . X

I - Alph Alg X X X X X X .

K - Out Data X X X . X

L - Out Alg X X X X X .

M - Master X X X X X X .

(A) Sequential Design

NOV = 0.26

(B) Information Hiding Design

NOV = 1.56

Parnas’s Two Designs in DSMs [Sullivan et al. 2001]

Quantifying two designs using Net Option Theory

The Information Hiding Design has much higher option values

Pieces to move from Theory to Practice

Architectural Model [Xiao et al. 2014]

New Metric [Ran et al. 2016]

Localization [Ran el al. 2015, Xiao et al. 2016]

Option to buy

Debts to Pay

Debt Quantification [Kazman et al. 2015, Xiao et al. 2016]

Price of an option

Principal, penalty, interest rate

Option to buy

Debts to Pay

Price of an option

A New Architecture Model [Xiao et al. 2014]

DR-based Architectural Model

Design Rule Space (DRSpace): Modeling

software as multiple overlapping spaces

Multiple Design Spaces in Software

Intuitively, a nontrivial software system must contain multiple

design spaces:

each feature implemented

each pattern applied

each concern addressed

Each file can participate in multiple design spaces

A design space may cross multiple packages

We propose that software architecture can, and should be

modeled as multiple, overlapping design spaces

Multiple Design Spaces in Software

Each design space should have its own structure

Design Rules

Modules

Layers

Each design space may evolve independently

Each design space should be measured and managed separately

These design spaces are implemented in source code and get

Each design space can be visualized using Design Structure Matrix

Design Rule Space (DRSpace) [Xiao et al. 2014]

A subset of files

Any subset of files may form a design space

Architectural connections

Structural couplings: call, inherit, aggregate, etc..

Evolutionary couplings

One or multiple types

Implicit or explicit

Design Rule Space A DRSpace is composed of a meaningful subset of a system’s files along with the architectural connections among these files.

Design Rule Space (DRSpace)

A DRSpace defines the following concepts:

1. A DRSpace contains one or more leading files:

The files that all other files in the space directly or indirectly depend on

Leading files may or may not be design rule files

2. A DRSpace contains a selected set of dependency types, either

structural or evolutionary coupling

3. A DRSpace can be clustered into Design Rule Hierarchy (DRH) [Cai and

Sullivan 2006, Wong et al. 2009], using one or more architectural

relations

Primary relation: selected, DRH-clustered dependency type

Secondary relation: other dependency types

The small calculator design spaces:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

1 mij.bnf.Node (1)

2 mij.io.Pipe (2)

3 mij.ast.Node (3)

4 mij.Filter (4)

5 mij.io.InputPipe ext (5)

6 mij.io.OutputPipe ext (6)

7 mij.io.WriterOutputPipe Impl (7)

8 mij.io.MemoryOutputPipe Impl (8)

9 mij.io.ReaderInputPipe Impl (9)

10 mij.io.MemoryInputPipe Impl (10)

11 mij.ast.TreeVisitor (11)

12 mij.Interpreter Impl Impl (12)

13 mij.parse.Parser Impl (13)

14 mij.lex.Lexer Impl (14)

15 mij.parse.Convert Impl (15)

16 mij.ast.Variable ext (16)

17 mij.ast.Number ext (17)

18 mij.ast.OperExpr ext (18)

19 mij.ast.FuncExpr ext (19)

20 mij.ast.UnaryOperExpr ext (20)

21 mij.bnf.ValueExpr ext (21)

22 mij.bnf.AddExpr ext (22)

23 mij.bnf.ParamExpr ext (23)

24 mij.bnf.UnaryExpr ext (24)

25 mij.bnf.LexExpr ext (25)

26 mij.bnf.ExponExpr ext (26)

27 mij.bnf.MultExpr ext (27)

File set

File 5: mij.io.InputPipe extends File 2 mij.io.Pipe

DRSpace 1: Polymorphism DRSpace• formed by “extend” and “implement”

File 7: mij.io.WriterOutputPipe implements File 6 mij.io.OutputPipe

Design Rule Space (DRSpace) Modeling

The small calculator example:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

1 mij.bnf.Node (1)

2 mij.io.Pipe (2)

3 mij.ast.Node (3)

4 mij.Filter (4)

5 mij.io.InputPipe ext (5)

6 mij.io.OutputPipe ext (6)

7 mij.io.WriterOutputPipe Impl (7)

8 mij.io.MemoryOutputPipe Impl (8)

9 mij.io.ReaderInputPipe Impl (9)

10 mij.io.MemoryInputPipe Impl (10)

12 mij.Interpreter Impl Impl (12)

13 mij.parse.Parser Impl (13)

14 mij.lex.Lexer Impl (14)

15 mij.parse.Convert Impl (15)

16 mij.ast.Variable ext (16)

17 mij.ast.Number ext (17)

18 mij.ast.OperExpr ext (18)

19 mij.ast.FuncExpr ext (19)

20 mij.ast.UnaryOperExpr ext (20)

21 mij.bnf.ValueExpr ext (21)

22 mij.bnf.AddExpr ext (22)

23 mij.bnf.ParamExpr ext (23)

24 mij.bnf.UnaryExpr ext (24)

25 mij.bnf.LexExpr ext (25)

26 mij.bnf.ExponExpr ext (26)

27 mij.bnf.MultExpr ext (27)

4 leading files and Design Rules

OutputPipe Module

InputPipe Module

Visitor Module

Filter Module

Ast Node Module

Bnf Node Module

DRSpace 1: Polymorphism DRSpace• formed by “extend” and “implement”

1 2 3 4 5 6 7 8 9 10

1 mij.ast.Node (1)

3 mij.ast.Variable Ext, dp dp (3)

4 mij.ast.UnaryOperExpr Ext, dp dp (4)

5 mij.ast.OperExpr Ext, dp dp (5)

6 mij.ast.Number Ext, dp dp (6)

7 mij.ast.FuncExpr Ext, dp dp (7)

8 mij.Interpreter dp Imp dp (8)

9 mij.parse.Convert dp dp dp dp dp dp dp (9)

10 mij.Console dp dp dp (10)

Design Rule Space (DRSpace) Modeling The small calculator example:

Ext: Extend Impl: Implement dp: General dependencies such as call, use, and create

DRSpace 2: Visitor Pattern DRSpace• formed by multiple dependency types

Visitor interface: mij.ast.TreeVisitor

Element interface: mij.ast.Node

Concrete elements of the pattern: m(rc3-7)

Concrete visitor role: Calculator

1 2 3 4 5 6 7 8 9 10

1 mij.ast.Node (1)

3 mij.ast.Variable Ext, dp dp (3)

4 mij.ast.UnaryOperExpr Ext, dp dp (4)

5 mij.ast.OperExpr Ext, dp dp (5)

6 mij.ast.Number Ext, dp dp (6)

7 mij.ast.FuncExpr Ext, dp dp (7)

8 mij.Interpreter dp Imp dp (8)

DRSpace 2: Visitor Pattern DRSpace• Forming a DRH with all dependency as the primary relation

DRSpace 2: Visitor Pattern DRSpace• Evolutionary coupling is added as the secondary relation

1 2 3 4 5 6 7 8 9 10

1 mij.ast.Node (1)

3 mij.ast.Variable Ext, dp, 5 dp (3)

4 mij.ast.UnaryOperExpr Ext, dp, 10 dp (4)

5 mij.ast.OperExpr Ext, dp, 25 dp 5 (5)

6 mij.ast.Number Ext, dp, 8 dp 20 (6)

7 mij.ast.FuncExpr Ext, dp, 6 dp 15 (7)

8 mij.Interpreter dp, 10 Imp dp (8)

The small calculator example: There can be many more DRSpaces in this small example, formed

by each dependency type, each pattern, and each feature

ag: aggregation

DRSpace 3: Aggregation DRSpace

Lex Module

Memory Buffer Module

Module

communicate

through pipe

Five leading classes in first layer

Three meaningful modules in second layer

DRSpace 4: Dependency DRSpace

dp: dependency

Parsing Function

Console

Function

The small calculator example: There can be many more DRSpaces in this small example, formed

by each dependency type, each pattern, and each feature

DRSpaces overlap with each other

Primary: AggregationPrimary: Realize

Inherit

Aggregation

Depend

Leading Classes

Viewing the evolutionary coupling and the modular

structure of a DRSpace simultaneously helps to reveal

flawed architectural connections.

1 2 3 4 5 6 7 8 9 10

1 mij.ast.Node (1)

3 mij.ast.Variable Ext, dp, 5 dp (3)

4 mij.ast.UnaryOperExpr Ext, dp, 10 dp (4)

5 mij.ast.OperExpr Ext, dp, 25 dp 5 (5)

6 mij.ast.Number Ext, dp, 8 dp 20 (6)

7 mij.ast.FuncExpr Ext, dp, 6 dp 15 (7)

8 mij.Interpreter dp, 10 Imp dp (8)

Three primary dependency:

inherit/implement, aggregate and depend

One secondary dependency:

Evolutionary dependency (faked)

Display modularity violation

Secondary dependency: Evolutionary coupling

OperExpr change with TreeVisitor for 7 times

Evolutionary coupling without

structure dependency

DRSpaces in Practice

A DRSpace in Camel

Camel ObjectHelper DRSpace

Jboss JDBCCMRFieldBridge DRSpace

A DRSpace in JBoss

A DRSpace in Hadoop

Hadoop FileSystem DRSpace

Option to buy

Debts to Pay

Debt Quantification [Kazman et al. 2015, xiao et al. 2016]

Price of an option

A New Metric [Ran et al. 2016]

DR-based Architectural Metrics

Decoupling Level (DL: an options-based metric,

measuring the system’s ability to generate options)

Software “metrics”:

Lines of Code (LOC)

CK metrics

McCabe complexity

Real metrics

Kilogram

Towards a Real Metric

Cross project Comparison

Precisely tell maintainability

differences

Independent of project domains,

sizes, ages, etc.

Alice:

Degradation Monitoring

Remain stable for non-refactored

versions

Non-trivial changes should reflect

substantial architecture variation

Architectural debt thermometer

Maintainability Health Chart

Similar to real health chart

An industrial benchmark 50th percentileBob

Quantification: the Essence of DRT

Module: A true module (with high option value) should be

Independent

System: A highly modularized system should

Have large numbers of true modules…

connected by design rules

Independence Level [Ran et al. 2016]

A simple measure of true modules1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

1 UI_java (1)

2 Answer_java x (2) x

3 Question_java x x (3)

4 Survey_java x (4)

5 SaveLoadFile_java x (5)

6 TextFileUI_java x (6)

7 CommandLineUI_javax (7)

8 Letters_java (8)

9 Match_java x x x x (9) x

10 MatchingAnswer_javax x x x x (10)

11 ChoiceAnswer_java x x x (11) x

12 Choice_java x x x x (12)

13 EssayAnswer_java x x x (13) x

14 Written_java x x x x (14)

15 Test_java x x x (15)

16 AnswerSheet_java x (16)

#Files in the Last Layer

#AllFilesIndependence Level (DL)

The more files are

clustered into true

module, the higher the

Decoupling Level

Improved from Independence Level (IL).

Measuring how well a system is decoupled:

Measure a module

How small is it?

Is it independently changeable?

Measure a system

How many true modules are there?

How tightly coupled are they?

Decoupling Level

Based on Design Rule Hierarchy (DRH)

Distinguish different types of modules:

Upper layer modules vs. True modules1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

1 UI_java (1)

4 Survey_java x (4)

8 Letters_java (8)

Upper Layer modules:

• The fewer dependents,

the higher the value

• The larger the module,

𝑫𝑳𝑳𝒊 =

𝒋=𝟏

[#𝑭𝒊𝒍𝒆𝒔 𝑴𝒋

#𝑨𝒍𝒍𝑭𝒊𝒍𝒆𝒔∗ (𝟏 −

#𝑫𝒆𝒑𝒔(𝑴𝒋)

#𝑳𝒐𝒘𝒆𝒓𝑳𝒂𝒚𝒆𝒓𝑭𝒊𝒍𝒆𝒔)]

Decoupling Level

Distinguish different types of modules:

Upper layer modules vs. True modules

True modules:

• The smaller a true module,

• The more true modules, the

higher the value

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

1 UI_java (1)

4 Survey_java x (4)

8 Letters_java (8)

True Modules

𝑫𝑳𝑳𝒊=𝒏 =

𝒋=𝟏

𝑺𝒊𝒛𝒆𝑭𝒂𝒄𝒕𝒐𝒓(𝑴𝒋)

𝑆𝑖𝑧𝑒𝐹𝑎𝑐𝑡𝑜𝑟 𝑀𝑗 =#𝐹𝑖𝑙𝑒𝑠(𝑀𝑗)

#𝐴𝑙𝑙𝐹𝑖𝑙𝑒𝑠

𝑆𝑖𝑧𝑒𝐹𝑎𝑐𝑡𝑜𝑟 𝑀𝑗 =#𝐹𝑖𝑙𝑒𝑠(𝑀𝑗)

#𝐴𝑙𝑙𝐹𝑖𝑙𝑒𝑠∗ 𝑙𝑜𝑔5(#𝐹𝑖𝑙𝑒𝑠 𝑀𝑗 )

𝒘𝒉𝒆𝒏 #𝐹𝑖𝑙𝑒𝑠 𝑀𝑗 ≤ 5

𝒘𝒉𝒆𝒏 #𝐹𝑖𝑙𝑒𝑠 𝑀𝑗 > 5

Decoupling Level

The value of the system

The system:

• The more true

modules, the smaller

they are, and the less

coupled, the higher

the value

𝑫𝑳 =

𝑳𝒊=𝟏

𝑫𝑳𝑳𝒊

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

1 UI_java (1)

4 Survey_java x (4)

8 Letters_java (8)

Given a DRH with n layers, its DL is equal

to the sum of the DLs of all the layers:

Case Study using 129 projects

Evaluating DL to assess if it can support:

Cross-project Comparison

Evolution Monitoring

Reflecting real maintainability

Compared with

Independence Level (IL)

Propagation Cost (PC)

The DLs of 129 projects

108 open source, 21 industrial

0 0.2 0.4 0.6 0.8 1

Decoupling Level

0 0.2 0.4 0.6 0.8 1

Decoupling Level

80th percentile

DL: 75%

20th percentile

DL: 46%

60% of these

projects have DLs

between 46%

and 75%

Open Source(%) Commercial(%)

DL PC IL DL PC IL

Avg 60 20 43 54 21 35

Median 58 18 41 56 20 28

Max 92 72 100 93 50 83

Min 14 2 12 15 2 9

20th Pt 47 8 28 36 6 24

40th Pt 55 14 37 46 17 26

60th Pt 66 21 45 59 24 38

80th Pt 75 34 55 65 35 46

Pt: Percentile

• Best DL (93%) is

from industrial

• Worst DL (14%) is

from open source

Towards a “Health Chart”

A potential “Maintainability Health Chart”

An industrial project:

DL: 29%, 10th percentile

Confirmed to have severe maintenance difficulty

0 0.2 0.4 0.6 0.8 1

Decoupling Level

10th percentile

55© Kazman, Cai 2017

Will DL values remain stable for consecutive, non-refactoring

releases ?

Can DL indicate non-trivial architecture variation ?

Day 1 Day 6

Will DL values remain stable for consecutive, non-refactoring

releases ?

Studied 16 projects:

13 open source, 3 commercial

Each having 4-13 releases

From 236 to 9011 files

Avg CV of DL is

only 2%

DL is more stable

than PC and IL

Can non-trivial variation in DL faithfully indicate major architecture

variation?

Studied the evolution of one industrial project

Evolved for 6 year, 29 releases, 1082-1852 files

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

Decoupling Level

Release

4 transition points

(1) DL increases from 45% to 74%:

Transforming from a prototype to a real product

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

Decoupling Level

Release

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

Decoupling Level

Release

(2) DL decreases from 78% to 68%:

New features were added

Technical debt accumulates

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

Decoupling Level

Release

(3) DL decreases from 65% to 48%:

Unsuccessful refactoring

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

Decoupling Level

(3)(4)

Release

(4) DL increases from 48% to 62%:

“Cleaning up” technical debt

Can DL Reflect Real Maintainability?

Are software systems with higher DLs more maintainable ?

Studied 41 projects

38 open source, 3 commercial

All releases

63-11130 files

#Committers: 18-915

#Commits: 346-74269

Can DL Reflect Real Maintainability?

The higher the DL

The more files were changed

in isolation / in parallel by different developers

The more likely developers were able to work

independently without communication overhead

CCOR BCOR CCFOR BCFOR CPCO BPCO

DL -0.74 -0.77 -0.67 -0.70 -0.61 -0.63

PC 0.68 0.65 0.62 0.59 0.54 0.53

IL -0.48 -0.48 -0.49 -0.52 -0.42 -0.45

Figure: DL shows strongly negative correlations with 3 maintainability indicators

Option to buy

Debts to Pay

Debt Quantification [Kazman et al. 2015, xiao et al. 2016]

Price of an option

Localization

Architectural Roots [Xiao el al. 2014]

The top 5 file groups

Hotspot Pattern [Ran et al. 2015]

Architecture Issues

Architectural Debt [Xiao et al. 2016]

Consecutive Debts

Localization

Architecture Issues

Consecutive Debts

The Application of DRSpaces [Xiao et al. 2014]

What new insights can DRSpace modeling bring to

software development practice?

Research Questions

RQ1: If the leading file of a DRSpace is error-prone,

are the files in the DRSpace also error-prone?

RQ2: Are the majority of the error-prone files

concentrated in a few DRSpaces?

RQ3: By combining evolution and structure

coupling, can we get more insight into architectural

problems? Can this help us find not just the

locations of errors, but also the reasons for them?

Case Studies

Subject History #Files #Releases #Commits #Issues

JBoss 3.2.4 Apr 00 – Jun 04 762 11 6458 550

Hadoop 0.15 Feb 06 – Oct 07 692 15 3001 490

Eclipse 3.0.2 May 01 – Mar 05 3,704 10 27,806 3,458

Subjects

These are the first 3 projects we studied

The results hold for hundreds of projects we have

studied so far

Research Question 1:

If the leading file of a DRSpace is error-prone,

are the files in the DRSpace also error-prone?

RQ1: Answer We selected DRSpaces led by the top 30 most error-prone

files as design rules

We then sub-selected just DRSpaces with at least 10 files

We computed dsb and bsc for each project with respect to

Bug2, Bug5 and Bug10

Conventions:

Bug Space: BugN—the files that were changed for bug fixing N times or more

Design Space bugginess: dsb---the percentage of files that are also in BugN

Bug Space coverage: bsc---the percentage of files in BugN covered by the DRSpace

RQ1: Answer

We selected DRSpaces led by the top 30 most error-prone

files as design rules

We then sub-selected just DRSpaces with at least 10 files

We computed dsb and bsc for each project with respect to

Bug2, Bug5 and Bug10

Consider JBoss: 9 out of 30 DRSpaces have at least 10 files

62% of the files in each selected DRSpace have more than 2 bug fixes.

The average JBoss DRSpace contains 10% of the project's Bug2 files.

RQ1: Answer cont’dJBoss:

• Consider org.jboss.ejb.Container: • 21 bug fixes; 4th most error-prone file • Its DRSpace contains 56 files

• 32 files out of the 56 files have more than 2 bug fixes• dsb = 32/56 = 57%• bsc = 32/206 = 16%

• JBoss's Bug2 space has 206 files

Are the majority of the error-prone files

concentrated in a few DRSpaces?

RQ2: Answer

1) What portion of the bug space can the top 5 and top 10 largest

DRSpaces cover?

The top 5 and top 10 largest DRSpaces can cover the majority of

the files in each bug space.

Architecture Roots

A group of files burdening a project with

ever-increasing maintenance costs

By combining evolution and structure coupling,

can we get more insight into architectural

problems? Can this help us find not just the

locations of errors, but also the reasons for them?

An Architecture Root Aggregation cycles

Shared Secrets

Figure: Jboss JDBCCMRFieldBridge DRSpace

JDBCStoreManageraggregates command classes and cache class

4 command classes and cache class also aggregate JDBCStoreManager

An Architecture Root

Problematic inheritance

Figure: Hadoop FileSystem Inherit DRSpace

Unusual evolutionary coupling

New Insights

A significant portion of the DRSpaces led by an error prone file is

also error-prone

A few (~5) DRSpaces capture more than half of the most error-

prone files. These are the roots of ever-increasing maintenance

costs.

Error-prone DRSpaces often have architectural flaws

Localization

Architecture Issues

Consecutive Debts

Architecture Hotspots [Ran et al. 2015]

A Group of Files with Architecture Flaw and High Maintenance Costs

Primary relationship: Inherit

Leading Class

Child modules

Secondary relationship: Depend

Issue 1: Parent class depends on child class

+ Evolutionary coupling

,5 ,8 ,7 ,9

,6 ,7 ,9

,4 ,6,5,5

Issue 2: Unusual evolutionary coupling between parent class and child class

Issue 3: Modularity violation

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

1 config.DatabaseDescriptor (1) dp,44 ,14 ,10 ,10 ,6 ,14 ,36 ,118 ,12 , ,16 ,12 ,42 ,52 ,4 ,18 ,30

2 utils.FBUtilities dp,44 (2) ,40 ,4 ,6 ,10 ,6 ,12 ,38 ,28 ,12 ,8 ,14 ,24 ,46 ,6 ,18 ,28

3 utils.ByteBufferUtil ,14 dp,40 (3) , , , ,4 ,10 ,20 ,4 ,4 ,10 ,26 ,12 ,4

4 service.WriteResponseHandler ,10 dp,4 ,2 (4) ,4 ,6 ,18 dp,22 , ,6 , ,

5 locator.TokenMetadata ,10 ,6 ,4 (5) ,4 ,10 dp,24 , ,8 ,4 ,6 ,4 ,

6 locator.NetworkTopologyStrategy ,6 dp,10 ,2 ,6 dp,4 (6) ,10 ih,22 ,4 , , ,16 , ,8

7 service.DatacenterWriteResponseHandler dp,14 dp,6 ,2 ih,18 ,10 dp,10 (7) ,20 , , ,6 ,6 ,

8 locator.AbstractReplicationStrategy ,36 dp,12 ,4 dp,22 ag,24 ,22 dp,20 (8) ,6 ,16 ,10 , ,10

9 config.CFMetaData ,118 dp,38 dp,10 , , ,4 ,6 (9) ,16 ,36 ,46 ,56

10 dht.RandomPartitioner ,12 dp,28 dp,20 ,8 , , (10) dp,4 ,4 ,16 ,50

11 utils.GuidGenerator , dp,12 ,4 , , ,4 (11) ,4 ,

12 io.sstable.SSTable ,16 ,8 dp,4 ag,16 (12) ,4 dp,68 ,10 ,

13 utils.CLibrary ,12 dp,14 ,4 (13) ,12 ,

14 io.sstable.SSTableReader dp,42 ,24 dp,10 ,36 ,4 ih,68 dp,12 (14) ,22 ,4 , ,10

15 cli.CliClient ,52 dp,46 dp,26 ,6 ,4 ,16 ,6 ,16 ,46 ,16 ,4 ,10 , ,22 (15) ,6 ,14 ,48

16 locator.PropertyFileSnitch ,4 dp,6 , dp,6 , ,6 ,10 ,4 ,6 (16) ,4

17 dht.OrderPreservingPartitioner dp,18 dp,18 dp,12 ,4 , ,50 , , ,14 (17)

18 thrift.ThriftValidation dp,30 ,28 dp,4 , , ,8 , dp,10 dp,56 , ,10 ,48 ,4 (18)

Hotspot patterns (1): Unstable Interface

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Hotspot patterns (2): Modularity Violation

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

1 path1.Bean_java (1) , , , , , , , , , , , , , , , , , , , , , , , , , ,

2 path2.Pear_java , (2) , , , , , , , , , , , , , , , , , , , , , , , , ,

3 path3.FirstFruit_java , , (3) , , , , , , , , , , , , , , , , , , , , , , , ,

4 path4.SecondFruit_java Create, , , (4) , , , , , , , , , , , , , , , , , , , , , , ,

5 path4.ThirdFruit_java Create, , , , (5) , , , , , , , , , , , , , , , , , , , , , ,

6 path4.FourthFruit_java Create, , , , , (6) , , , , , , , , , , , , , , , , , , , , ,

7 path5.FifthFruit_java , , , , , , (7) , , , , , , , , , , , , , , , , , , , ,

8 path5.Pear_java , , , , , , , (8) , , , , , , , , , , , , , , , , , , ,

9 path5.FirstJuice_java , , , , , , , , (9) , , , , , , , , , , , , , , , , , ,

10 path5.SecondJuice_java , , , , , , , , , (10) , , , , , , , , , , , , , , , , ,

11 path5.SixthFruit_java , , , , , , , , , , (11) , , , , , , , , , , , , , , , ,

12 path5.SeventhFruit_java , , , , , , , , , , , (12) , , , , , , , , , , , , , , ,

13 path5.EighthFruit_java , , , , , , , , , , , , (13) , , , , , , , , , , , , , ,

14 path5.NinethFruit_java , , , , , , , , , , , , , (14) , , , , , , , , , , , , ,

15 path5.TenthFruit_java , , , , , , , , , , , , , , (15) , , , , , , , , , , , ,

16 path5.EleventhFruit_java , , , , , , , , , , , , , , , (16) , , , , , , , , , , ,

17 path5.TwelvethFruit_java , , , , , , , , , , , , , , , , (17) , , , , , , , , , ,

18path5.ThirteenthFruit_java , , , , , , , , , , Use, , , , , , , (18) , , , , , , , , ,

19path5.FourteenthFruit_java , , , , , , , , , , , , , , , , , , (19) , , , , , , , ,

20 path6.FirstFood_java , , , , , , , , , , , , , , , , , , , (20) , , , , , , ,

21 path7.SecondFood_java , , , , , , , , , , , , , , , , , , , Use, (21) , , , , , ,

22 path8.ThirdFood_java , , , , , , , , , , , , , , , , , , , Use, , (22) , , , , ,

23 path7.Apple_java , , , , , , , , , , , , , , , , , , , Use, , , (23) , , , ,

24 path9.FourthFood_java Create, , , , , , , , , , , , , , , , , , , Use, , , , (24) , , ,

25 path9.FifthFood_java Create, , , , , , , , , , , , , , , , , , , Use, , , , , (25) , ,

26 path10.FirstFig_java Cast,Use, , , , , , , , , , , , , , , , , , , , , , , , , (26) ,

27 path2.SecondFig_java Cast,Use, , , , , , , , , , , , , , , , , , , , , , , , , , (27)