icse2011 build maintenance

An Empirical Study of Build Maintenance Effort

Ahmed E. Hassan

BramAdams

YasutakaKamei

Thanh H.D.Nguyen

ShaneMcIntosh

Slides available online!

2

http://goo.gl/yXHHS

http://goo.gl/eICXC

http://goo.gl/yXHHS

http://goo.gl/yXHHS

http://goo.gl/yXHHS

http://goo.gl/yXHHS

Feedback

3

[email protected]

http://goo.gl/yXHHS

http://goo.gl/yXHHS

Feedback

3

[email protected]

Visit my posterduring the break!

http://goo.gl/yXHHS

http://goo.gl/yXHHS

The Build System Translates Sources into Deliverables

4Autotools

Make

Step 1 - Configuration

5

Features

Autotools

Step 1 - Configuration

5

Features

Tools

Autotools

Step 2 - Construction

6

Prescriptions

Make


6

=

Prescriptions

Make


6

=

.cPrescriptions

Make


6

=

.cPrescriptions

.o

Make


6

=

.cPrescriptions

Dependencies

.o

Make


6

=

.cPrescriptions

Dependencies

.o

.o.o.o

Make


6

=

.cPrescriptions

Dependencies

.o

.exe

.o.o.o

Make

Build code is complex...

7

Build Maintenance is a Nuisance for Developers

8


8Autotools

Make


8Autotools

Make

Migration to newer build technology

9

Build Bugs Even Affect the End User!

Address Bar

9


Address Bar Search Bar

9


Build Systems Require 12%of a Developer’s

Time (on average)

10

Kumfert, G., and Epperly, T.Software in the DOE: The

Hidden Overhead of the “Build”

Build Systems Require 12%of a Developer’s

Time (on average)

10

Kumfert, G., and Epperly, T.Software in the DOE: The

Hidden Overhead of the “Build”

Build maintenanceslows development!

11

Size Evolution Coupling People

Four Dimensional Study of Build Maintenance

.mk⇒?

.c

Empirical Study of 10 Systems (>25 MLOC)

PLplot12

Empirical Study of 10 Systems (>25 MLOC)

PLplot12

PDE Build, ANT,GNU Autotools, Make

Source, Test, or Build?

13

.mk.c.c.ac.dat.h

Source, Test, or Build?

13

.mk.cTest

.cBuildSource

.ac.dat.hBuildTestSource

14

Evolution Coupling People

How Large is a Typical Build System?

.mk⇒?

.c

Size

The Build Accounts for 9% of All Files (median)

15

0

0.09

0.18

0.27

0.36

16%

31%

12%

8%

3%5%

10%12%

1%

7%

% Build files

PLplot

16

Evolution Coupling People

How Large is a Typical Build System?

.mk⇒?

.c

Size

9% ofall files

17

Coupling People

How Does the Build Evolve?

.mk⇒?

.c

Size

9% ofall files

Evolution

b.c

a.c

File Churn Adjusted by System Size (# files)

18

Feb 1Jan 1

3 source files

x.c

b.c

a.c


18

Feb 1Jan 1

3 source files

Source churn2 ÷ 3 = 67%

x.c

b.c

a.c

.mk


18

Feb 1Jan 1

3 source files 2 Build specs


cfg

x.c

b.c

a.c

.mk


18

Feb 1Jan 1

3 source files 2 Build specs


Build churn1 ÷ 2 = 50%

cfg

x.c

Build File Churn is on Par with the Source Churn

0.00.2

0.40.6

0.81.0

ArgoUML Hibernate−core Eclipse−core Jazz GCC Git Linux Mozilla PLplot PostgreSQL

Projects

Normalize

d Churn

19

PLplot

0.00.2

0.40.6

0.81.0


Projects

Norm

alized

Churn

0.0

0.2

0.4

0.6

0.8

1.0

ArgoUML Hibernate Eclipse Jazz GCC Git Linux Mozilla PLplot PostgreSQL

Projects

Nor

mal

ized

Chu

rn SourceBuild

File

020

4060

80100

(%)

Build File Churn is on Par with the Source Churn

0.00.2

0.40.6

0.81.0


Projects

Normalize

d Churn

19

PLplot

0.00.2

0.40.6

0.81.0


Projects

Norm

alized

Churn

0.0

0.2

0.4

0.6

0.8

1.0

ArgoUML Hibernate Eclipse Jazz GCC Git Linux Mozilla PLplot PostgreSQL

Projects

Nor

mal

ized

Chu

rn SourceBuild

Build technologymigration

File

020

4060

80100

(%)

Line Churn Adjusted by System Size (# files)

20

x.c

b.c

a.c3 source files

Feb 1Jan 1


20

x.c

b.c

a.c3 source files

Feb 1Jan 1

65 4 5


20

x.c

b.c

a.c3 source files

Feb 1Jan 1

65 4

Source Lines per Change list (5+6+4+5) ÷ 4 = 5

5


20

x.c

b.c

a.c3 source files

Feb 1Jan 1

65 4

Source Lines per Change list (5+6+4+5) ÷ 4 = 5

Adjust for system size5 ÷ 3

= 1.67

5

Build Line Churn is Higher than the Source Churn

21

0

0.175

0.35

0.525

0.7

0.52

0.62

0.330.37

0.28

0.19

Ave

rage

Lin

es p

er C

hang

e Li

st (

norm

aliz

ed b

y sy

stem

siz

e)

SourceBuild

22

Coupling People

How Does the Build Evolve?

.mk⇒?

.c

Size

9% ofall files

Evolution

Build churn >> source

23

People

How Tightly Coupled are Build and Source Code?

.mk⇒?

.c

Size

9% ofall files

Evolution Coupling


Changes, Change Lists, and Work Items

24

.c .cChanges

.mk⇒?

.c


24

.c .cChanges

Change Lists Fix for bug #1234

.mk⇒?

.c


24

.c .c .mkChanges


.mk⇒?

.c


24

.c .c .mkChanges


Missed code for bug #1234

.mk⇒?

.c


24

.c .c .mkChanges


Missed code for bug #1234

Work itemsBug #1234

.mk⇒?

.c

foo.c

bar.c

Makefile

configure.ac

CL1 CL2 CL3 CL4 CL5

Bui

ldFi

les

Change Lists

Sour

ceFi

les

Logical Coupling Example

25

.mk⇒?

.c

foo.c

bar.c

Makefile

configure.ac

CL1 CL2 CL3 CL4 CL5

Bui

ldFi

les

Change Lists

Sour

ceFi

les

Logical Coupling Example

25

LC(Source ⇒ Build)

= 2 ÷ 4 = 50%

.mk⇒?

.c

Change List Coupling is Below 12% Estimate

26

4%

7%7%

3%

5%5%4%

7%

4%3%

5%

2%

Source ⇒ BuildTest ⇒ Build

.mk⇒?

.c

12%

Work Item Coupling is High

27

8%

20%

44%

4%

16%

27%


.mk⇒?

.c

12%


27

8%

20%

44%

4%

16%

27%


Mozilla’s build coupling is very high!

.mk⇒?

.c

12%


27

8%

20%

44%

4%

16%

27%


Mozilla’s build coupling is very high!

Very low due touse of higher level build abstraction

.mk⇒?

.c

12%

28

People

How Tightly Coupled are Build and Source Code?

.mk⇒?

.c

Size

9% ofall files

Evolution Coupling

Tightly coupled


29

How Do Projects Distribute Build Maintenance Work?

.mk⇒?

.c

Size

9% ofall files

Evolution Coupling

Tightly coupled

People


Identifying build maintainers

.c .c .c .ac.c

SourceBuild

.c .c


Change list #1

.c .c .c

Change list #3

Change list #4

.ac.c

Change list #2

SourceBuild

.c .c


Change list #1

.c .c .c

Change list #3

Change list #4

.ac.c

Change list #2

25% build

SourceBuild

.c .c


Change list #1

.c .c .c

Change list #3

Change list #4

.ac.c

Change list #2

0% build

25% build

SourceBuild

.c .c


Change list #1

.c .c .c

Change list #3

Change list #4

.ac.c

Change list #2

0% build 50% build

25% build

SourceBuild

.c .c


Change list #1

.c .c .c

Change list #3

Change list #4

.ac.c

Change list #2

0% build 50% build

SourceBuild

.c .c

≥

If you maintain source code, do you maintain build code?

31

25%22%

79%

% of developers

Dispersed Ownership:

Build maintenance isdistributed amongst most team members

32

Dispersed Ownership:

Build maintenance isdistributed amongst most team members

32

Concentrated Ownership:A small team is

responsible for mostof the build maintenance

33

How Do Projects Distribute Build Maintenance Work?

.mk⇒?

.c

Size

9% ofall files

Evolution Coupling

Tightly coupled

People

Dispersed or concentrated


Tool support needed for build maintenance!

34

Build churn >> source churn

Build maintenance required for 27% of source code tasks

Threats to ValidityStudiedprojects

Allprojects

35

Semi-automaticclassification

36

PLplot

37

PLplot

Size

9% ofall files

38

PLplot

Size

9% ofall files

Evolution


39

PLplot

.mk⇒?

.c

Size

9% ofall files

Evolution Coupling

Tightly coupled


40

PLplot

.mk⇒?

.c

Size

9% ofall files

Evolution Coupling

Tightly coupled

People



41

PLplot

.mk⇒?

.c

Size

9% ofall files

Evolution Coupling

Tightly coupled

People



41

PLplot

.mk⇒?

.c

Size

9% ofall files

Evolution Coupling

Tightly coupled

People


Questions?


icse2011 build maintenance

Documents