effecitve uml modeling quality asurance and its economics

Leiden University. The university to discover.

Dr. Michel R.V. Chaudron, Leiden Institute of Advanced Computer ScienceProgram director M.Sc. ICT & Business

Effective UML Modelling Does Software Modeling Pay ?

- empirical studies in UML -

http:/iib.liacs.nl

Leiden University. The university to discover.Michel Chaudron, Jeudi des Modeles, 31 March 2011, Nantes

Collaborate with us!

Your ideas on software engineering research?

Collaborate with us! [email protected]

- Can we study your project (for free)?

- We would like to study : - impact on maintenance, requirements

- Communication in off-shored projects

- UML-based test generation

- Compare UML-based with MDA-based development

- Measure risk exposure in iterative sw development

mailto:[email protected]


Outline of talk

- Introduction

- Modeling – State of the Practice

- Styles of modeling

- Quality of models

- Costs of modeling

- Benefits of modeling

Johan Muskens

Christian Lange

Ariadi Nugroho

Werner Heijstek


Introduction Leiden Netherlands

Eindhoven

Amsterdam

Leiden


Research Interest- Effectiveness of software modelling

- Quality of ModelingWhen is a model good (enough)

- Analysis of Non-Functional Properties of component-basedarchitectures (model-based)

- What are the pay-offs of investing inearly design/architecture?

Fewer defects? Cheaper maintenance? …

- Software Design


State of the Practice


Styles of ModelingSketch

Communication

Recipe for construction


Characteristics of Case Studies

Your project here?


Industrial use of UML diagrams

Class

How

UML i

s used

B Dob

ing, J

Parso

ns -C

ommu

nicati

ons o

f the A

CM, 2

006

http://www.csa.com/partners/viewrecord.php?requester=gs&collection=TRD&recid=20061190153974CI


Typical Word document mixes:- Text and diagrams

500-800 words / diagram

- Formal UML & inventions40-50% of diagrams is UML60-50% of diagrams is not UML

Text is used for:- Design decisions / Rationale- Elaboration/explanation- Linking to context

‘Design’ Tooling must support: models, sketches & textUri Dekel, James D. Herbsleb: Notation and representation in collaborative

object-oriented design: an observational study. OOPSLA 2007: 261-280

Statistics about Design Doc’s in general

http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/h/Herbsleb:James_D=.html

http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/h/Herbsleb:James_D=.html

http://www.informatik.uni-trier.de/~ley/db/conf/oopsla/oopsla2007.html


Panel

Modelling style and model purpose

understanding

communicating

managing complexity

implementation blueprint

Styles of using UML– as a sketch – thinking tool/understanding

– for communicating system design

– as a blueprint – guide the implementation work

– as a implementation (MDA)

C.F.J.L

ange,

M.R.

V. C

haud

ron, J.

Muske

ns: In

Pract

ice: U

ML So

ftware

Ar

chitec

ture a

nd D

esign

Desc

riptio

n. IEE

E Soft

ware

23(2):

40-46

(2006

)

http://www.informatik.uni-trier.de/~ley/db/journals/software/software23.html


UML in practiceSketch

Communication


More effort More expensive


Suitability of UML

Sketch

Communication


UML Blueprint: - Ok

UML for code generation:- Not formal enough

UML for Design: - UML is too formal- Informal information is added to diagrams

- designers do use UMLconcepts

UML for Communication: - Internationally standardized

meaning seems to work


Quality of UML Models

- Can we distinguish styles by means of some objective measure of model-quality?


Quality ModelsExisting models

- Decomposition of characteristics- Bottom level: metrics

- Differences in - Relations between characteristics

- Vocabulary Boehm’s Quality Model

Boehm McCall

ISO 9126 Rombach

…


How to Define UML Model Quality?The quality attributes of a UML model is determined by its purpose...

Frame

work

for qu

ality o

f UML

mod

els (L

ange

& Ch

audro

n, 200

5)


How to measure goodness of UML model?- Design Heuristics

- Completeness

- Interaction of classes must be described in SD

- Methods of CD must be called in SD

- Consistency

- Messages in SD must correspond to method in class diagram

- Correspondence

- Dependencies in the implementation are allowed only if they occur in the design

In many cases it can not be automatically determined

whether a flaw is an incompleteness or an inconsistency


Automated Model Checking and Visualization

UML model

Visualization of model + metrics

Quality Metrics/Rules• Completeness• Consistency

Analysis Tool


Screenshot of the MetricView tool

= violation

= ok

Quality = absence of ‘bad things’such as omissions, spaghetti


Design Heuristics for UML

– Metrics based on Chidamber & Kemerer OO metricsMaintainability relates to Coupling, Cohesion, …

Highest coupling = 7 Highest coupling = 4


Distribution of Coupling

21

0

50

100

150

200

250

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

coupling

num

ber

of

cla

sses

System of several hundred classes

Leiden University. The university to discover.Michel Chaudron, Jeudi des Modeles, 31 March 2011, NantesChristian Lange 22

MetaViewto visualize the (hidden) inter-diagram relationsExample: Which classes contribute to a certain use case?

Supporting task-oriented modeling using interactive UML views, C.F.J. Lange, M. A.M. Wijns, and M. R.V. Chaudron,Journal of Visual Languages and Computing

MetricView Tool

http://www.youtube.com/watch?v=G3HJ_QR9EG4

Leiden University. The university to discover.Christian Lange. 23

MetricViewThe values of metrics are visualized on class diagrams using colors Example: Coupling-Between-Objects (CBO)



MetricView Tool

Leiden University. The university to discover.Christian Lange. 24

3D visualization of design measurements




Industrial Experience with MetricViewas Quality Assurance Tool

- Based on 15+ industrial projects

- If there are weak spots in the design, then these are indicated as ‘suspects’ by MetricView

- MetricView indicates many weak spots that do not require improvement according to project architects (false positives?) quality is not applied uniformly across UML models

- The later MetricView is applied, the fewer ‘weak spots’ are removed from the design process issue


How do developers apply detail to UML models?

Developers apply more detail on criticaland complex parts of a system

61%

52%

37%

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

Equal details for all parts

Details proportionalto complexity

More details forcritical parts

Programmersdetermine details

Percen

tag

e o

f th

e r

esp

on

den

ts

Disagree

Somewhat Disagree

Neutral

Somewhat agree

Agree

Ariad

i Nug

roho,

Miche

l R. V

. Cha

udron

: A Su

rvey o

f the P

ractic

e of D

esign

-Co

de Co

rrespo

nden

ce am

ongst

Profe

ssion

al So

ftware

Engin

eers.

ESEM

2007:

467

-469


How is Level of Detail distributed in a diagram?

Case 1 Case 2

Robin van den Broek B.Sc. Thesis in CS, 2009, Leiden University

Low detail Low detail

High detail High detail


M.Sc. Thesis, Dennis van Opzeeland TU/e Computer Science, System Architecture and Networking

Does the implementation resemble the design?

design implementation


Model - Source Code Conformance

Outlier

X-Axis

Metrics of Design

Y-Axis

Metrics of Implementation

Points represent Classes

Points off the diagonal indicate outliers

Does the implementation look like the original design?


Does strictness of implementing a model differ for different modeling constructs?

Developers favor to implement dependency and inheritance relations more strictly than other constructs

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

40.0

45.0

50.0

Packagestructure

Dependencyrelations

Inheritancerelations

Class andmethod names

The order ofmethod calls

Pe

rce

nta

ge o

f th

e r

esp

on

de

nts

Loosely

Somewhat loosely

Neutral

Somewhat strict

Strictly

31%

46%

36%

Ariad

i Nug

roho,

Miche

l R. V

. Cha

udron

: A Su

rvey o

f the P

ractic

e of D

esign

-Co

de Co

rrespo

nden

ce am

ongst

Profe

ssion

al So

ftware

Engin

eers.

ESEM

2007:

467

-469


Which factor most often drives developers to non-correspondence?

Among other factors, developers considered incomplete modelmore often drives them to non-correspondence

29%

48%

31%

0

10

20

30

40

50

60

Meeting deadline

Impracticalmodel

Incompletemodel

Model doesn’t satisfyrequirements

Perc

enta

ge o

f th

e r

espondents

NeverSometimes

OftenVery Often

Ariad

i Nug

roho,

Miche

l R. V

. Cha

udron

: A Su

rvey o

f the P

ractic

e of D

esign

-Co

de Co

rrespo

nden

ce am

ongst

Profe

ssion

al So

ftware

Engin

eers.

ESEM

2007:

467

-469


Findings from State of the Practice

- UML is used in many different ways

- Incompleteness

- Not all parts of system are modelled

- Disproportion

- Different parts of the system are modeled different in level

of quality

- Inconsistency

- Conflicting information in different views of one model

Is this a problem?


- Subjects are given UML models- Ask questions about UML model

- Two types• Which implementation matches the diagrams?

• How do you interpret these diagrams?

- Answer from the perspective of the developer.

- Multiple-choice test (15 Questions)

- 4 options + 1 option (“There’s something wrong, I can’t give an answer”)

- + Background questions (possible confounding factors: training, experience, …)

Does Quality Matter? Experiment

Effect

s of D

efects

in U

ML M

odels

, C.F.

J. Lan

ge, M

.R.V.

Chaud

ron, (I

CSE 2

006)


Example Question

- Suppose you are developer in this banking software project. It is your task to implement class

ATM. Please indicate how you would implement the class ATM given these two UML diagrams?

A)

getCardInserted(){c.requestPIN();dosomething;a.open()}

B)

getCardInserted(){c.requestPIN();dosomething;a.lock()}

C)

getCardInserted()

{c.requestPIN();dosomething;a.acknowledge()}

D)

getCardInserted()

{c.requestPIN();dosomething;a.validate()}

E)

Ring the bell!

Something

is

wrong!

?

Many don’t see the defect

and just implement

Inconsistency:

Message Name does not correspond to Method

0

10

20

30

40

50

60

70

80

a b c d e(rror)

Message does not correspond to Method - Control (Q4)

0

20

40

60

80

100

120

a b c d e(rror)

Effect

s of D

efects

in U

ML M

odels

, C.F.

J. Lan

ge, M

.R.V.

Chaud

ron, (I

CSE 2

006)

64% don’t see the defect

and just implement


Does Quality Matter? Case Study

operations

attributes

associations

Parts of the design are more detailed than others


Paym ent

-amount

-cashTendered

Cash

+authorized()

-name

-bankID

Check

+authorized()

-number

-type

-expDate

Credit

+calcTax()

+calcTotal()+calcTotalWeight()

-date

-status

Order

+calcSubTotal()

+calcWeight()

-quantity

-taxStatus

OrderDetail

1

1

1* Preprocessor Parser Analyser Inheritance Relator Statistic Filter Statistic CalculatorDB Creator DB Filler DB Checker

SAAT.BAT

Select ‘defects’ in defect DB

Find classes in source code that were

repaired for solving this defects

Find corresponding classes in UML models

Determine LoDfor CD and SD

Payment

-amount

-cashTendered

Cash

+authorized()

-name

-bankID

Check

+authorized()

-number

-type

-expDate

Credit

+calcTax()

+calcTotal()+calcTotalWeight()

-date

-status

Order

+calcSubTotal()

+calcWeight()

-quantity

-taxStatus

OrderDetail

1

1

1* Preprocessor Parser Analyser Inheritance Relator Statistic Filter Statistic CalculatorDB Creator DB Filler DB Checker

SAAT.BAT

component C2

requires I2

requires I3

uses I2.g

uses I3.h

component C2

requires I2

requires I3

uses I2.g

uses I3.h

defectdensity

LoD per class

Relation between UML-LoD and Code Quality


•Does high LoD in sequence diagrams correspond to lower defect density?

Level of Detail for Sequence Diagrams is significantly (negatively) correlated with defect density.

More detailed model => fewer defects

Relation between Level of Detailand Defect Density

= 1 class


Economic Model for Cost of QualityFrom: H. Krasner, Cost of Quality, 1998

Cost of SQ = Achievement Cost + Non-conformance Cost


Cost of Modeling

Focus on effort

“Is there a correlation between class-count and the effort spent in modeling?”

Sources:‐ Experiment‐ Set of industrial case studies


Empirical Data

Experiment

106 MSc students in (TU/e) Organized in 35 teams

Task: Model Car Navigation System

Duration: 6 weeks assignment

Three treatments were applied:1. No modeling conventions

2. With modeling conventions

3. Tool-supported modeling conventions


Results Experiment

Effort (hours)

Class-count

Sig. (2-tailed)

.302

.171

There is no significant correlation between class-count and the effort spent in modeling.

Spearman’s Correlation Coefficient

0,00

5,00

10,00

15,00

20,00

25,00

30,00

35,00

40,00

0 10 20 30 40 50 60 70

Class-count

Eff

ort

(h

ou

rs)

class-count is not a good measure for the effort spent on modeling?


Measurement of Modeling/Design in RUP Effort Distribution

designingmodeling


incep.

elaboration

construction

transition

constr.

transitionelaboration

‘traditional’ (avg. of 17 projects)

Model-driven (1 project)

Impact of MDD on Effort Distributionas % of total project effort

architecture & code generator


Cost of Modeling - Summary- Size is not the issue- Large variety in cost per

- Developer (expertise)

- project- Purpose/use of models

- Level of abstraction / target quality of models

- Maintaining model-code correspondence

- Common to all new technology:

- training, licensing, …


The Use of UML and Software Quality

A large number of developers indicated the use of UML improves understandability and modularity

45%41%

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

40.0

45.0

50.0

CoverRequirements

Correctness Modularity Testability Understandability

Percen

tag

e o

f th

e r

esp

on

den

ts

Reduce

Somewhat Reduce

Neutral

Somewhat Improve

Improve

Ariad

i Nug

roho,

Miche

l R. V

. Cha

udron

: A Su

rvey o

f the P

ractic

e of D

esign

-Co

de Co

rrespo

nden

ce am

ongst

Profe

ssion

al So

ftware

Engin

eers.

ESEM

2007:

467

-469


Benefits of Modeling

Modeling

Req. engineering

Designing

Implementing

Testing

Maintenance

Modularity

Defects

Doing the right things

Project ManagementEarly insight in system quality

Better traceability

Improved shared understanding


Do Models improve understanding?

Eye tracking experiment


Eye tracking UML designs: gazeplots

structure behaviour

Attention is determined by task


Heatmap Person A

structure behaviour


Heatmap Person B

There are large differences in styles of reading


Improved

Quality

Modeling

Fewer

Defects

Reduced

maintenance

effort

Better Understanding

Improved Project

Management

Reduced

testing

effort

Improved

Communication

Reduced

rework

More Efficient

Maintenance

“validation”

building the

right system

“verification”

building the

system right

Improved

requirements

Better Risk

management

Improved

Productivity

More

Accurate

Estimating

Improved

Modularity

Overview Benefits of Modeling


What is next?

- Domain specific modelling (using UML)

- Domain specific tooling, V&V, generation

- Generation of/from UML

- From requirements

- Test generation based on UML

- Tool integration

- Synchronization between model and code


Concluding Remarks- UML modeling is being done in a variety of styles- Modelers ‘economize’ by focusing on complex and critical

parts. QA needs to be risk-aware

- Low hanging fruit: ‘common’ QA techniques should be applied to models (versioning, reviews, coding-conventions)

- There is a payoff of modeling in: - improved modularity- seq.diagrams code quality- communication (suspected)

- Future work: maintaining model-code correspondence

Please send your projects to: [email protected]


What is next?

Your ideas on research?

Collaborate with us! [email protected]

- Can we study your project?

- We would like to study : - impact on maintenance, requirements

- Communication in off-shored projects

- UML-based test generation

- Compare UML-based with MDA-based development

- Measure risk exposure in iterative sw development

mailto:[email protected]


How often do design issues lead to problems?

0

10

20

30

40

50

60

70

Inconsistency Understandability Inaccuracy Incompleteness

Pe

rce

nta

ge

of

th

e r

esp

on

de

nts

Never

Sometimes

Often

Very Often

Incompleteness of models was considered most often to lead to implementation problems


Panel

Opinions on Value of Modelling

ValueAgile

Formal Methods

“LooseUML”

completeness

100%

MDA


Which classes are modeled?

Complexity and Coupling is higher for classes that are modeled than not modeled


Recommendations- Apply Quality Assurance to UML modeling

- Use ‘coding conventions’ for UML:

- Naming – correspond to implementation

- Layout

- Use of patterns/stereotypes

- Version Control

- Organize traceability from Use Cases to Sequence

diagrams (tool/conventions)

- Quality of SD’s is important when UML is used as blueprint for implementation


Maintenance paradox

- Maintainers of software would like better documentation

- Even if UML models are created during design, these are used in maintenance.The more detailed models are, the mode quickly they become outdated.

Better abstraction in reverse- / round trip-engineering is needed in the setting of incremental & iterative development


Conformance

- Between UML model and source code implementation - It is the degree of similarity between model and implementation

- Reasons for lack of conformance- Poor model quality (therefore misunderstood in implementation)- “I know it better”-Attitude of Programmer- Evolution of implementation during maintenance

- Why is conformance important?- Ensure similarity between model and implementation- Model is used for understanding the implementation in maintenance (understanding

the corresponding model is easier than digging through thousands of lines of source code)

- We developed methods to - Measure conformance / Identify Deviations- (Re)construct a mapping between model and implementation elements- Support in keeping Model and Implementation ‘in sync’

MRV ChaudronSheet 60


MetricView contributors

- Johan Muskens- Christian Lange- Martijn Wijns- Dennis van Opzeeland- Ariadi Nugroho- Werner Heijstek- Maurice Vermeer- Alex Telea- Robin van den Broek



Publications- Journal Publications

- Supporting task-oriented modeling using interactive UML views. PDFC.F.J. Lange, M.A.M. Wijns, M.R.V. Chaudron. Journal of Visual Languages and Computing (Elsevier), Volume 18, Issue 7, August 2007.

- In Practice: UML Software Architecture and Design Description.C.F.J. Lange, M.R.V. Chaudron, J. Muskens. IEEE Software, Volume 23, Issue 2, March 2006.

- Conference Publications

- A Visualization Framework for Task-Oriented Modeling using UML.C.F.J. Lange, M.A.M. Wijns, M.R.V. Chaudron. Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07). January 2007.

- An Experimental Investigation of UML Modeling Conventions.C.F.J. Lange, Bart DuBois, M.R.V. Chaudron, Serge Demeyer. Proceedings of the ACM/IEEE International Conference on Model-driven Engineering Languages and Systems (MoDELS 2006). October 2006.

- Effects of Defects in UML Models - An Experimental Investigation.C.F.J. Lange, M.R.V. Chaudron. Proceedings of the International Conference on Software Engineering (ICSE 2006). May 2006. (acceptance rate: 9%)

- Improving the Quality of UML Models in Practice.C.F.J. Lange. Proceedings of the International Conference on Software Engineering (ICSE 2006). May 2006.

- Visual Exploration of Combined Architectural and Metric Information.M. Termeer, C.F.J. Lange, A. Telea, M.R.V. Chaudron. Proceedings of VISSOFT 2005, IEEE CS Press. September 2005.

- Combining Metrics Data and the Structure of UML Models using GIS Visualisation Approaches.C.F.J. Lange, M.R.V. Chaudron. IEEE International Conference on Information Technology 2005. Las Vegas, April 2005.

- An Exploratory Study on the Industrial Use of UML: Improving Control over Design Quality.C.F.J. Lange, M.R.V. Chaudron. JACQUARD Conference. February 2005.

- Investigations in Applying Metrics to Multi-View Architecture Models.J. Muskens, M.R.V. Chaudron, C.F.J. Lange. In Proceedings of EUROMICRO’04, Rennes, France, September 2004.

- An Empirical Assessment of Completeness in UML Design.C.F.J. Lange, M.R.V. Chaudron. Proceedings of the 8th Conference on Empirical Assessment in Software Engineering (EASE04). May 2004.

- Konsistenz und Vollständigkeit industrieller UML Modelle.C.F.J. Lange, M.R.V. Chaudron. Proceedings of “Modellierung 2004“, Marburg, Germany, 24-26 March 2004. 62Michel Chaudron, Christian Lange, Martijn Wijns

http://www.langomat.de/research/papers/LANGE_MVEexperiment_JVLC.pdf


- Workshop Publications

- Towards Task-Oriented Modeling using UML.C.F.J. Lange, M.A.M. Wijns, M.R.V. Chaudron. Proceedings of the 10th QAOOSE workshop, co-located with ECOOP2006. July 2006.

- Managing Model Quality in UML-based Software development.C.F.J. Lange, M.R.V. Chaudron. Proceedings of IEEE Conference on Software Technology and Engineering Practice 2005 (STEP), in the "Workshop on Software Process Improvement, Quality Assurance and Measurement. September 2005.

- Quantitative Techniques for the Assessment of Correspondence between UML Designs and Implementations.Dennis J.A. van Opzeeland, Christian F.J. Lange, Michel R.V. Chaudron. Proceedings of the 9th QAOOSE workshop, co-located with ECOOP2005. July 2005.

- An Empirical Investigation in Quantifying Inconsistency and Incompleteness of UML Designs.C. Lange, M.R.V. Chaudron, J. Muskens, L.J. Somers, H.M. Dortmans. 2nd workshop on consistency problems in UML-based software development, October 20, 2003, in San Francisco, USA as part of the “International Conference on Unified Modeling Language 2003”.



- Journal PapersAriadi Nugroho, "Level of detail in UML models and its impact on model comprehension: A controlled experiment", Information and Software Technology Journal (in Press).

-Conference PapersAriadi Nugroho and Michel R.V. Chaudron, "Evaluating the Impact of UML Modeling on Software Quality: An Industrial Case Study", ACM/IEEE 12th International Conference on Model Driven Engineering Languages and Systems (MODELS), Denver, Colorado - USA, 2009.

- Ariadi Nugroho, Bas Flaton, and Michel R.V. Chaudron, "An Empirical Analysis of Level of Detail in UML Models and its Relation with Defect Density", ACM/IEEE 11th International Conference on Model Driven Engineering Languages and Systems (MODELS), Toulouse - France, 2008. (received best paper awards). PDF

- Ariadi Nugroho and Michel R.V. Chaudron, "A survey into the Rigor of UML Use and its Perceived Impact on Quality and Productivity", 2nd International Symposium of Empirical Software Engineering and Measurement (ESEM), Kaiserslautern -Germany, 2008. PDF

- Ariadi Nugroho and Michel R.V. Chaudron, "A Survey of the Practice of Design - Code Correspondence amongst Professional Software Engineers", 1st International Symposium on Empirical Software Engineering & Measurement (ESEM), Madrid, 2007. PDF

- Workshop PapersAriadi Nugroho and Christian F.J. Lange, "On the Relation between Class-count and Modeling Effort", Model Size Metrics Workshop, co-located with MODELS Conference, Nashville, Tennessee, 2007. (best paper). PDF

- Chapters in BooksAriadi Nugroho and Michel R.V. Chaudron, “Managing the Quality of UML Models in Practice”, in Model-Driven Software Development: Integrating Quality Assurance, Idea Group Inc., New York, 2008. More info


http://www.irit.fr/models/awards.html

http://dx.doi.org/10.1007/978-3-540-87875-9_42

http://doi.acm.org/10.1145/1414004.1414020

http://doi.ieeecomputersociety.org/10.1109/ESEM.2007.12

http://dx.doi.org/10.1007/978-3-540-69073-3_11

http://www.liacs.nl/~anugroho/MDSDIQA.pdf


- W. Heijstek and Michel R. V. Chaudron (2010) The Impact of Model Driven Development on the Software Architecture Process, 36th Euromicro Conference on Software Engineering and Advanced Applications (SEAA 2010) Lille, Francebibtex

W. Heijstek, Michel R. V. Chaudron, Libing Qiu and Christian C. Schouten (2010) A Comparison of Industrial Process Descriptions for Global Custom Software Development, 5th International Conference on Global Software Engineering (ICGSE 2010) Princeton, New Jersey, USA - Pages 277-284bibtex

W. Heijstek and Michel R. V. Chaudron (2009) Empirical Investigations of Model Size, Complexity and Effort in Large Scale, Distributed Model Driven Development Processes - A Case Study, 35th Euromicro Conference on Software Engineering and Advanced Applications (SEAA 2009) Patras, Greecebibtex

W. Heijstek and Michel R. V. Chaudron (2008) Evaluating RUP Software Development Processes Through Visualization of Effort Distribution, 34th Euromicro Conference on Software Engineering and Advanced Applications (SEAA 2008) Parma, Italybibtex

W. Heijstek and Michel R. V. Chaudron (2008) Exploring Effort Distribution in RUP Projects 2nd International Symposium on Software Engineering and Measurement (ESEM 2008) Kaiserslautern, Germanybibtex

W. Heijstek and Michel R. V. Chaudron (2007) Effort distribution in model-based development. 2nd Workshop on Model Size Metrics (MSM 2007)bibtex


http://www.liacs.nl/~chaudron/

http://seaa2010.liacs.nl/

http://www.liacs.nl/~heijstek/bibtex/heijstek_2010.bib




http://www.inf.pucrs.br/icgse/

http://www.liacs.nl/~heijstek/bibtex/heijstek_2010a.bib




http://seaa2009.vtt.fi/





http://seaa2008.isti.cnr.it/





http://www.esem-conferences.org/2008/index.php





http://www.win.tue.nl/~clange/MSM2007/
