mining correlations of atl transformation and metamodel metrics

Dipartimento di Ingegneria e Scienze

Università degli Studi dell’Aquila

dell’Informazione e Matematica

Mining Correlations of ATL Transformation and Metamodel Metrics

Juri Di RoccoDavide Di RuscioLudovico IovinoAlfonso Pierantonio

7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio

3IntroductionOver the last decades many MDE technologies have been conceived to support a wide range of modeling and model management activities


4IntroductionAvailability of powerful languages and tools for developing, testing, and chaining model transformations

Limited support for analysing and understanding common characteristics of model transformations

• what are the main constructs typically used when developing transformations ?

• to what extent is the development of model transformations affected by the complexity of the corresponding metamodels ?


5IntroductionAvailability of powerful languages and tools for developing, testing, and chaining model transformations

Limited support for analysing and understanding common characteristics of model transformations

• what are the main constructs typically used when developing transformations ?

• to what extent is the development of model transformations affected by the complexity of the corresponding metamodels ?

MMs MMtT

?


6IntroductionSeveral metrics are available to measure ATL transformations

Quality attributes have been also defined and they have been aligned to a set of metrics

None of the existing approaches deal with transformation metrics correlation

Correlating transformation and metamodels metrics is also unexplored


7Contribution

Correlation of several metrics

• 28 metamodel metrics

• 35 transformation metrics

It has been applied on a corpus of

• 91 ATL transformations

• 72 corresponding metamodels

Preparatory study to estimate the required effort to develop model transformations depending on the structural characteristics of the input and target metamodels

Analysis process for understanding model transformations characteristics


8Proposed analysis process


9

Metrics calculation


10Metrics calculationConsists of the application of metrics on a data set of metamodels and transformations

Sample metamodel metrics

Sample transformation metrics

Davide Di Ruscio

11Metrics calculationThe metrics calculation has been implemented by exploiting a model-driven toolchain

Artifact 1

Artifact 2

Artifact n

MetricsCalculator

MetricsCSV generator

Davide Di Ruscio


Artifact 1

Artifact 2

Artifact n

MetricsCalculator


The Metrics Calculator is able to calculate for each artifact all

the considered metrics

Davide Di Ruscio


Artifact 1

Artifact 2

Artifact n

MetricsCalculator


The Metrics Calculator is an ATL transformation whose

target models conform to the Metrics metamodel

Davide Di Ruscio


Artifact 1

Artifact 2

Artifact n

MetricsCalculator


Generating CVS files enables the adoption of statistical

tools like IBM SPSS, Microsoft Excel, and Libreoffice Calc for

subsequent analysis of the generated data


15

Calculation, selection, and statistical

significance of metrics correlation


16Calculation of metrics correlations

Correlation is used to detect cross-links and assess relationships among observed data

Pearson’s and Spearman’s coefficients to measure the correlations among calculated metamodel and transformation metrics

Pearson’s correlations Spearman’s correlations


17Calculation of metrics correlations

Both Pearson’s and Spearman’s correlation indexes assume values in the range of -1.00 (perfect negative correlation) and +1.00 (perfect positive correlation)

A correlation with value 0 indicates that between two variables there is no correlation


18Selection of metrics correlations

Pearson’s correlation indexes for all the values of the ATL transformation metrics

Spearman’s correlation indexes for all the values of the ATL transformation and metamodel metrics



ATL transformation metrics correlations

All the values greater than 0.8 have been highlighted to select the metrics that are most related according to the Pearson’s index.





Number of output patterns (OP) <->

number of bindings (B)





Number of Transformation Rules (TR) <->

Number of Rules with a Filter Condition on the Input Pattern (RWF)





Number of Helper (H) <->

Number of Helper with Context (HWC)



ATL transformation and metamodel metrics correlations

All the values greater than 0.6 have been highlighted to select the metrics that are most related according to the Spearman’s index.



ATL transformation and metamodel metrics correlations

All the values greater than 0.6 have been highlighted to select the metrics that are most related according to the Spearman’s index.Number of structural features in the output metamodel (SF - OUTPUT)

<-> Number of binding (B)


25Statistical significance of metric correlations

Just because two variables are related, it does not necessarily mean that one directly causes the other.

It is necessary to assess that the performed analysis is statistically

significant or not


26Statistical significance of metric correlations

Significance level of a statistical hypothesis refers to the probability that the random sample that has been chosen is not representative

• the lower the significance level, the more confident one can be in replicating the performed results

T-test has been used to establish if the identified correlation coefficients were statistically significant

• the threshold 0.05 has been considered

• correlations which induce a T-test value above the threshold have been rejected


27Data analysisThe aim is to discuss and interpret the most relevant correlations between structural characteristics which have been found in the previous stages


28Data analysisHow transformation rules are influenced by target metamodels

Number of output metaclasses (OUT MC)

<-> Number of Transformation Rules

(TR)

Transformation development is typically output driven (developer tries to cover all the metaclasses

in the target metamodel)

Spearman’s index: 0.746Significance value: 0.0002


29Data analysisHow the total number of transformation input patterns are influenced by the source metamodels

Number input metaclasses (IN MC)

<-> Number of Input Patterns

(IP)

Even though not evident like in the previous case, IN MC and IP seem

to increase together and this might be related to the “coverage”

characteristic of the transformations in the considered

corpus



30Data analysisHow the structural features in the target metamodel influence the number of bindings

Number of output structural features

(OUT SF) <->

Number of Bindings (B)

Both OUT SF and B seem to increase together and this might

be related to the “coverage” characteristic of the

transformations in the considered corpus



31Data analysisHow general purpose and domain specific metamodels affect the complexity of model transformations



Number of Rules with a Filter Condition on the Input Pattern

In GPL2GPL and GPL2DSL transformations only parts of metamodels and hierarchies are considered



Number of Rules with a Do Section

In GPL2GPL and DSL2GPL the use of the imperative “do” block is higher than the other cases

Typically imperative constructs are used when the input and output metamodels are completely different



Number of Rules with a Using clause

The use of the using clause is very limited and it seems to be one of the less used constructs of ATL



Number of calls to resolveTemp

It seems to be never used in GPL2DSL transformations and equally distributed in the other cases

It results to be one of the most complex construct of the ATL language


36Data analysisHow developers use the ATL language

Transformations in the considered corpus are mainly developed in a declarative way: • Most of the transformations are developed by means of matched

rules• Helper with contexts are more than those without contexts, which

are usually used as variables in transformations described in an imperative way


37ConclusionsAn approach to analyze model transformations by considering also the corresponding metamodels has been discussed

The main goal is to better understand the characteristics of model transformations and how their complexity is related to the complexity of metamodels

A correlation analysis has been performed to identify the most cross-linked metrics


38Future workExtend the corpus of artifacts in order to:

• validate the identified correlations

• better investigate the significance value for those correlations that currently are below the threshold

Include in the analysis further kinds of artifacts typically involved in any MDE approach

Rely on the results achieved so far in order to define an approach supporting the early cost estimation for developing model transformations


39Future workExtend the corpus of artifacts in order to:

• validate the identified correlations

• better investigate the significance value for those correlations that currently are below the threshold

Include in the analysis further kinds of artifacts typically involved in any MDE approach

Rely on the results achieved so far in order to define an approach supporting the early cost estimation for developing model transformations

Item for the panel discussion ?

?Thank you

mining correlations of atl transformation and metamodel metrics

Engineering

metamodels metrics

metrics calculation

set of metrics

application of metrics

software engineering

international workshop

model management activities

wide range of modeling