mining metrics for understanding metamodel characteristics

Dipartimento di Ingegneria e Scienze

Università degli Studi dell’Aquiladell’Informazione e Matematica

Mining Metrics for Understanding Metamodel Characteristics

Alfonso Pierantonio Juri Di RoccoDavide Di RuscioLudovico Iovino

Alfonso Pierantonio

2OutlineIntroduction MotivationMeasuring metamodelsCalculation of metrics correlationSelection of metrics correlationData AnalisysConclusionsFuture works

Alfonso Pierantonio

3IntroductionMetamodels are a key concept in Model-Driven Engineering: they represent the «trait-d'union» among all constituent components Metamodels formally define the modeling primitives used in modeling activities

Alfonso Pierantonio

4MotivationIt is of crucial relevance to investigate common characteristics of metamodels, in order to have a better understanding on– how they evolve over time– what is the impact of metamodel changes throughout the

modeling ecosystem

Metamodel

Alfonso Pierantonio

5MotivationIt is of crucial relevance to investigate common characteristics of metamodels, in order to have a better understanding on– how they evolve over time– what is the impact of metamodel changes

throughout the modeling ecosystem

Analyse metamodels in order to evalutate their structural characteristics and the impact they might have during the whole metamodel life-

cycle especially in case of metamodel evolutions

Analysing metamodel characteristics by investigating

the correlations of different metrics applied on a corpus of more than

450 metamodels

Alfonso Pierantonio

7Measuring MetamodelsThe applied process is able to identify linked structural characteristics Understand how they might change depending on the nature of metamodels

Alfonso Pierantonio

8Metrics calculationConsists of the application of metrics on a data set of metamodelsThe applied metrics are borrowed from [1] and we added new ones by leading to a set of 28 metricsAn excerpt of the considered metrics have been considered in rest of the presentation

[1] W. James, Z. Athansios, M. Nicholas, R. Louis, K. Dimitios, P. Richard, and P. Fiona. What do metamodels really look like? Frontiers of Computer Science, 2013.

Alfonso Pierantonio

9Metrics calculationThe corpus of the analyzed metamodels has been obtained by retrieving metamodels from different repositories, i.e., EMFText Zoo, ATLZoo, Github, GoogleCodeFor a total number of 466 metamodels belonging to different technical spaces and domains

Alfonso Pierantonio

10Metrics CalculationThe metrics calculation has been implemented by exploiting a model-driven toolchain

Alfonso Pierantonio

11Metrics CalculationThe metrics calculation has been implemented by exploiting a model-driven toolchainThe Metamodel Metrics

Calculator is able to calculate for each metamodel all the

considered metrics

Alfonso Pierantonio

12Metrics CalculationThe metrics calculation has been implemented by exploiting a model-driven toolchain

The Metamodel Metrics Calculator is an ATL

transformation whose target models conform to the Metrics

metamodel

Alfonso Pierantonio

13Metrics CalculationThe metrics calculation has been implemented by exploiting a model-driven toolchain

Generating CVS files enables the adoption of statistical

tools like IBM SPSS, Microsoft Excel, and Libreoffice Calc for

subsequent analysis of the generated data

Alfonso Pierantonio

14Calculation of metrics correlationsCorrelation is used to detect cross-links and assess relationships among observed dataWe have considered Pearson’s and Spearman’s coefficients to measure the correlations among calculated metamodel metricsSpearman only for highlighting curvilinear correlationsBoth Pearson’s and Spearman’s correlation indexes assume values in the range of -1.00 (perfect negative correlation) and +1.00 (perfect positive correlation)A correlation with value 0 indicates that between two variables there is no correlation

Alfonso Pierantonio

15Selection of metrics correlationsWe have calculated the Pearson’s and Spearman’s correlation indexes for all the values of the considered metricsFor each couple we have selected the coefficient index (between Pearson and Spearman) having higher correlation values

Alfonso Pierantonio

16Selection of metrics correlationsThe bar chart shows the degree of metric correlation: the higher the bar, the more the metric is correlated with other metrics.

Alfonso Pierantonio

17Pearson’s correlation matrixAll the values greater than 0.7 or lesser than -0.73 have been highlighted to select the metrics that are most related according to the Pearson’s index.

Alfonso Pierantonio

18Pearson’s correlation matrixAll the values greater than 0.7 or lesser than -0.73 have been highlighted to select the metrics that are most related according to the Pearson’s index.

Alfonso Pierantonio

19Data analysisThe aim is to discuss and interpret the most relevant correlations between structural characteristics which have been found in the previous stages

Alfonso Pierantonio

20Data analysis: abstractionHow the number of metaclasses is related to the adoption of abstraction constructs, ie. abstract metaclasses and supertypes

Alfonso Pierantonio

21Data analysis: abstractionHow the number of metaclasses is related to the adoption of abstraction constructs, ie. abstract metaclasses and supertypes

Alfonso Pierantonio

22Data analysis: abstractionHow the number of metaclasses is related to the adoption of abstraction constructs, ie. abstract metaclasses and supertypes

MGHLMaximum hierarchical depth

Alfonso Pierantonio

23Data analysis: abstractionHow the number of metaclasses is related to the adoption of abstraction constructs, ie. abstract metaclasses and supertypes

MHSMaximum Hierarchy Siblings

Alfonso Pierantonio

24Data analysis: abstractionHow the number of metaclasses is related to the adoption of abstraction constructs, ie. abstract metaclasses and supertypes

MTCWSNumber of metaclasses having at least one super type

Alfonso Pierantonio

25Metrics

Alfonso Pierantonio

26Data analysis: abstractionThe #metaclasses and the #classes with supertypes are strongly correlated (with Pearson index 0.99) When the metamodel grows in size, the number of inheritance hierarchies also increases, however the hierarchy depth is somehow independent from the metamodel size.

Interestingly, metamodel designers prefer to add siblings in hierarchies instead of adding new hierarchy levels

Alfonso Pierantonio

27Data analysis: abstractionThis is testified by the graph that shows the values of the MHS (Max Hierarchy Sibling) and MGHL (Max generalization hierarchical level) metricsConfirmed by the Pearson correlation indexes between MC and MHS (0.70) and that between MC and MGHL (0.66).

Alfonso Pierantonio

28Data analysis: abstractionIn metamodels with at most 50 metaclasses:– the number of supertypes in

hierarchy is in between 0 and 20

– the number of siblings in a hierarchy is in between 0 and 10

– the maximum height of a hierarchy is in between 0 and 5

Alfonso Pierantonio

29Data analysis: structural featuresHow structural features are used with hierarchiesWe can consider the average number of features (ASF) and the total number of metaclasses with supertypes (MCWS) metricsThe Spearman approach permits to identify a greater correlation index

Alfonso Pierantonio

30Data analysis: structural featuresIncreasing the #metaclasses with supertypes, the average # of structural features in a metaclass decreases

Alfonso Pierantonio

31Data analysis: structural featuresBy considering metamodels having in between 1 and 50 metaclasses, the average number of features (excluding the inherited ones) of a metaclass ranges between 1 and 5

Alfonso Pierantonio

32Data analysis: structural featuresHow the number of featureless metaclasses is related to hierarchies heightThis can indicate how specializations of metaclasses can introduce or reduce structural features in metamodelsMCWS and IFLMC are strongly correlated as supported by the Pearson’s index having value 0.890

Alfonso Pierantonio

33Data analysis: structural featuresHow the number of featureless metaclasses is related to hierarchies height

Alfonso Pierantonio

34Data analysis: structural featuresBy increasing the number of metaclasses with super types, the number of metaclasses without attributes or references increases tooThis means that when hierarchies are introduced, usually existing features are subject to refactoring operations mainly to move them to super classes and to create leaves in the hierarchies inheriting features from the super types.

Alfonso Pierantonio

35Data analysis: isolated metaclassesHow isolated metaclasses are distributed In this case we considered another statistical instrument named Pareto analysis, which is commonly referred to as the 80-20 principle (20% of the causes account for 80% of the defects)About 80% of the analyzed metamodels have a percentage of isolated metaclasses in the range 0-19%, by testifying the fact that isolated metaclasses are not commonly used

Alfonso Pierantonio

36Data analysis: isolated metaclassesWe can claim that they are used only for testing or educational purposes

Alfonso Pierantonio

37ConclusionsWe proposed a number of metrics which can be used to acquire objective, transparent, and reproducible measurements of metamodels.The major goal is to better understand the main characteristic of metamodels, how they are coupled, and how they change depending on the metamodel structure A correlation analysis has been performed to identify the most cross-linked metrics, which have, in turn, been computed over 450 metamodels

Alfonso Pierantonio

38ConclusionsThese figures have been discussed in detail highlighting the followings- the adoption of inheritance is proportional to

the size of metamodels- the number of metaclasses with supertypes are

inversely proportional to the average number of structural features

- the number of metaclasses with supertypes is proportional to the number of metaclasses without attributes or references

- isolated metaclasses are not commonly used apart from testing or educational purposes.

Alfonso Pierantonio

39ConclusionsThese figures have been discussed in detail highlighting the following

Thank you