metodi statistici per la valutazione - home page | iris ... › retrieve › handle › 11574 ›...
TRANSCRIPT
IES 2013
Innovazione e società
Innovation and Society
Metodi statistici per la valutazione
Università degli Studi di Milano-Bicocca 9 - 10 dicembre 2013
CONTRIBUTI SPONTANEI
A cura di
Rosa Falotico, Paolo Mariani, Biancamaria Zavanella
Patrocini
B-ASC (Bicocca Applied Statistics Center Business Intelligence & Data Mining)
Università degli Studi di Milano Bicocca
SIS (Società Italiana di Statistica)
Comune di Milano
IES 2013 - INNOVAZIONE E SOCIETÀ, METODI STATISTICI PER LA VALUTAZIONE INNOVATION AND SOCIETY, UNIVERSITY OF MILANO – BICOCCA, DECEMBER, 9 - 10 2013
PROGRAM OF THE MEETING December, 9 09:00-10:00 U7.12 Registration 10:00-10:15 U7.12 Opening Address 10:15-11:00 U7.12 Plenary Session:
Chair/Discussant: Luigi D’Ambra Speaker: Ron S. Kenett (Bayesian Network Applications to Customer Surveys)
11:00-11:15 Break (U7, 2nd floor, common room) 11:15-12:15 U7.12 Invited Session: “Evaluation of Teaching”
Chair/Discussant: Carla Rampichini Speakers: Donata Marasini, Piero Quatto and Enrico Ripamonti (A measure of ordinal inter-rater agreement); Stefania Mignani and Roberto Ricci (Learning statistics at compulsory education: the achievement results based on the Invalsi tests); Bruno Bertaccini, Leonardo Grilli and Carla Rampichini (Are course ratings predictive of freshmen performances? Evidence from the University of Florence)
12:15-13:15 U7.12 Invited Session: “Effectiveness Evaluation and Career Opportunities”
Chair/Discussant: Ornella Giambalvo Speakers: Lucio Masserini and Monica Pratesi (A small area model for analysing job placement survey data of the STELLA Consortium); Giovanna Boccuzzo and Luigi Fabbris (A model for measuring the educational effectiveness of Italian universities); Soru A. “L’esperienza Specula”
U7.17 Contributed Session: “Evaluation of Public Administration and Justice” Chair/Discussant: Giuseppe Boari
13:15-14:15 Lunch (U7, 2nd floor, common room) 14:15-15:15 U7.12 Invited Session: “Evaluation of Research”
Chair/Discussant: Eugenio Brentari Speakers: Sergio Scippacercola (The contradictions of the evaluations ANVUR, GEV13, SIS, VQR of scientific journals in statistics); Daniela Cocchi, Giuseppe Cavaliere, Marzia Freo, Simone Giannerini, Mario Mazzocchi, Carlo Trivisano and Cinzia Viroli (A support for classifying the scientific papers in a University Department); Roberta Paroli (Journals Classification: the Università Cattolica del Sacro Cuore experience)
15:15-16-30 U7.12 Invited Session: “Assessment of the Italian University System: the Role of Statistics and Statisticians” Chair/Discussant: Bruno Chiandotto Speakers: Massimo Attanasio, Vincenza Capursi, and Giulio Ghellini (Statistical tools to understand teaching activities and students paths in the Italian Universities); Maurizio Carpita and Donata Marasini (Statisticians and quality assurance in the University)
16:30-16:45 Break (U7, 2nd floor, common room) 16:45-17:45 U7.17 Contributed Session: “Evaluation of Teaching, Research and Career
Opportunities A” Chair/Discussant: Carla Rampichini
U7 De Lillo Contributed Session: “Evaluation of Teaching, Research and Career Opportunities B” Chair/Discussant: Marisa Civardi
U7.12 Contributed Session: “Evaluation of Public and Private Sector A” Chair/Discussant: Bruno Chiandotto
December, 10 09:00-10:00 U7.17 Contributed Session: “Statistical Methods for Evaluation A”
Chair/Discussant: Gianna Monti
U7.12 Contributed Session: “Evaluation of Public and Private Sector B” Chair/Discussant: Eugenio Brentari
10:00-11:00 U7.12 Invited Session: “Evaluation in Public Administration and Policy”
Chair/Discussant: Mario Bolzan Speakers: Giovanni Urbani (Evaluation policy and performance management in the current crisis); Lodovico Conzimu (The evaluation of active labour policies: the case of Employment Service Centres); Michele Benedetti (eGovernment under certain conditions: the needs of the PA who wants to renew); Anna Giraldo (Impact evaluation of public policy)
U7.17 Contributed Session: “Statistical Methods for Evaluation B” Chair/Discussant: Giovanna Nicolini
11:00-11:30 Break (U7, 2nd floor, common room) 11:30-12:30 U7.17 Invited Session: “Evaluation of Justice”
Chair/Discussant: Francesco D’Ovidio Speakers: Fabio Bartolomeo (The performance of the Italian Civil Justice and comparison with other European judicial systems); Marilene Lorizio and Antonia Rosa Gurrieri (Efficiency of Justice and Economic Systems); Laura Antonucci, Corrado Crocetta and Francesco d’Ovidio (Evaluation of Italian Judicial System)
U7.12 Round Table: “Evaluation of Public and Private Managers”
Chair/Discussant: Michele Gallo / Paolo Mariani Speakers: Donato Limone, Pietro Traverso, Angela Paladino and Federico Neri, Nicoletta Magnetti and Mario Bolzan
12:30-13:45 U7.12 Plenary Session: “Statistical methods for developing, testing and analyzing the customer satisfaction questionnaire”
Chair/Discussant: Luigi D’Ambra Speakers: Pasquale Sarnacchiaro, Michele Gallo and Luigi D’Ambra (Three-way decomposition of weighted log-odds ratio for customer satisfaction analysis); Maurizio Carpita and Enrico Ciavolino (MEM and SEM in the GME framework: Modelling Perception and Satisfaction); Alper Basaran (Multiple correspondence analysis for the student perception in quality in higher education)
TITOLO AUTORI
EVALUATION OF PUBLIC ADMINISTRATION AND JUSTICE
Efficiency COMPARISON ON ADMINISTRATIVE business regulation in the G20 Pianura, Pavone
Multivariate analysis for the ex post evaluation of the environmental fiscal reform: a methodological proposal Arbolino , Romano
Evaluation indices of the judicial system and ITC developments in civil procedure Cusatelli, Giacalone
THE EFFECT OF RETIREMENT ON ITALIAN COUPLES’ LABOUR DIVISION: A SIMULTANEOUS EQUATION APPROACH
Di Pino Incognito, Caltabiano, Campolo
Migration policy: An assessment on the North Africa emergency provisions. Saraceno, della Pellegrina, Suardi
The evaluation of active labour policies: the case of Employment Service Centres (Centri Servizi per il Lavoro). Conzimu
On measuring polarization for ordinal data Mussini
EVALUATION OF TEACHING, RESEARCH AND CAREER OPPORTUNITIES A
Evaluation of University Human Capital: knowledge accumulation estimation in a real case Simonetto, Zavarrone
A Multilevel Model to analyze the Italian students’ achievement Crisci, Paletta, D'ambra
Some Indicators for Tracking University Student Progression Boari, Cantaluppi, Torriero
A finite mixture model approach on the first year university drop out probability Matilde Bini, Masserini
Assessing the quality of universities: a value addedapproach Ghiselli, Ferrante EVALUATION OF TEACHING, RESEARCH AND CAREER OPPORTUNITIES B
Do students answer correctly? An empirical study using Rasch and CUB models
dOvidio, Leogrande, Mancarella, Schinzano
Mathematics and reading literacy and cultural behaviors of pupils. A Multilevel Latent Class Analysis Porcu, Tedesco
Examination of collaborations between universities and companies Kovacs, Marton Vilmanyi
Job opportunities and academic drop-out: the case of the University of Trieste Pauli, Chies, Graziosi
Longitudinal Analysis of Post-diploma Orientation among Young People of Bari Basile, d'Ovidio
EVALUATION OF PUBLIC AND PRIVATE SECTOR A
Outsourcing services in the Italian National Health Service: the satisfaction of private and public operators. Falotico, Mariani, Zavanella
A multivariate analysis of the quality of public transport services
d'Ovidio, Leogrande, Mancarella, Schinzano, Viola
SOCIAL INITIATIVES INFLUENCE ON CUSTOMER BEHAVIOUR: A STRUCTURAL EQUATION model Sarnacchiaro, Boccia
The multilevel nature of knowledge sharing processes: Preliminary evidence from innovative manufacturing sectors Cavaliere, Lombardi
Evaluating the current evolution of GDP towards quality
Frenda, Piana, Scippacercola
EVALUATION OF PUBLIC AND PRIVATE SECTOR B
Hospitalization in geriatric wards: considerations on the evaluation of the outcome Crippa, Marshall, Merchich, Zenga,
PLS path analysis and its application in an examination of psychological sense of brand community Kazar
The factors of the bank customer (dis)loyalty in portfolio choices Salinas, Schinzano
Evaluating the financial literacy in Italy. Mariangela Zenga, Bongini, Trivellato
STATISTICAL METHODS FOR EVALUATION A Statistical nomogram for the evaluation of mooring activities in port areas Raffaele Giuseppe
A latent variable model for market segmentation Greselin, Ingrassia
Componential Segmentation based Conjoint Analysis: the Dummy-coded Parametrization of the Model De Luca
Confidence intervals for ranks with application to performance indicators Solari, van Zwet, Meier, Goeman
STATISTICAL METHODS FOR EVALUATION B
Estimating technical efficiency in the italian Municipalities Settimi, Vidoli, Fusco, Ballanti
Evaluating inter-rater agreement for ordinal data: A comparison of weighted indexes Ripamonti, Quatto
A comparing among the different approaches for DIF analysis. Camminatiello, Manini, Gallo
A statistical based H index for the evaluation of e-markets Giudici, Cerchiello
A comparing among the different approaches for DIF analysis.
T. Menini1, I. Camminatiello2, M. Gallo3
Our study focuses on measurement instrument for teaching evaluating (Carpita; 2008). One of the most proper statistic methodology is Item Response Theory - IRT (Baker, Kim; 2004), that collects several models, the simplest is Rasch Model - RM (1960). The RM makes several assumptions. In some cases, it can happen it is not possible to identify any set of items consistent with the hypothesis of the RM: this can be caused by a ill calibrated questionnaire or a mixture of individuals apparently belonging to the same population, but in reality related to different populations (Camminatiello, Gallo, Menini; 2010). The latter case can be a symptom of a different functioning of the items corresponding to distinct groups of individuals: this phenomenon is called Differential Item Functioning or DIF. In literature there are several DIF diagnostics (Glas & Verhelst, 1995), but the most used and implemented in the most commonly used software (Wu, Adams, Wilson; 1998) is based on the residual analysis among the subgroups identified by one or more aggregation variables. Other authors (Wang; 1998) treat the DIF as a parameter to be estimated rather simply a sign to be detected. In this study we are going to compare the different approaches for analysing the DIF.
References
1. Baker, F., Kim S.: Item response theory. Parameter estimation techniques, Dekker, New York (2004).
2. Camminatiello I., Gallo M., Menini T.: The Rasch Model for Evaluating Italian Student Performance Journal of Applied Quantitative Methods, Volume 5, Issue 2, June 30 (2010).
3. Carpita M.: Presente e futuro dei sistemi di valutazione dell’università, Statistica & Società, anno VI, n. 1, pp. 2-18.
4. Glas, A., Verhelst, N.: Tests of fit for polytomous rasch models, in G. H. Fischer e I. W. Molenaar, eds, Rasch models. Foundations, recent developments, and applications. Springer-Verlag, pp. 325–352 (1995).
5. Wang, W.C.: An ANOVA-like Rasch analysis of differenzial item functioning. Report (1998).
6. Wu, M., Adams, R., Wilson M.: Acer Conquest. Generalised item response modelling software, Acer Press (1998).
1 Tullio Menini, “Orientale” University of Naples; [email protected] 2 Ida Camminatiello, Federico II University of Naples; [email protected]: 3 Michele Gallo,“Orientale” University of Naples, [email protected]