valutazione della ricerca
DESCRIPTION
La valutazione della ricerca. presentazione all'incontro di Milano del 30 settembreTRANSCRIPT
![Page 1: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/1.jpg)
Francesco Sylos Labini
Centro Fermi & ISC-CNR Roma
http://ricercatorialberi.blogspot.com/
http://www.ilfattoquotidiano.it/blog/flabini/
http://ricercatorialberi.blogspot.com1
Valutazione
![Page 2: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/2.jpg)
•How should Research be Organised? Donald Gillies, College Publication, 2008
•Economics and Research Assessment Systems Donald Gillies, In the press, 2011
•La Scienza Malata? Come la burocrazia soffoca la ricerca Laurent Ségalat, Raffaello Cortina Editore, 2010
•Du bon usage de la bibliométrie pour l’évalutation individuelle des chercheurs Institut de France, Académie de sciences, 2010
•Valutare la ricerca scientifica, Uso ed abuso degli indicatori bibliometrici Alberto Baccini, Il Mulino, 2010
Bibliografia essenziale
![Page 3: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/3.jpg)
3
Valutazione: problema chiave
Se una ipotetica commissione che avesse dovuto
decidere del finanziamento di un “progetto” al
tempo T0 che poi ha portato al “Nobel” al tempo
T1>T0, ed avesse applicato un dato criterio (ad
esempio metrico) avrebbe preso la decisione
giusta?
![Page 4: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/4.jpg)
4
Prevedere il futuro …..
Valutazione: problema chiave
![Page 5: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/5.jpg)
5
Analisi dei criteri di valutazione da un punto di vista storico
Valutazione: problema chiave
![Page 6: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/6.jpg)
6
•Top 5%
•Spazio alle fluttuazioni (50%)
•Finanziamento a pioggia (100%)
Strategie per la divisione della torta
La competizione è il motore della ricerca ?
![Page 7: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/7.jpg)
“…. Nella ricerca conta solo l' eccellenza: ciò che non è eccellente non lascia traccia nella Storia.”
(Francesco Giavazzi, 22.11.2004, Corriere della Sera)
![Page 8: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/8.jpg)
8
The two papers in Science 2004 and Nature 2005 are the most cited papers on graphene … The Science paper has also been acknowledged as “one of the
most cited recent papers in the field of Physics”.
2010
Valutazione: problema chiave
![Page 9: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/9.jpg)
A Research Assessment System (or RAS) is a system in which groups of researchers are assessed at intervals. If the assessment is good, the group retains its funding or gets more, while, if the assessment is bad, the
group’s funds are reduced or perhaps removed altogether.
Research Assessment Exercise
![Page 10: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/10.jpg)
• RAE introduced by Thatcher in 1986
•RAEs are carried out every few years in all the universities of the UK.
•Committee of assessors (academics) in each subject.
•The department submits the selected papers to the assessment committee.
•Committee study this research output and grade the department
Research Assessment Exercise
![Page 11: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/11.jpg)
•The departments which score well on the RAE are provided with research funds.
•Those which don’t score so well are provided with much smaller funds for research
•The members of such departments have to spend more time on teaching.
•Recently there have even been moves in some universities to close altogether departments which perform badly on the RAE
Research Assessment Exercise
![Page 12: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/12.jpg)
•RAE is a costly operation.
•Professional administrators (extra administrative staff)
•Extra work involved in the RAE (the time of academics also costs money, this too adds to the cost of the RAE).
Research Assessment Exercise
The question then naturally arises as to whether this expensive procedure has
actually improved the research produced in the UK. Strange to say this
question is rarely asked: Could it be making the research output of the UK
worse instead of better?
![Page 13: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/13.jpg)
•Peer review: the original RAE was based entirely on peer review.
•Biblio-metrics
Assessment of the value of the research work
![Page 14: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/14.jpg)
•Mature natural sciences as ‘normal science’ same ‘paradigm’
•Scientific revolutions new paradigm.
•Social sciences -->competing schools with different paradigms
Single vs. multi-paradigm situation
(Economics and Research Assessment Systems Donald Gillies, In the press, 2011)
![Page 15: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/15.jpg)
•Theoretical physics: relativity theory and quantum mechanics
•Economics: different schools with different political ideologies: Neo-Classical Economics, the various versions of Keynesianism, and Marxist Economics.
Physics vs Economics
(Economics and Research Assessment Systems Donald Gillies, In the press, 2011)
![Page 16: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/16.jpg)
My claim is that the research work of the members of whichever school has the largest number of members will receive the highest
valuation. So if school A is in the majority, the members of school A will receive the highest valuation. If school B is in the majority, then
the members of school B will receive the highest valuation, and so on.
If a Research Assessment System is applied to such a community, what result will it give?
(Economics and Research Assessment Systems Donald Gillies, In the press, 2011)
![Page 17: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/17.jpg)
http://www.letteraapertavalutazionericerca.it/17
![Page 18: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/18.jpg)
http://ricercatorialberi.blogspot.com 18
http://lanl.arxiv.org/abs/1008.1586
![Page 19: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/19.jpg)
19
![Page 20: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/20.jpg)
20
E’ possibile fare una classifica di scienziati (o dei progetti di ricerca) come se si trattasse di tennisti ?
Valutazione: strumenti bibliometrici
![Page 21: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/21.jpg)
http://ricercatorialberi.blogspot.com 21
Valutazione: strumenti bibliometrici
“Before we develop a pseudoscience of citation analysis, we should remind ourselves that what matters absolutely is the scientific content of a paper and that nothing will substitute for either
knowing it or reading it. We should also recognize that citation often tells us more about the sociology
of than about the science itself”
Sydney Brenner “Citation Science”
Science 3 July 1998: Vol. 281 no. 5373 p. 53
![Page 22: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/22.jpg)
![Page 23: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/23.jpg)
![Page 24: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/24.jpg)
http://ricercatorialberi.blogspot.com 24
Valutazione: strumenti bibliometrici
![Page 25: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/25.jpg)
25
Valutazione: strumenti bibliometrici
![Page 26: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/26.jpg)
26
Valutazione: strumenti bibliometrici
![Page 27: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/27.jpg)
http://ricercatorialberi.blogspot.com 27
Valutazione: strumenti bibliometrici
![Page 28: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/28.jpg)
http://ricercatorialberi.blogspot.com 28
Valutazione: fisici
![Page 29: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/29.jpg)
http://ricercatorialberi.blogspot.com 29
Valutazione: economisti
![Page 30: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/30.jpg)
http://ricercatorialberi.blogspot.com 30
Valutazione: economisti
![Page 31: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/31.jpg)
Valutazione delle chiacchere
![Page 32: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/32.jpg)
•Campi con diversi paradigmi
•Campi in cui i soggetti non sono “universali” (o globali)
•Campi interdisciplinari
•Campi applicativi
Aspetti critici (ed irrisolti)
![Page 33: Valutazione della ricerca](https://reader033.vdocuments.site/reader033/viewer/2022060120/5592feca1a28ab9f1e8b4623/html5/thumbnails/33.jpg)
Conclusioni
•Valutare è necessario ma molto delicato
•Cultura della valutazione
•Ogni campo ha una storia a sé
•E’ difficile valutare chi lavora
•E’ facile identificare chi non lavora