unica core group on evaluation in research and higher education 29 october 2010 vienna 1
TRANSCRIPT
UNICA Core Group on Evaluation in Research and Higher Education
29 October 2010
Vienna
1
I. Context
2
Data “mysteriously” aggregatedMultidimensionality of research excellence neglected unreliable data etc.
Current rankings not very useful for benchmarking or to develop research in one university
II. Objectives
3
to provide to the UNICA network a toolbox of research evaluation methodologies adapted to different “cases” identified as interesting by the UNICA members
to pool resources in learning from each other about best practices in the context of benchmarking activities
to develop a methodology accepted by Unica members, and based on expertise sharing, complementarities and harmonization of databases
III. Working Group
4
10 universities 4 meetings
Carte d’europe et universitésLeurs classements ectuels
IV. Methodology (1)
5
1. To list the possible objects to evaluateProjectsResearchers Teams, faculties, centres, ……
2. To list the possible goals of an evaluationTypologyRankingSwot analysis …
3. To list the possible users of an evaluationRectorsResearchersGovernments, funding agencies Media, public, …
One case is defined by a combination of answers to each of the 3 questions
IV. Methodology (2)
Focus on Case 2
Case study number
Object Goals Users
1 Faculties Strong and weak points Rectors
2 Group/department (smaller groups than “faculties”)
Strong and weak points Rectors
3 Teams, groups, faculties
Strong and weak points Public
6
III. Methodology (3)
7
Agreement to limit the number of indicators per dimension Dimensions: what do we want to take into account Indicators: how do we want to take into account
Two first fields: Economic/business, Physics
5 Dimensions and 20 indicators D1: scientific output (7 indicators) D2: Attractiveness/internationalization (5 indicators) D3: Research training (3 indicators) D4: Transfer of technology (5 indicators) D5: Links to education (indicators still to be selected)
+ Basic Data on university (Global budget, staff, etc)
8
DIMENSION 1: SCIENTIFIC OUTPUT
1. Publications in Scopus, WoF
2. Peer review chapters of books
3. Peer review books
4. # Citations in databases
5. # Patents
6. # ERC grants
7. # Highly scientists
III. Methodology (4)
IV. What has been done ?
9
Agreement on dimensions and indicatorsData collection from several members of the group Analysis of difficulties
IV. Challenges and next steps
10
To get data from all membersAnalysis and graphical representationNeed for common definitions of indicatorsTry to use existing dataTo have a meeting with Unica members involved in Multirank.