Danube Rectors’ Conference. University of Excelence. Teaching, Learning, Research and Community Services

4th - 7th November, 2010, Cluj-Napoca

Peer evaluation and the UBB Excellence Initiative

Prof. dr. Andrei Mărcuș - vice-rectorGelu Gherghin – Centre for Quality Management

BABEȘ-BOLYAI UNIVERSITY’ EXCELLENCE COMPETITION 2010

• Rector’s initiative – December 2009

• Peer valuation in Romania:

- professorships (CNATDCU) - research grants (CNCSIS) - UBB internal competitions

UBB - EXCELLENCE COMPETITION 2010

DOCUMENTS

• Strategy to obtain and consecrate excellence in UBB’s performance

• The participation of UBB in competitions for recognition of Excellence

• Students as partners. The statements of UBB’s Rector, regarding the Excellence Policy

• Criteria for the recognition of Excellence within Babeș-Bolyai University

• Excellence Competition 2010 - Methodology

UBB EXCELLENCE COMPETITION 2010

DEBATES

• Fields• Categories• Criteria (Inspiration from CHE ExcellenceRanking, THE-QS, Shanghai ARWU, CNCSIS 2010 – Excellence in Research, U-Map, U-Multirank)

• What to report, especially in fields like arts, sports?• Assigning Points?

UBB EXCELLENCE COMPETITION 2010

MathematicsComputer SciencePhysicsChemistryGeologyGeographyBiologyChemical Engineering,Biotechnology, Food Engineering and SecurityEnvironmental Science and EngineeringLegal and Administrative SciencesEconomic Sciences

Political Science and International RelationsCommunication Sciences and MediaSociology, Anthropology and SocialWorkPsychologyEducational Sciences Physical Education and SportPhilosophyHistoryTheologyPhilologyCinema and Performing ArtsMusic

UBB EXCELLENCE COMPETITION 2010

CATEGORIES

• Research groups

• Research and services groups

• Personalities

• Bachelor Programmes

• Master Programmes

• Doctoral / Postdoctoral Programmes

INDICATORS - CHE EXCELLENCE RANKING

http://www.excellenceranking.org/eusid/EUSID

Number of publications in Web of Science

Citations (normalized)

Outstanding researchers

Number of projects in the Marie Curie programme

Student mobility

Teaching staff mobility

Erasmus-Mundus Master programmes

Book citations

INDICATORS: THE-QS WORLD UNIV RANKINGS

http://www.topuniversities.com

Indicator Description Weight

Academic Peer Review

Composite score drawn from peer review survey (divided into five subject areas)

40%

Employer Review Score based on responses to employer survey. 10%

Faculty Student Ratio Score based on student faculty ratio 20%

Citations per Faculty Score based on research performance factored against the size of the research body 20%

International Faculty Score based on proportion of international faculty 5%

International Students

Score based on proportion of international students 5%

http://www.topuniversities.com

THE WORLD UNIV RANKINGS 2010-2011“objective”, “rigorous”, “robust”, “transparent”

http://www.timeshighereducation.co.uk/world-university-rankings/

Citation - research influence 32,50%

Citation impact (normalised average citations per paper) 32,50%

Research - volume, income and reputation 30 %

Reputational survey - research 19,50% Papers per academic and research staff 4,50%

Research income (scaled) 5,25% Public reseach /total research income 0,75%

Teaching - the learning environment 30%

Reputational survey – teaching 15% Income per academic 2,25%

PhD awards per academic 6% PhD awards/bachelor’s awards 2,25%

Undergraduates admitted per academic 4,5%

International mix - staff and students 5%

Ratio of international to domestic staff 3%

Ratio of international to domestic students 2%

Industry income - innovation 2,5%

Research income from industry (per academic staff) 2,5%

ACADEMIC RANKING OF WORLD UNIVERSITIES (ARWU - SHANGHAI)

http://www.arwu.org

Criteria Indicator Weight

Education Quality Alumni of an institution winning Nobel Prizes and Fields Medals 10%

Faculty QualityNobel Prizes and Fields Medals

20%

Highly cited researchers in 21 broad subjects 20%

Research OutputArticles published in Nature and Science 20%

Articles Indexed in Science Citation Index-Expanded and Social Science Citation Index 20%

Per Capita Performance Per capita academic performance10%

http://www.arwu.org

CNCSIS 2010 – EXCELLENCE IN RESEARCH

4. Primary performance criteria

4.1. ISI publications 4.2. Patents 4.3 Research grants

5. Secondary performance criteria

5.1. Papers published in indexed journals

5.2. Papers published at international conferences

6. Qualitative performance criteria

6.1. Products and technologies

7. Professional prestige

7.1. Invited talks at prestigious international conferences

7.5. Members of the Romanian Academy

7.2. Visiting professor at prestigious universities

7.6. PhD Advisors

7.3. Leadership positions in international organizations

7.7. National prizes granted by the Romanian Academy and by CNCSIS

U-MAP, U-MULTIRANK

U-MAP a tool to create

“transparency of diversity”

U-MULTIRANKa multidimensional classification

• educational profile

• student profile

• research involvement

• knowledge exchange

• international orientation

• regional engagement

• multidimensional perspective of “institutional profiles”

• no overall “league table”

• no composite indicators

• two-level analysis (institutional and “field”)

• stakeholders driven approach

http://www.u-map.eu

FIRST PROPOSAL OF INDICATORS

The assessment involves a qualitative analysis of the contributions, not just their counting

A. Personalities• books, papers and patents positively cited in the international system

B. Research groups• individual performance of group members, according to the criteria from A.• income generating contracts with companies • international research projects (FP7, etc.)• international researchers in the group

C. Post-Graduate Programs (Masters / PhD)• individual performance of teachers, as in A.• international publications of PhD students.• Participation in international programs (Erasmus Mundus, etc.).• International students in the program.• International teachers involved in the program.

FINAL LIST OF INDICATORS

1. Research programs - national and international publications relevant in the field; fundamental conferences organized in the field; technological impact, domestic and foreign master/PhD students attracted; national and international grants, national and internationals awards; internationally relevant journals published.

2. Personalities - international publications relevant in the field, visiting professor at prestigious universities; invited speaker at relevant international conferences; citation in national and international literature; membership in the boards of international journals; national awards; postgraduates students attracted; financial resources attracted; membership in international professional committees; international expertise.

FINAL LIST OF INDICATORS

3. Master, PhD, Postoctoral programmes - papers published by scietific advisor/doctoral school, cited in international literature; international awards; national awards; the impact of the doctoral school results in society; the number of domestic and foreign master and PhD students attracted; graduates employed in important positions, in relevant institutions.

4. Bachelor programmes - cited papers of the academic staff; international awards; national awards; the impact of the academic staff works in society; the number of domestic and foreign students attracted; graduates employed in important positions, in relevant institutions.

5. Research and services groups

EVALUATION: ART OR SCIENCE ?

• Definitely, it is not science, but methodologies can examined !

• Institutional and individual rankings depend on the methodology

• The use of statistical indicators in individual appraisal in not appropriate

REFERENCES

UBB (2010). Criteria for the recognition of Excellence within Babeș-Bolyai University

http://www.arwu.org

http://www.excellenceranking.org/eusid/EUSID

http://www.timeshighereducation.co.uk/world-university-rankings

http://www.topuniversities.com

http://www.u-map.eu

S. Berghoff et al. (2009, 2010). Identifying the Best: The CHE Excellence Ranking for Natural Science, Economics, Political Science and Psychology in Europe.

REFERENCES

Dehon, C., McCathie and V. Verardi (2009) Uncovering excellence in academic rankings: a closer look at the Shanghai ranking.

J.-C. Billaut, D. Bouyssou and Ph. Vincke. (2009) Should you believe in the Shanghai ranking? An MCDM view.

J. Ewing, R. Adler and P. Taylor. (2008) Joint IMU/ICIAM/IMS-Committee on Quantitative Assessment of Research. A report on Citation Statistics.