university rankings; an overview for the municipality of delft july 2013
DESCRIPTION
I gave this presentation for several strategic policy advisor the municipality of Delft in the ‘Metropoollab’. Thanks to Johan Verweij & Jan Salden I elaborated on Johans presentations & used U-Multirank sheets of Jan.TRANSCRIPT
1Challenge the future
International Rankings of Universities
An overview for the municipality of Delft Kim Huijpen, Corporate Policy Affairs | 04/07/13
2Challenge the future
About me
Kim Huijpen• Policy Advisor TU Delft
• Strategic Development / Corporate Policy Affairs
• Member of the Delft city council• Committee Society and Housing • Member of D66
3Challenge the future
International university rankings
1. Context2. Criticism3. Overview4. The position of the TU Delft in important rankings5. New initiatives to improve rankings6. How do we use international rankings?
And how do you use international university rankings?
4Challenge the future
Context
Rankings fill in a need• Stakeholders – students, parents, governments,
accreditation councils, industry (inter)national organizations – want to know the differences between HEI’s and how they perform
Rankings are more and more used (directly or indirectly via reputation)• By the media• By governmental institutions (reallocation of funds)• By students (Asia)• By HEI’s themselves! For marketing purposes or to
select partners for cooperation• By local governments?
5Challenge the future
International rankings, criticism and new developments
Most important international rankings in 2013• QS-, THE-, Shanghai-, Leiden-ranking• Not 4 rankings, but much more (also subject & reputation
rankings)
Criticism• Content: bias for big & old universities, focus on research,
bias for natural & medical sciences, language bias, comparison of whole HEI’s
• Methodology: adding up all kind of indicators, numbering, dubious weighting, intransparency, institutions deliver data, methodological changes
6Challenge the future
Criticism
Conceptual1. Some universities have an advantage: Anglo-Saxon,
beta- and medical disciplines, focus on research, big, old, general
2. You can’t compare whole universities3. You can’t add up all the indicators
Methodology4. Underpinning of the weight factors5. Sensitivity for outliers: best HEI=100 (z-scores are
better)6. Methodological changes in time
Data7. Limited or no insight in the raw data8. Data provided by HEI’s themselves: mistakes,
manipulation
7Challenge the future
Overview: similarities and differences
Ranking
Focus Indicators
Data Time Type
THE ResearchEducationInternat.Income
SubjectiveObjective
Own ResearchDbase (WoS)Data HEI’s
Present GeneralField
QS ResearchEducationInternat.
SubjectiveObjective
Own ResearchDbase (Scopus)Data HEI’s
Present GeneralField
Shangha
i
Research Objective Dbases (e.g. WoS, Nobel-prize.org)
PastPresent
GeneralFieldSubject
Leiden Research Objective Dbase (WoS) Present General
HEEACT Research Objective Dbase (WoS/ESI) Present GeneralFieldSubject
8Challenge the future
TU Delft in rankings ‘12 & spring ‘13
World University Rankings
Engineering/Technology
RankingsOther Rankings
Subject Rankings
Top 10 UIRC Scoreboard (3)
QS Civil & Struct. Eng. (4)
QS Chemical Eng. (10)
Top 50 QS (18)
QS Environmental Sciences (17)
THE (32) QS Materials Science (32)
QS Mechanical Eng. (18)
QS Electrical Eng. (42)
Top 100
THE (77) Shanghai (76-100)
THE Reputation (51-60)
4 QS Subject Rankings
Top 200 QS (103) 2 QS Subject Rankings
Leiden (164)
Top 300
Shanghai (201-300)
Taiwan (276)
9Challenge the future
THE-ranking (with Thomson Reuters)
Fields:• ‘Engineering &
Technology’• ‘Life Sciences’• ‘Clinical, pre-
clinical & Health’
• ‘Physical Science’
• ‘Social Sciences’• ‘Arts &
Humanities’
Ranking by field: based on same 13 indicators with slightly different weights
10Challenge the future
What is citation impact?
• Citation impact is one of the key indicators in most rankings
• With a citation an author acknowledges the original author, year, title, and source of an idea in a new publication
• Citations are measures of the impact of the cited work
• Citation ‘cultures’ differ between disciplines therefore we calculate citation impact normalized for field differences
11Challenge the future
Position of 3TU, LDE & IDEA League in THE ranking
University World University Rankings 2012-13 (2011)
Engineering & Technology Ranking 2012-13 (2011)
TU Delft 77 (104) 32 (22)TU Eindhoven 114 (115) -Universiteit Twente 187 (200) -Universiteit Leiden 64 (79) -Erasmus Universiteit Rotterdam
72 (157) -
Imperial College London 8 (8) 10 (10)ETH Zürich 12 (15) 8 (9)Ecole Polytechnique* 62 (63) 29 (29)Aachen RWTH 154 (168) -* ParisTech exists of eleven ‘Grandes Ecoles Paris’ of which Ecole Polytechnique is the most well known.
12Challenge the future
THE Engineering and Technology Universities 2012
Position THE Engineering & Technology ranking 2012-13 (2011)
1 Caltech, US (2)2 Princeton University, US (3) 2 MIT, US (1)4 University of California, Berkeley,
US (4)5 University of Cambridge, VK (6) /
Stanford University, US (5)Highest non UK/US nr. 8 ETH Zürich (Switzerland)Highest European (non UK/US)
nr. 8 ETH Zürich (Switzerland)
13Challenge the future
Indicators QS ranking
Fields:• Arts & Humanities• Engineering & Technology• Life Sciences & Medicine• Natural Sciences• Social Sciences &
Management
Ranking by field: • Based on same
indicators• Weightings are
different
14Challenge the future
Position of 3TU, LDE & IDEA League in QS ranking
* ParisTech exists of eleven ‘Grandes Ecoles Paris’ of which Ecole Polytechnique is the most well known.
University General ranking (2011)
Engineering and Technology ranking (2011)
Natural Sciences ranking (2011)
TU Delft 103 (104)
18 (18) 91 (79)
TU Eindhoven 158 (146)
67 (61) 186 (177)
Universiteit Twente 224 (226)
101 (116) 267 (229)
Universiteit Leiden 75 (88) - 89 (80)Erasmus Universiteit Rotterdam
99 (103) - -
Imperial College London 6 (6) 6 (6) 11 (11)ETH Zürich 13 (18) 8 (8) 10 (10)Ecole Polytechnique* 41 (36) 36 (36) 43 (40)Aachen RWTH 150
(140)30 (35) 83 (82)
15Challenge the future
Position of TU Delft in QS Subject rankings
16Challenge the future
Indicators Shanghai-ranking (since '03)
Focus Indicators Weighting
Quality of
education
Alumni winning Nobel
prizes and fields medals
Alumn
i
10%
Quality of
faculty
Staff winning Nobel prizes
and fields medals
Award 20%
Highly cited researchers HiCI 20%
Research
output
Articles and papers in
Nature and Science
N&S 20%
Articles and papers in SCI
and SSCI
PUB 20%
17Challenge the future
Position of 3TU, LDE & IDEA League in Shanghai ranking
* ParisTech exists of eleven ‘Grandes Ecoles Paris’ of which Ecole Polytechnique is the most well known.
University Academic Ranking of World Universities - 2012 (2011)
TU Delft 201-300
(151-200)
Universiteit Twente 301-400
(301-400)
TU Eindhoven 301-400
(301-400)
Universiteit Leiden 73 (65)Erasmus Universiteit Rotterdam
151-200
(151-200)
ETH Zürich 23 (23)Imperial College London 24 (24)Ecole Polytechnique* 301-
400(301-400)
RWTH Aachen University 201-300
(201-300)
18Challenge the future
Shanghai-ranking calculated trend
2012 2011 2010 2009 2008 2007 2006 2005 2004 20030
50100150200250300
214 197 185 193 197 194 191234 224 242
Calculated positions TU Delft on Shanghai ranking
These calculated scores are based on calculations per indicator by University of Groningen
19Challenge the future
Leiden ranking
20Challenge the future
Position TU Delft on impact indicators Leiden Ranking 2010-2013
Ranking 2010 2011 2011* 2013 2013*Leiden-ranking (Worldwide top 500)
Old ‘Crown Indicator' (CPP/FCSm)
123
‘Alternative Crown Indicator' (MNCS)
176 189 99 219 168
Proportion top 10% publications (PPtop 10%)
179 115 203 164
* Size-independent indicators, impact indicators using fractional counting & exclude publications in special types of journals
21Challenge the future
Position Dutch universities Industry Research Connections 2013
World NL Uni
PP(UI collab)
1 1 Eindhoven University of Technology 15,6%2 2 Delft University of Technology 14,0%
18 3Wageningen University and Research Centre 10,1%
24 4 University of Twente 9,8%137 5 Erasmus University Rotterdam 7,4%147 6 Leiden University 7,3%153 7 Maastricht University 7,2%160 8 Utrecht University 7,1%186 9 University of Groningen 6,9%214 10 VU University Amsterdam 6,5%253 11 Radboud University Nijmegen 6,1%257 12 University of Amsterdam 6,1%
Or #3 with different counting
22Challenge the future
New developments
Improvement of existing rankings• More attention for education, finance and field (QS,
THE)• More representative survey on reputation (QS, THE)• Rankings per field and subject (Shanghai, QS)
New rankings and classifications (education and third mission, fields and subjects, ranking per indicator, no numbering)• CHE university ranking: BSc-students• CHE excellence ranking: MSc/PhD-students • U-Map (CHEPS): types/profiles• U-Multirank (CHERPA/CHE): institutional and field*
rankings* e.g. engineering
23Challenge the future 2
3
24Challenge the future
What is U-Multirank?
24
Multi-dimensional
• Performance comparison not only on research but also on education, exploitation, international orientation and regional involvement.
Multi-level
• Performance profiles based on a broad set of indicators. The performance profiles are available at two levels: institution as a whole and underlying disciplinary fields.Multi-stakeholder
• Designed in close consultation with stakeholders and intended for - students, administrators, policy makers, employers, etc. - to meet their needs.
Multi-ranking
• Users can decide which areas of performance to include in the comparison of the selected group of universities; in this way U-Multirank produces personalised rankings.
25Challenge the future
Multi-dimensional, -level, -stakeholder, -ranking
25
26Challenge the future
QS Best student cities
Methodology• “Two pre-requisites have been established to identify
the cities evaluated in this exercise. The first is that each city must have a population of over 250,000, the second that it must be home to at least two ranked institutions. Current calculations suggest that 98 cities in the world qualify on this basis.”
1: Paris, 2: London, 3: Boston, 4: Melbourne , 5: Vienna
and 36: Amsterdam
27Challenge the future
Indicators of QS Best student cities
• Student mix• Student Population: as a proportion of the city’s population• International Volume: number of international students in
the city International Ratio: number of international students as a proportion of all students
• Quality of living • Mercer Quality of Living Survey 2011
• Employer activity • Domestic Employer Popularity: Number of domestic
employers who identified one institution in the city as producing excellent graduates
• International Employer Popularity [x2]• Affordability
• Tuition Fees [x2], Big Mac Index & Mercer Cost of Living Index
28Challenge the future
Mercer's Quality of Living ranking
The Quality-of-living index encompasses 39 different factors within the following 10 categories:• Political and social environment• Economic environment• Socio-cultural environment• Medical and health considerations• Schools and education• Public services and transport• Recreation• Consumer goods• Housing• Natural environment
29Challenge the future
How do we use international rankings?
Until now• Participation in
rankings• Internal memos
for the Executive Board
• Annual report• Roadmap 2020 • Website‘facts
and figures’• Marketing and
PR
30Challenge the future
Messages
• More and more international rankings (need)• Are used by several stakeholders and affect your
reputation• Are biased and have methodological drawbacks
• However, methodologies are improving• Nevertheless, important to be in the rankings• It is difficult for specialized universities to reach a high
position in general rankings (TU Delft: technology/engineering)• However, field normalization is improving
• New initiatives to improve international rankings:• U Multirank
31Challenge the future
Questions and discussion:
Which rankings are relevant for the municipality of Delft ?Which rankings do you choose for marketing purposes?
• More information: • www.3tu.nl/uploads/media/Rankings_en_3TU.pdf
• Thanks to Johan Verweij & Jan Salden• I elaborated on Johans presentations & used U-Multirank sheets
of Jan
Kim Huijpen, Policy Advisor, TU Delft / Corporate Policy AffairsT +31 (0)15 27 85296 | E [email protected] | @KimHuijpen
32Challenge the future
References
Some ranking websites:• http://
www.timeshighereducation.co.uk/world-university-rankings/2012-13/world-ranking
• http://www.topuniversities.com/university-rankings/world-university-rankings/2012
• http://www.shanghairanking.com/ARWU2012.html
• http://www.leidenranking.com/ranking
• http://www.topuniversities.com/best-student-cities
• http://nturanking.lis.ntu.edu.tw/DataPage/OverallRanking.aspx
• http://
www.timeshighereducation.co.uk/world-university-rankings/2013/reputation-ranking
Twitter: #unirankings