1 academic ranking of world universities methodologies and problems may 15, 2007 by professor nian...
Post on 20-Dec-2015
217 views
TRANSCRIPT
1
Academic Ranking of World Academic Ranking of World
UniversitiesUniversities
Methodologies and ProblemsMethodologies and Problems
May 15, 2007May 15, 2007
By Professor Nian Cai LiuBy Professor Nian Cai LiuInstitute of Higher Education and Center for World-Class UniversitiesInstitute of Higher Education and Center for World-Class Universities
Shanghai Jiao Tong University, ChinaShanghai Jiao Tong University, China
Colloque de l’Académie des sciences "Évolution des publications scientifiques - Le regard des chercColloque de l’Académie des sciences "Évolution des publications scientifiques - Le regard des chercheurs" des 14-15 mai 2007heurs" des 14-15 mai 2007
2
Outline
Purposes
Methodologies &
Results
Problems & Discussion
Ranking by Subject
Fields
Final Remarks
3
Purposes
4
Dream of Chinese for WCU
World-class university (WCU) is a dream
for generations of Chinese. It’s not only
for pride, but also for the future of
China.
Recently, Chinese government has
launched several initiatives for research
universities. The best-known one is
specially designed to build WCU (985
Project).
5
Goals of Top Chinese Universities
Many top Chinese universities have
setup their strategic goals as WCU.
Most of them have also set time tables
for reaching the goal of WCU. For
example:
2016 for Peking University
2020 for Tsinghua University
6
Questions About WCU
Is there a clear definition for WCU?
How many WCU should there be in the
world?
What are the positions of top Chinese
universities in the world?
How can Chinese universities improve
themselves to reach the goal of WCU?
7
Academic Ranking of World Universities
Our original purpose of doing the Academic
Ranking of World Universities (ARWU) was to
find out the position of Chinese universities in
the world and the gap between them and WCU.
ARWU was put on the internet upon the
encouragement of colleagues from all over the
world. There have been an average of 2000
visitors every day since 2003.
8
Features of ARWU
ARWU uses a few carefully selected, objective criteria and internationally comparable data that everyone could verify in some way.
It has been carried out by a ranking team of four persons in the Institute of Higher Education of Shanghai Jiao Tong University for their academic interests.
It has been done independently without any financial support from any sources outside the Institute of Higher Education.
9
Methodologies & Results
10
Selection of Universities
Any university that has any Nobel Laureates,
Fields Medals, Highly Cited Researchers, or
papers published in Nature or Science.
Major universities of every country with
significant amount of papers indexed by
Thomson.
Number of universities scanned: >2000
Number of universities actually ranked: >1000
Number of ranked universities on our web: 500
11
Ranking Criteria and Weights
Criteria Indicator Code Weight
Quality of Education
Alumni of an institution winning Nobel Prizes and Fields Medals
Alumni 10%
Staff of an institution winning Nobel Prizes and Fields Medals
Award 20%
Quality of Faculty Highly cited researchers in 21 broad subject categories
HiCi 20%
Articles published in Nature and Science* N&S 20%
Research Output
Articles in SCIE, SSCI and AHCI SCI 20%
Size of Institution Academic performance with respect to the size of an institution
Size 10%
Total 100%
For institutions specialized in humanities and social sciences such as London School of Economics, N&S is not considered, and the weight of N&S is relocated to other indicators.
12
Definition of Indicator: Alumni
The total number of the alumni of an institution
winning Nobel Prizes and Fields Medals.
Alumni are defined as those who obtain bachelor,
Master’s or doctoral degrees from the institution.
Different weights are set according to the periods
of obtaining degrees. The weight is 100% for
alumni of 1991-2000, 90% for alumni of 1981-
1990, 80% for alumni of 1971-1980, and so on.
If a person obtains more than one degrees from an
institution, the institution is considered once only.
13
Definition of Indicator: Award
The total number of the staff of an institution winning Nobel prizes in physics, chemistry, medicine and economics and Fields Medal in Mathematics.
Staff is defined as those who work at an institution at the time of winning the prize.
Different weights are set according to the periods of winning the prizes. The weight is 100% for winners since 2001, 90% for winners in 1991-2000, 80% for winners in 1981-1990, 70% for winners in 1971-1980, and so on.
If a winner is affiliated with more than one institution, each institution is assigned the reciprocal of the number of institutions.
For Nobel prizes, if a prize is shared by more than one person, weights are set for winners according to their proportion of prize.
14
Definition of Indicator: HiCi
The number of highly cited researchers in 21
broad subject categories in life sciences,
medicine, physical sciences, engineering and
social sciences.
The definition of categories and detailed
procedures can be found at the website of
Institute of Scientific Information.
The total number of HiCi is about 5000, about
4000 of which is university staff.
15
Definition of Indicator: N&S
The annual average number of articles published
in Nature and Science in the past five years.
To distinguish the order of author affiliation, a
weight of 100% is assigned for corresponding
author, 50% for first author (second author if the
first author is the same as corresponding author),
25% for the next author, and 10% for other
authors.
Only publications of article type are considered.
16
Definition of Indicator: SCI
Total number of articles indexed in Science
Citation Index-expanded (SCIE) and Social
Science Citation Index (SSCI) in the past
year.
A weight of 2 is assigned to articles
indexed in SSCI to compensate the bias
against humanities and social sciences.
Only publications of article type are
considered.
17
Definition of Indicator: Size
The sub-total scores of the above five indicators
divided by the number of full-time equivalent
academic staff.
If the number of academic staff for institutions
of a country cannot be obtained, the total
scores of the above five indicators is used.
For ranking 2005, the number of full-time
equivalent academic staff is obtained for
institutions in USA, China, Australia, Italy,
Netherlands, Sweden, and Belgium etc.
18
Main Sources of Data
Nobel laureates:
http://www.nobel.se Fields Medals:
http://www.mathunion.org/medals/ Highly-cited researchers:
http://www.isihighlycited.com Articles published in Nature and Science:
http://www.isiknowledge.com Articles indexed in SCIE and SSCI:
http://www.isiknowledge.com
19
Results of ARWU
http://ed.sjtu.edu.cn/ranking.htm
Top 500 universities in the world Top 100 universities in North and Latin
America Top 100 universities in Asia/Oceania Top 100 universities in Europe Statistics of top universities by region and country Percentage distribution of top universities by
country as compared with the share of global population and GDP
20
Problems & Discussion
21
Methodological: Education and Service
Education is the basic function of any university,
however, it would be impossible to rank the quality of
education due to the huge differences among the
national systems.
Contribution to the national economic development is
becoming increasingly important for universities,
however, it is impossible to obtain internationally
comparable indicators and data.
The academic or research performance of
universities, a good indication of their reputation, can
be ranked internationally.
22
Methodological: Humanities & Social Sciences
Many well-known institutions specialized in humanities and social sciences are ranked relatively low.
Since 2004, the indicator of N&S is not considered for institutions specialized in humanities and social sciences, its weight is relocated to other indicators.
Since 2005, a weight of 2 for articles indexed by SSCI is considered.
Nevertheless, if a university specialized in social sciences and humanities had Nobel Laureates in economics and Highly Cited Researchers in social sciences, it should have good standing.
23
Methodological: Language Bias
English is the language of international academic community.
Any ranking based on academic performance will be biased towards institutions in English-speaking countries.
One possible solution: papers published in non-native languages are offered a special weight.
Another possible solution: normalization of total articles by the proportion of journal editors of each country.
24
Methodological: Award and Alumni
Universities which started after 1911 do not have a fair chance.
Disciplines not related to the awarding fields do not have a fair chance. Other important awards include Abel, Pulitzer, Turing, Tyler, Pritzker, etc.
Institutions for winning awards and those for doing the researches may not be the same.
Institutions for obtaining degrees and those for pursuing the studies may not be the same.
Postdoctoral training is not considered.
25
Methodological: Per Capita Performance
The weight of the Size indicator for per capita
performance is rather low. Large institutions have
relatively high positions in the ranking.
However, it’s very difficult to obtain internationally
comparable data on the number of academic staff.
The types of academic staff: such as purely teaching
staff, teaching and research staff, purely research staff.
The ranks of academic staff: such as professor, associate
professor, reader, lecturer, research scientist etc.
26
Technical: Attributions
Many universities have more than one commonly used names: such as Virginia Tech and Virginia Polytechnic and State University.
Variations due to translation: such as Univ Koln and Univ Cologne, Univ Vienna and Univ Wien.
Abbreviated names: such as ETH Zurich for Swiss Federal Institute of Technology Zurich.
Some authors only write their departmental or institute names without mentioning their university names.
27
Technical: Definition of Institution
University systems: such as Univ California
system, Univ London system.
Affiliated institutions and research
organizations: such as Ecole Polytechnique
Montreal (affiliated to University of Montreal),
CNRS Labs (affiliated to French universities).
Teaching and affiliated Hospitals: complex!
Our answer: according to author’s expression.
28
Other Technical Problems
Merging, splitting, inheriting, discontinuing, name-changing of institutions such as:
Univ Kwazulu-Natal in South Africa merged from Univ Natal and Univ Durban-Westville.
University of Innsbruck in Austria splitted into Univ Innsbruck and Innsbruck Medical Univ.
Humboldt Univ Berlin and Free Univ Berlin inheriting the Nobel Prizes of the Berlin University before world war II.
29
Ranking by Broad Subject Fields
(ARWU-FIELD)
30
Requests for Ranking of World Universities by
Broad subject fields or schools, colleges
and
Subject fields or programs, departments
In addition, many top Chinese universities
want to learn their positions in the world
by broad subject fields or disciplines.
31
Definition of Broad Subject Fields
Natural Sciences and Mathematics (SCI)
Engineering/Technology and Computer Sciences
(ENG)
Life and Agriculture Sciences (LIFE)
Clinical Medicine and Pharmacy (MED)
Social Sciences (SOC)
Arts and humanities are not ranked
32
ARWU-FIELD Indicators and Weights
Code SCI ENG LIFE MED SOC
Alumni 10% 10% 10% 10%
Award 15% 15% 15% 15%
HiCi 25% 25% 25% 25% 25%
TOP 25% 25% 25% 25% 25%
PUB 25% 25% 25% 25% 25%
Fund 25%
33
Changes in Indicators and Definition
N&S in ARWU is not used in ARWU-FIELD.
TOP is the percentage of articles published in the top 20% journals of each broad subject field.
Fund is the total engineering-related research expenditures. It’s used only for ENG ranking.
Alumni and Award since 1951 are used for all rankings fields except ENG.
PUB is the total number of articles indexed by Thomson in the past year.
34
Results of ARWU-FIELD
http://ed.sjtu.edu.cn/ranking.htm
Top 100 universities in SCI Top 100 universities in ENG Top 100 universities in LIFE Top 100 universities in MED Top 100 universities in SOC Statistics of top universities by region &
country List of top universities by number of top fields
35
Special Problems in ARWU-FIELD
It’s difficult to obtain data on engineering-
related research expenditures and make
them comparable. For 2007, Fund was
obtained only for US and Canadian
universities.
It’s difficult to separate the Nobel
Laureates in Physiology or Medicine. They
are used in both LIFE and MED ranking.
36
Final Remarks
37
Controversy of Ranking
Any ranking is controversial and no
ranking is absolutely objective.
University rankings become popular in
many countries. Whether we agree or
not, ranking systems clearly are here to
stay.
The key issue then becomes how to
improve ranking systems for the
benefits of higher education.
38
Future Efforts
Study all the above mentioned problems and
continuously improve the ranking
methodologies.
Establish more comprehensive databases of
WCU.
Update ARWU (every August) and ARWU-FIELD
annually (every February).
39
Future Efforts (Cont’ed)
Provide ranking of universities specialized
or strong in engineering, medicine, etc.
based on the classification of world
universities.
Provide ranking of universities with
different size, history, budget and function
etc. once internationally comparable data
are obtained.
40
Thank you very much
for your attention!
http://ed.sjtu.edu.cn/ranking.htm
http://ed.sjtu.edu.cn/en/