Latest Developments in Research Performance Evaluation

Yasushi AdachiRegional Manager, Asia PacificElsevier

研究產出 競爭力剖析 洞悉全球趨勢

2

Agenda

•Background - Use of research performance measurement by researchers, administrators and policy makers•Trends in Taiwan’s research output •Application of latest search tools in biomedical research

3

Increasing need to measure research performance

Dr. Kuan-Teh JeangNational Institutes of Health　

(Editor-in-Chief, Retrovirology)

• Impact factor, H index, peer comparisons, and Retrovirology: is it time to individualize citation metrics? 　 (Retrovirology 2007, 4:42)

“…a couple of things are clear about scientists. First, we like functions that are easy to use; and second, we like to be able to compare or rank ourselves against colleagues. Because the world is not a place of unlimited resources and positions, rankings become very important for career development and job promotions.”

4

THES Top 200 Ranking 2007 – Scopus replaces WoS

200

7R

an

k

Co

un

try

16 Australian National University Australia17 University of Tokyo Japan18 University of Hong Kong Hong Kong25 Kyoto University Japan27 University of Melbourne Australia31 University of Sydney Australia33 University of Queensland Australia33 National University of Singapore Singapore36 Peking University China38 Chinese University of Hong Kong Hong Kong40 Tsinghua University China43 Monash University Australia44 University of New South Wales Australia46 Osaka University Japan50 University of Auckland New Zealand51 Seoul National University South Korea

53Hong Kong University of Science & Technology Hong Kong

62 University of Adelaide Australia64 University of Western Australia Australia69 Nanyang Technological University Singapore85 Fudan University China90 Tokyo Institute of Technology Japan

102 National Taiwan University Taiwan102 Tohoku University Japan112 Nagoya University Japan114 University of Otago New Zealand

5

Research Performance Measurement (RPM)

•Grant Allocations• Policy Decisions•Benchmarking• Promotion•Collection management• Funding allocations•Research Researchers

Research　groups

Institution

Country

Policy makersand

Administrators

Policy makersand

Administrators

Researchersat

large

Researchersat

large

6

Taiwan – Research Output

Taiwan - Journal Article Output

20000

22000

24000

26000

28000

30000

32000

34000

36000

38000

40000

2001 2002 2003 2004 2005 2006 2007

Published Year

Nu

mb

er

of

Art

icle

s

Source: http://www.info.scopus.com/researchtrends/index.htm

Global : 5.5%

7

Subject Area Tai

wan

Au

stra

lia

Ger

man

y

Sp

ain

Fra

nce

Gre

ece

Ital

y

Jap

an

New

Zea

lan

d

So

uth

Afr

ica

Multidisciplinary 30.0% 24.4% 15.4% 24.3% 19.4% 62.7% 13.9% 12.0% 6.5% 32.3%Agricultural and Biological Sciences 8.5% 8.3% 7.7% 10.3% 5.8% 10.4% 10.0% 3.5% 7.1% 6.7%Arts and Humanities 28.4% 14.1% 23.6% 6.8% 10.4% 10.7% 10.8% 24.4% 20.0% 0.0%Biochemistry, Genetics and Molecular Biology 18.9% 11.5% 9.8% 13.4% 6.6% 20.0% 9.9% 3.8% 13.7% 13.3%Business, Management and Accounting 30.0% 20.5% 5.6% 25.8% 16.1% 37.7% 11.2% 4.0% 15.9% 11.0%Chemical Engineering 13.3% 5.9% 2.1% 8.3% 5.2% 12.6% 3.4% 6.7% 10.7% 9.9%Chemistry 10.9% 10.7% 5.3% 6.1% 5.4% 9.2% 6.0% 0.1% 7.6% 11.3%Computer Science 22.3% 25.7% 20.9% 28.6% 20.2% 26.1% 16.9% 13.8% 25.9% 25.3%Decision Sciences 9.6% 11.8% 20.9% 18.9% 16.8% 16.2% 18.5% 3.9% 21.3% 20.7%Earth and Planetary Sciences 12.0% 10.4% 8.2% 10.6% 10.4% 12.9% 12.2% 6.7% 7.5% 2.4%Economics, Econometrics and Finance 16.7% 16.8% 17.9% 15.4% 13.2% 14.7% 18.5% 17.7% 15.7% 8.4%Energy 12.6% 6.3% 2.0% 14.5% 9.8% 10.2% 8.2% -0.6% 8.4% 5.2%Engineering 15.8% 10.1% 7.0% 13.9% 9.8% 12.3% 9.6% 10.6% 13.1% 2.0%Environmental Science 13.5% 10.4% 2.9% 14.0% 8.9% 16.3% 12.9% 9.0% 11.1% 12.2%Immunology and Microbiology 14.0% 6.9% 6.1% 7.5% 3.6% 11.0% 5.6% 2.0% 8.8% 16.9%Materials Science 15.7% 9.4% 1.3% 8.1% 5.0% 12.1% 4.8% 0.8% 15.1% 6.7%Mathematics 21.4% 21.1% 15.2% 20.5% 12.5% 24.4% 14.1% 12.9% 19.5% 18.6%Medicine 12.7% 10.0% 5.0% 8.5% 2.8% 16.6% 7.8% 0.7% 13.5% 8.7%Neuroscience 12.1% 12.6% 7.4% 7.3% 1.9% 24.0% 8.2% -0.6% 11.1% 17.3%Nursing 32.5% 19.8% 23.1% 24.5% 34.3% 24.5% 9.3% 6.4% 23.7% 23.0%Pharmacology, Toxicology and Pharmaceutics 9.0% 8.9% 3.7% 4.2% 2.3% 11.2% 5.2% -0.4% 1.9% 9.7%Physics and Astronomy 16.7% 12.4% 6.3% 13.0% 8.5% 12.8% 7.4% 3.7% 18.5% 12.6%Psychology 23.0% 9.0% 5.0% 11.9% 3.8% 17.4% 11.9% 1.2% 10.1% 17.0%Social Sciences 26.5% 12.8% 2.7% 18.0% 9.9% 26.7% 17.7% 10.7% 13.4% 14.6%Veterinary 5.1% 7.9% 4.1% 14.3% -0.7% 21.1% 22.0% 4.8% 4.9% 1.7%Dentistry 10.5% 3.6% 13.9% 14.2% 15.4% 21.9% 15.7% 3.6% 0.7% -3.0%Health Professions 10.4% 10.5% 5.9% 12.2% 5.8% 12.2% 10.0% 4.3% 27.1% 10.5%Total 15.5% 11.2% 6.7% 11.4% 7.0% 15.8% 9.1% 3.9% 11.9% 9.7% Article Output

0.0%

50.0%

100.0%

150.0%

0 1 2 3 4 5 6

Published Year

Nu

mb

er o

f A

rtic

les

Australia

#REF!

#REF!

Australia

#REF!

#REF!

Taiwan - Journal Article Output

20000

25000

30000

35000

40000

0 1 2

Published Year

Nu

mb

er o

f A

rtic

les

Article Output Growth per Subject Area(2002 ~ 2006)

> 10%> 20%

0~10%< 0%

8

Biochemistry, Genetics and Molecular Biology(Output from Taiwan)

0

20

40

60

80

100

120

140

2002 2003 2004 2005 2006

2.0%

2.5%

3.0%

3.5%

4.0%

4.5%

5.0%

Number of articles in top 5%

Percentage of articles in top 5%

percentage

No of articles

9

0

5

10

15

20

25

30

35

40

2002 2003 2004 2005 2006

2.0%

2.5%

3.0%

3.5%

4.0%

4.5%

5.0%

5.5%

Immunology and Microbiology(Output from Taiwan)Number of articles in top 5%

Percentage of articles in top 5%

percentage

No of articles

10

0

2

4

6

8

10

12

14

16

18

2002 2003 2004 2005 2006

1.0%

1.5%

2.0%

2.5%

3.0%

3.5%

4.0%

Neuroscience(Output from Taiwan)Number of articles in top 5%

Percentage of articles in top 5%

percentage

No of articles

11

Summary of trends

•Article output growth in Taiwan is at 16% - three times higher than global increase.•Growth in biomedical sciences is strong in the comparison made in this study.

increased productivity•Percentage of articles published in Taiwan appearing in the top 5% highly cited have been steadily increasing over the years. (for the selected three areas in biomedical science)

improvement in impact

12

What is Scopus?

• 15,000 titles from more than 4,000 publishers• 1,000+ Open Access journals• 500 Conference Proceedings• 400M web pages• 21M patents• Repositories• Digital Archives

WebsitesWebsitesand digital and digital archivesarchives

Peer Peer reviewedreviewed

literatureliterature

ScienceScienceMedicineMedicineTechnologyTechnologySocial sciencesSocial sciences

PatentsPatents

Institutional Institutional repositoriesrepositories

Scopus covers:

13

Example search : Keyword “biochip”

WebsitesWebsitesand digital and digital archivesarchives

Peer Peer reviewedreviewed

literatureliterature

ScienceScienceMedicineMedicineTechnologyTechnologySocial sciencesSocial sciences

PatentsPatents

Institutional Institutional repositoriesrepositories

903articl

es

6,835patents

22,312web pages

Depends on source selected

14

Country comparison (keyword “biochip”)

Taiwan

• 66 articles

• Top 3 journals• Proceedings of SPIE the International Society for Optical Engineering

• Sensors and Actuators B Chemical

• Journal of Micromechanics and Microengineering

• Top 3 prolific authors• Chen, P.H.• Tsai, C.Y.• Lee, G.B.

Japan

• 43 articles

• Top 3 journals• Polymer Preprints Japan• Biomaterials• Japanese Journal of Applied Physics Part 2 Letters

• Top 3 prolific authors• Ito, Y.• Tamiya, E.• Ishihara, K.

15

Highest cited author in Scopus on biochip research

16

Detailed analysis of Dr. Cheng’s output

No of articles

h-index plot

No of citations

17

Summary

• Driven by strong demand, the use of metrics for research performance measurement (RPM) will continue to increase in the future.

• Significant improvements were made during the recent years in terms of bibliometric data retrieval and the invention of new analysis tools.

• Technology will enable performance benchmarking at all levels and effective post processing of results.

• Elsevier will continue to invest in the development of RPM tools for both research administrators and researchers.

18

Thank you.