Deutsche Forschungsgemeinschaft

www.dfg.de DFG

The Use of Research Funding Databases for Research Assessment Information Systems

Presented at the 8th international Conference onCurrent Research Information Systems

May 11-13th, 2006 in Bergen, Norway

Daniel Bovelet, German Research Foundation (DFG)

Department of Information Management

1 Agenda Page 2 of 9

8th international Conference on Current Research Information Systems (2006 in Bergen, Norway) DFG

Agenda

Short Introduction into research assessment systems and the DFG Funding Ranking

Development of Research Assessment Information Systems based on research funding databases

2 Research Assessment Systems and the DFG Funding Ranking Page 3 of 9

8th international Conference on Current Research Information Systems (2006 in Bergen, Norway) DFG

Idea of Research Assessment Systems

History of the DFG Funding Ranking

First Ranking in 1997 DFG funding instruments and programmes

Second Ranking 2000 new: regional aggregation of data

Third Ranking 2003 new: networks in science

new: several other indicators based on heterogeneous research funding databases

Fourth Ranking 2006 => Extended Version: new indicators, detailed analysis of 14 research areas

Systems of research assessment, such as ranking exercises, are internationally discussed as important strategic management tools for publicly funded research institutions and, more than ever, are becoming determinants for research policy decision-making processes. Idea:

To give information on research activity and quality of research institutions

To inform about the disciplinary research profile of these institutions

To inform about regional research clusters and networks in science

The research indicators are primarily derived from process produced databases covering funding processes of research funding agencies

No surveys, no biliometric data – „only“ funding!

New approach of the DFG Funding Ranking

8th international Conference on Current Research Information Systems (2006 in Bergen, Norway) DFG

3 Developement of a CRIS based on the DFG Funding Ranking Page 4 of 9

Research Funding Databases (DFG funding & reviewers, AvH, DAAD, Bund, EU, StatBund)

The different properties of funding databases can be used for the derivation of research indicators suitable for the representation of the research activities and achievements of

research institutions differentiated by several units of analysis (e.g. scientific disciplines)

Research Assessment Information System covering 14 research areas

research indicators

Institutional Reports - Geographical Information Systems - Network Analytical Representations

integration

consolidation of heterogeneous databases

unique framework for the subject classification concordance/classifications for institutional codes

standardization process

derivation of indicators

STANDARDIZATION

HETEROGENEOUS

RESEARCH FUNDING DATABASES

OtherResearch Councils

Government FoundationsFederal

StatisticalOffice

researchfunding and

infrastructuraldata

Businessand

Industry

research areaclassifications

institutionalclassifications

InternationalOrganisations

researcharea

researchinstitution

8th international Conference on Current Research Information Systems (2006 in Bergen, Norway) DFG

3.1 Data sources and the need of standardization processes Page 5 of 9

DFG

8th international Conference on Current Research Information Systems (2006 in Bergen, Norway) DFG

3.2 Institutional reports Page 6 of 9

Characteristics

Enable comprehensive information on the parameter values of the research indicators

Several indicators can be examined in an overall context for each research institution

For the representation and interpretation of the respective values, the relation to a specific benchmark (e.g. by using ranking groups) should be taken into consideration

The information system is additionally able to provide qualitative information on the institution and its infrastructure (e.g. titles of funded research programmes)

Example for institutional reports

Figure on the University of Munich

The report offers information on three goups of research indicators: financial capital structure, the human resources and indicators suituable for the representation of the internationality

The research indicators can be differentiated according to scientific discipline or research funding programmes...

University of Munich

Regular Expenditure (= total)

Administrative income

Third party funding income Regular core funds

Financial resources (in mio. €)

1947.5

(100%)

919.7

(47.2%)

212.7

(10.9%)

815.1

(41.9%)

PrizesTotalIndividual Grants Programme

Coordinated Programme

Direct Promotion of Young Researchers

TotalHumanities and Social Sciences Life Sciences Natural Sciences Engineering Sciences

116.9

(100%)40.0(34.2%)

67.6

(57.8)

5.7

(4.9%)

3.6

(3.1%)

DFG approvals

(in mio. €)

Financial Data

Total Humanities and Social Sciences

Life Sciences Natural Sciences Engineering Sciences

Scientists (Total)

5129 (100%)

1270

(24.8%)

3006

(58.6%)

790

(15.4%)

48

(0.9%)

Professors710

(100%)306(43.1%)

274

(38.6%)

122

(17.2%)7(1%)

DFG

reviewers

309

(100%)

110

(35.6%)

148

(47.9%)

47

(15.2%)

4

(1.3%)

Personnel Data

Total Humanities and Social Sciences

Life Sciences Natural Sciences

Internationality

Engineering Sciences

DAAD scientists& academics

103

(100%)

70

(68.0%)

22

(21.4%)

11

(10.7%)

-

(0%)

AvH research fellows & award winners

115

(100%)

44

(38.3%)

24

(20.9%)

45

(39.1%)

2

(1.7%)

DFG approvals

(in mio. €)

116.9

(100%)

22.9

(19.6%)

70.0

(59.9%)

21.7

(18.6%)

2.3

(2.0%)

University of Munich

Regular Expenditure (= total)

Administrative income

Third party funding income Regular core funds

Financial resources (in mio. €)

1947.5

(100%)

919.7

(47.2%)

212.7

(10.9%)

815.1

(41.9%)

PrizesTotalIndividual Grants Programme

Coordinated Programme

Direct Promotion of Young Researchers

TotalHumanities and Social Sciences Life Sciences Natural Sciences Engineering Sciences

116.9

(100%)40.0(34.2%)

67.6

(57.8)

5.7

(4.9%)

3.6

(3.1%)

DFG approvals

(in mio. €)

Financial Data

Total Humanities and Social Sciences

Life Sciences Natural Sciences Engineering Sciences

Scientists (Total)

5129 (100%)

1270

(24.8%)

3006

(58.6%)

790

(15.4%)

48

(0.9%)

Professors710

(100%)306(43.1%)

274

(38.6%)

122

(17.2%)7(1%)

DFG

reviewers

309

(100%)

110

(35.6%)

148

(47.9%)

47

(15.2%)

4

(1.3%)

Personnel Data

Total Humanities and Social Sciences

Life Sciences Natural Sciences

Internationality

Engineering Sciences

DAAD scientists& academics

103

(100%)

70

(68.0%)

22

(21.4%)

11

(10.7%)

-

(0%)

AvH research fellows & award winners

115

(100%)

44

(38.3%)

24

(20.9%)

45

(39.1%)

2

(1.7%)

DFG approvals

(in mio. €)

116.9

(100%)

22.9

(19.6%)

70.0

(59.9%)

21.7

(18.6%)

2.3

(2.0%)

8th international Conference on Current Research Information Systems (2006 in Bergen, Norway) DFG

3.3 Geographic Information Systems Page 7 of 9

Characteristics

Using a GIS in the context of several types of research indicators provides perspectives on the various research institutions and surrounding regions with their scientific infrastructure

The figures provide insight into the research regions and enable regional focal points to be identified

Several levels of analysis can be displayed for different indicators, such as an overall view for Germany, distributions for the federal states, or allocation by administrative districts

Example for cartographical representations

Figure on allocations of DFG approvals

The data is processed in such a way that conclusions are drawn according to administrative rural and urban districts

A district code is assigned to each institution implemented in the research institution directory and standardized (address) database

For institutions located at multiple locations, each site is allocated ist own code

Humanities and Social SciencesLife SciencesNatural SciencesEngineering Sciences

Scientific DisciplineHumanities and Social SciencesLife SciencesNatural SciencesEngineering Sciences

Scientific Discipline

8th international Conference on Current Research Information Systems (2006 in Bergen, Norway) DFG

3.4 Network Analytical Representations Page 8 of 9

UniversityHelmholtz Association (HGF)Leibniz Association (WGL)Max Planck Society (MPG)Fraunhofer Society (FhG)Others

threshold value: joint participation in at least three research network programmes

UniversityHelmholtz Association (HGF)Leibniz Association (WGL)Max Planck Society (MPG)Fraunhofer Society (FhG)Others

threshold value: joint participation in at least three research network programmes

Characteristics

The network or relationship structures between research institutions can be visualized in the form of network analytical representations

The graphs can be analysed in the following way

institutions that are central to the structure are positioned centrally

institutions that interact frequently are positioned close to each other

the line thickness indicates the frequency of interaction between two institutions

the diameter of circles represents the total number of interactions

Example for Network Analytical Representations

Figure on DFG coordinated programmes

The relationship structure results from joint participation of institutions in DFG coordinated programmes

The figure shows the core network in biology

Thank you for your attention!

More Infos at www.dfg.de/

Daniel Bovelet, Department of Information ManagementDaniel.Bovelet@dfg.de

Deutsche Forschungsgemeinschaft

www.dfg.de DFG