an approach to interdisciplinarity through bibliometric indicators

20
Jointly published by Kluwer Academic Publishers, Dordrecht Scientometrics, and Akadémiai Kiadó, Budapest Vol. 51, No. 1 (2001) 203–222 An approach to interdisciplinarity through bibliometric indicators FERNANDA MORILLO, MARÍA BORDONS, ISABEL GÓMEZ Centro de Información y Documentación Científica (CINDOC) Consejo Superior de Investigaciones Científicas (CSIC), Madrid (Spain) Interdisciplinarity has become of increasing interest in science in the past few years. This paper is a case study in the area of Chemistry, in which a series of different bibliometric indicators for measuring interdisciplinarity are presented. The following indicators are analysed: a) ISI multi- classification of journals in categories, b) patterns of citations and references outside category and c) multi-assignation of documents in Chemical Abstracts sections. Convergence between the different indicators is studied. Depending on the size of the unit analysed (area, category or journal) the most appropriate indicators are determined. Introduction The importance of interdisciplinary research is widely accepted nowadays, and cross-disciplinary initiatives are promoted through national and international research programmes. However, we do not have appropriate indicators to measure interdisciplinarity. These type of indicators could allow us to compare research areas or countries according to their interdisciplinary behaviour, to track discipline changes over time, and to detect highly interdisciplinary emerging new areas. In bibliometrics, links between research areas or topics have been analysed through the “maps of science”, especially those based on co-word analyses (Tijssen, 1992), or through the flow of citations among fields (Porter and Chubin, 1985; Urata, 1990; Cronin and Pearson, 1990; Tomov and Mutafov, 1996). Interdisciplinarity in publications was also analysed through the collaboration between researchers from different disciplines (Qiu, 1992), whilst collaboration itself was found to be associated to higher interdisciplinarity in some disciplines (Qin et al, 1997). An interesting classification scheme of journals into categories, derived from the ISI classification, but modified to identify groups of interdisciplinary journals, was produced by Katz and Hicks (1995). Their classification scheme distinguished different levels of cross- disciplinary journals: inter-field journals, inter-disciplinary journals and multi- 0138–9130/2001/US $ 15.00 Copyright © 2001 Akadémiai Kiadó, Budapest All rights reserved

Upload: fernanda-morillo

Post on 02-Aug-2016

214 views

Category:

Documents


2 download

TRANSCRIPT

Jointly published by Kluwer Academic Publishers, Dordrecht Scientometrics,

and Akadémiai Kiadó, Budapest Vol. 51, No. 1 (2001) 203–222

An approach to interdisciplinarity throughbibliometric indicators

FERNANDA MORILLO, MARÍA BORDONS, ISABEL GÓMEZ

Centro de Información y Documentación Científica (CINDOC) Consejo Superiorde Investigaciones Científicas (CSIC), Madrid (Spain)

Interdisciplinarity has become of increasing interest in science in the past few years. Thispaper is a case study in the area of Chemistry, in which a series of different bibliometric indicatorsfor measuring interdisciplinarity are presented. The following indicators are analysed: a) ISI multi-classification of journals in categories, b) patterns of citations and references outside category andc) multi-assignation of documents in Chemical Abstracts sections. Convergence between thedifferent indicators is studied. Depending on the size of the unit analysed (area, category orjournal) the most appropriate indicators are determined.

Introduction

The importance of interdisciplinary research is widely accepted nowadays, andcross-disciplinary initiatives are promoted through national and international researchprogrammes. However, we do not have appropriate indicators to measureinterdisciplinarity. These type of indicators could allow us to compare research areas orcountries according to their interdisciplinary behaviour, to track discipline changes overtime, and to detect highly interdisciplinary emerging new areas.

In bibliometrics, links between research areas or topics have been analysed throughthe “maps of science”, especially those based on co-word analyses (Tijssen, 1992), orthrough the flow of citations among fields (Porter and Chubin, 1985; Urata, 1990;Cronin and Pearson, 1990; Tomov and Mutafov, 1996). Interdisciplinarity inpublications was also analysed through the collaboration between researchers fromdifferent disciplines (Qiu, 1992), whilst collaboration itself was found to be associatedto higher interdisciplinarity in some disciplines (Qin et al, 1997). An interestingclassification scheme of journals into categories, derived from the ISI classification, butmodified to identify groups of interdisciplinary journals, was produced by Katz andHicks (1995). Their classification scheme distinguished different levels of cross-disciplinary journals: inter-field journals, inter-disciplinary journals and multi-

0138–9130/2001/US $ 15.00Copyright © 2001 Akadémiai Kiadó, BudapestAll rights reserved

F. MORILLO et al.: Interdisciplinarity through bibliometric indicators

disciplinary journals, based on the ISI multi-assignation of journals into differentcategories which are increasingly distant. Their approach was especially useful forproviding a general overview of the activity of a given country in all the sciences, asthey did for the UK. However, they did not give evidence of the higherinterdisciplinarity of the multi-assigned journals vs. the single-assigned ones.

This study presents several bibliometric indicators to measure interdisciplinarity.Some of these have been used by other authors previously, however, this timeconvergence between the indicators as well as their usefulness at different levels ofaggregation are analysed. From a conceptual point of view, differences between“interdisciplinarity” and “multidisciplinarity” can be noted. Strictly speaking, weconsider “multidisciplinarity” as a basic situation in which elements from differentdisciplines are present, whilst “interdisciplinarity” is a more advanced stage of therelationship between disciplines in which integration between them is attained. It isfrequently difficult to distinguish between the two, so the term “interdisciplinarity” istherefore used in a broad sense. The exception is when we refer to journals.“Multidisciplinary journals” contain documents from different disciplines, whereasdocuments themselves may be single-disciplinary. As regards “interdisciplinaryjournals”, interdisciplinarity is to be expected in each of their documents.

The following questions are raised in this study:• Are the different indicators convergent? Which is the most sensitive one?• Is interdisciplinarity a journal-dependent or a field-dependent feature?• Can we characterise disciplines and journals according to their interdisciplinary

behaviour through these indicators?• Which are the most appropriate indicators for each level of aggregation?

A case study is presented in the area of Chemistry. Indicators based on both amultidisciplinary and a specialised bibliographic database are introduced and tested.

Methods

This study introduces several interdisciplinarity indicators and assesses their validityin two specific research fields: Applied Chemistry (ACHEM), selected as a sample of ahighly interdisciplinary category, and Polymer Science (POLYM), as a lowinterdisciplinary one. Indicators are based on data provided by two different databaseproducers: Institute for Scientific Information (ISI), and Chemical Abstracts Service (CA).

204 Scientometrics 51 (2001)

F. MORILLO et al.: Interdisciplinarity through bibliometric indicators

The indicators proposed should be able to detect links between disciplines or subjectcategories, this being the main unit of analysis in this study. In the following lines theindicators used are described.

ISI based indicators

Disciplines are delimited according to subject categories as defined by the ISI in itsSciSearch database, in which journals are classified into approximately 225 categories.Most of the journals are included in a single category, but assignation of journals intomore than one category is also allowed. Because of the multi-assignation of journals,interesting links between categories emerge, which are the basis of the indicators hereproposed. Our basic assumptions are that multi-assigned journals have a moreinterdisciplinary nature than single-assigned ones and, secondly, that a higherpercentage of multi-assigned journals is present in the most interdisciplinary categories.

In this study, the 225 categories are aggregated into nine different research areas:Agriculture/Environment, Arts & Humanities, Clinical Medicine, Chemistry,Engineering/Technology, Life Sciences, Mathematics, Physics and Social Sciences. Themulti-assignation of journals into different categories in the same area is called “internalrelationship”, and it establishes a connection between close categories (e.g., AnalyticalChemistry and Organic Chemistry, both from the area of Chemistry). On the other hand,an “external relationship” is set up by the multi-assignation of journals into categories ofdifferent areas, and this type of relation links more distant categories (e.g.,Pharmacology and Applied Chemistry, which belong to the areas of Life Sciences andChemistry respectively).

In this context, every subject category can be described through the followingindicators:• Percentage of multi-assigned journals (% Mult): percentage of journals classified in

the category analysed and in any other SciSearch category at the same time.• Pattern of multi-assignation: the multi-assignation pattern of a given category is

described through the percentage of journals that are classified in another categoryinside the area (internal multi-assignation, %I), outside the area (external multi-assignation, %O) and both inside and outside the area (%IO).

Chemical Abstracts based indicators

This very comprehensive Chemistry database includes keywords in all documentsand classifies them in at least one of the 80 sections in which the CA divides the area

Scientometrics 51 (2001) 205

F. MORILLO et al.: Interdisciplinarity through bibliometric indicators

(CA, 2000). Every document is assigned by professional indexers to one main sectionand optionally to several secondary sections that are called section cross-reference(s). Inthis work, multi-assignation of a given document in different sections is considered as ameasure of interdisciplinarity. The analysis is carried out at the document level but dataare aggregated into journals or disciplines.

Starting from the SciSearch classification of journals into subfields, thirty-threeACHEM journals and sixty-one POLYM journals covered simultaneously by SciSearchand Chemical Abstracts in 1996 were studied. Documents published in these journalsduring 1996 were downloaded and the links between disciplines were studied accordingto the different CA subject sections. The following indicators were calculated for eachjournal:• Percentage of documents with only the main section and no secondary sections, by

journal (p1sec).• Average number of sections per document, by journal (mnsec).• Percentage of different sections in each journal. Two different indicators were

calculated: either considering only main sections (psec) or also including secondarysections (pcross).

• Concentration/scattering of documents over sections measured through the Pratt Index(prattcross) (Pratt, 1977). Both main and secondary sections were considered. This indexranges from 0 (no concentration at all) to 1 (the highest concentration).

Journal Citation Reports (JCR) based indicators

Citation and reference patterns of journals were analysed according to the dataprovided by the JCR in 1996 (ISI, 1997). The “Citing journal listing” provides for everysource journal the list of journals it has cited (references) during the year analysed. Weregistered the disciplines of each of the cited journals, following the ISI classification ofjournals into categories. The “Cited journal listing”, which provides the list of journalsthat have cited a particular one (citations), was analysed in the same way.

This study focuses on a sample of 9 ACHEM and 9 POLYM journals. The followingindicators were calculated for each journal:• Percentages of citations outside category (pco) and references outside category

(pro). This indicator had been used previously by Porter and Chubin (1985), as ameasure of cross-disciplinarity among scientific journals. The subject category ofall the journals cited by a given journal or citing it are considered, and thepercentages of references/citations from categories different to that of the analysedjournal are calculated. Citations and references are classified as belonging to “the

206 Scientometrics 51 (2001)

F. MORILLO et al.: Interdisciplinarity through bibliometric indicators

same” or to a “different” category of that of the analysed journal. In the case ofmulti-assigned journals which belong to ACHEM or POLYM and also to othersubject categories, citations and references are considered as belonging only to thesame category as the analysed journal. Interdisciplinary journals are expected toshow a high weight of references and citations from categories different to that ofthe analysed journal, that is, a high percentage of citations and references outside itscategory.

• Concentration/scattering of citations/references over categories was measuredthrough the Pratt Index (prattc, prattr). Interdisciplinary journals are expected toshow a large variety of disciplines in their citation/reference patterns, that is, lowconcentration of citations/references as measured by the concentration Pratt Index.

Convergence between the different interdisciplinarity indicators proposed is studied,as well as their usefulness in describing disciplines and journals. The SPSS package isused for the statistical analysis of data. Differences between means are analysed throughtests for non-parametric samples.

Results

ISI based indicators

POLYM and ACHEM are two of the eight categories included in the area ofChemistry. The main features of the eight categories are shown in Table 1. The size ofthe categories, measured by the number of journals within each category, and the totalpercentage of multi-assignation are displayed. It is interesting to note the heterogeneityof the area in relation to the behaviour of its disciplines. The area of Chemistry as awhole shows 43% of multi-assignation, but differences among disciplines are observed:categories with a high multi-assignation percentage, such as Applied Chemistry, coexistwith low multi-assigned categories, such as Organic Chemistry and Polymer Science.

The last three columns of Table 1 show the multi-assignation pattern of thecategories. The percentage of external multi-assignation is usually higher than theinternal one. Only Inorganic & Nuclear Chemistry and Organic Chemistry show higherinternal multi-assignation. According to the multi-assignation patterns, amultidimensional map of categories is shown in Figure 1. Multidimensional scalingtechniques, using euclidean distance and ratio measures, were used. Dimension 1towards the right represents a high relation between categories of the same area (%I),and towards the left it represents high relation with categories of other areas (%O).Dimension 2 is related to higher single or multi-assignation percentages.

Scientometrics 51 (2001) 207

F. MORILLO et al.: Interdisciplinarity through bibliometric indicators

Table 1Chemistry area: number of journals, percentage of multi-assignation

and multi-assignation pattern of the different chemical categories

No.Journals

% Mult. % I % O % IO

Applied Chemistry 36 83.3 13.9 55.6 13.9

Physical Chemistry 83 53.0 10.8 34.9 7.2

Inorg. & Nucl. Chemistry 40 52.5 27.5 17.5 7.5

Analytical Chemistry 56 51.8 5.4 37.5 8.9

Electrochemistry 12 50.0 16.7 25.0 8.3

Polymer Science 69 39.1 2.9 33.3 2.9

Chemistry 127 38.6 1.6 35.4 1.6

Organic Chemistry 42 38.1 21.4 11.9 4.8

%Mult.= percentage of multi-assigned journals%I = percentage of multi-assignation inside area only%O = percentage of multi-assignation outside area only%IO = percentage of multi-assignation inside and outside area

Figure 1. Multidimensional map of Chemistry categoriesAbbreviations: Anal.Ch.= Analytical Chemistry; Appl.Ch.= Applied Chemistry;Chem.= Chemistry; Elec.= Electrochemistry; Inorg.Ch.= Inorganic & Nuclear Chemistry;Org.Ch.= Organic Chemistry; Phys.Ch.= Physical Chemistry; Pol.Sci.= Polymer Science.

208 Scientometrics 51 (2001)

F. MORILLO et al.: Interdisciplinarity through bibliometric indicators

In the Chemistry area, Polymer Science (POLYM) shows a low percentage of multi-assigned journals, while Applied Chemistry (ACHEM) is at the opposite extreme, with ahigh percentage of multi-assigned journals. ACHEM is highly related to other non-Chemistry areas, as shown by its high percentage of external multi-assignation, which isthe highest one in the chemical area (55.6% vs. 33.3% in POLYM). POLYM is mostlyrelated to the Engineering area, to which some 60% of its externally multi-assignedjournals belong (especially to Chemical Engineering and Materials Science). ACHEM isless related to the areas of Engineering (30% of its externally multi-assigned journals,especially to the categories of Chemical Engineering and Materials Science), and itshows a higher link to Agriculture (Food Science, Agriculture), Clinical Medicine andLife Sciences (Pharmacology) (Figure 2).

In summary, ACHEM showed higher interdisciplinarity than POLYM, as supportedby its higher multi-assignation rate, higher external multi-assignation and higher varietyof related categories.

Figure 2. Distribution of related categories by areas

Chemical Abstracts based indicators

The distribution of documents published by 61 POLYM and 33 ACHEM journalsaccording to the number of sections is shown in Figure 3. Around 66% of ACHEMdocuments and 77% of POLYM documents were classified only in one section. Thehigher number of sections per document in ACHEM points towards the higherinterdisciplinary links of Applied Chemistry as a whole versus Polymer Science.

Scientometrics 51 (2001) 209

F. MORILLO et al.: Interdisciplinarity through bibliometric indicators

In Table 2 the same trend is shown by the following CA based indicators calculatedat the journal level: percentage of documents with one section (p1sec = 66.2% inACHEM vs. 78.6% in POLYM); average number of sections per document (mnsec =1.5 in ACHEM vs. 1.3 in POLYM); percentage of different main sections (psec =19.1% in ACHEM vs. 12.5% in POLYM) and percentage of different main andsecondary sections (pcross = 30% in ACHEM vs. 20.7% in POLYM). Polymer Scienceshowed a higher percentage of one-section documents, a lower number of sections perdocument, and a lower percentage of main and secondary sections than ACHEM, datathat support the higher interdisciplinarity of the latter category. Moreover, theconcentration of documents by sections was also higher for POLYM than for ACHEM(prattcross = 0.672 for ACHEM vs 0.775 for POLYM).

Figure 3. Distribution of documents of Polymer Science and Applied Chemistry by CA main andsecondary sections

210 Scientometrics 51 (2001)

F. MORILLO et al.: Interdisciplinarity through bibliometric indicators

Table 2Description of Polymer Scienceand Applied Chemistry according to CA indicators

Polymer Science(n = 61)

Applied Chemistry(n = 33)

Kruskal-Wallis Test

χ2 Sig

p1sec (%) 78.6±13.5(36.8 - 100.0)

M = 80.0

66.2±19.6(10.7 - 95.0)

M = 71.8

8.5 0.004*

mnsec 1.3±0.2(1.0 - 1.9)M = 1.2

1.5±0.4(1.1 - 3.0)M = 1.4

11.8 0.001*

psec (%) 12.5±8.6(1.7 - 42.1)

M = 9.7

19.1±15.0(2.9 - 61.7)M = 16.7

3.9 0.048*

pcross (%) 20.7±13.4(3.0 - 63.2)M = 17.2

30.0±18.2(5.6 - 80.9)M = 29.8

7.0 0.008*

prattcross 0.775±0.113(0.400 - 0.967)

M = 0.767

0.672±0.153(0.310 - 0.887)

M = 0.721

10.3 0.001*

Data expressed as: Mean ± Standard Deviation; (Minimum - Maximum); M= Median;* Significant differences (p < 0.05)Note: p1sec=percentage of one-section documents; mnsec= average number of sections perdocument; psec=percentage of different main sections per journal; pcross=percentage of differentmain and secondary sections per journal; prattcross= concentration of documents over sections,measured through the Pratt index.

Journal Citation Reports based indicators

The analysis of citations and references outside category through JCR is presented inTable 3. These data support the higher interdisciplinarity of ACHEM as compared toPOLYM. Applied Chemistry shows a higher percentage of citations outside category(65.0% in ACHEM vs. 40.9% in POLYM); a higher percentage of references outsidecategory (70.9% in ACHEM vs. 54.5% in POLYM); and a lower Pratt Index ofreferences (0.678 in ACHEM vs. 0.753 in POLYM). However, the differences betweenboth categories were statistically significant only for the percentage of citations outsidecategory, possibly due to the small size of the sample used.

Scientometrics 51 (2001) 211

F. MORILLO et al.: Interdisciplinarity through bibliometric indicators

Table 3Description of Polymer Science and Applied Chemistry according to JCR citation based indicators

Polymer Science

(n = 9)

Applied Chemistry

(n = 9)

Kruskal-Wallis Test

χ2 Sig.

pco (%) 40.9±21.6(16.4 - 73.8)

M = 40.6

65.0±18.6(30.9 - 94.4)

M = 61.2

5.5 0.019*

pro (%) 54.5±24.1(28.9 - 91.8)

M = 46.1

70.9±17.1(49.8 - 95.4)

M = 73.0

2.1 NS

prattc 0.724±0.154(0.425 - 0.862)

M = 0.748

0.734±0.089(0.539 - 0.840)

M = 0.758

0.1 NS

prattr 0.753±0.071(0.648 - 0.835)

M = 0.757

0.678±0.133(0.439 - 0.835)

M = 0.738

0.9 NS

Data expressed as:Mean ± Standard Deviation; (Minimum - Maximum); M=Median;* Significant differences (p < 0.05)Note: pco=percentage of citations outside category; pro=percentage of references outside category;prattc=Pratt index for the concentration of citations over categories; prattr=Pratt index for theconcentration of references over categories.

Do multi-assigned journals show higher interdisciplinarity

than single-assigned journals?

Our basic assumption was a higher interdisciplinarity for the multi-assigned journalsthan for the single-assigned ones. To check the validity and convergence of the differentindicators, the sensitivity of CA and JCR based indicators to separate single vs. multi-assigned journals was assessed. The average values of JCR-based indicators for one-category, two-category and three-category journals are shown in Figure 4. In POLYM,multi-assigned journals showed higher interdisciplinarity as measured through JCR thansingle-assigned journals and interdisciplinarity tended to increase with the number ofcategories as expected. However, this was not the case for ACHEM journals.Surprisingly, ACHEM single-classified journals showed higher interdisciplinarity thanits multi-assigned journals.

212 Scientometrics 51 (2001)

F. MORILLO et al.: Interdisciplinarity through bibliometric indicators

Polymer Science

Figure 4a. JCR citation based indicators according to ISI multi-assignation of journals into categoriesNote: pro= percentage of citations outside category; pro=percentage of references outsidecategory; prattc=Pratt index for the concentration of citation over categories;prattr=Pratt index for the concentration of references over categories.

Scientometrics 51 (2001) 213

F. MORILLO et al.: Interdisciplinarity through bibliometric indicators

Applied Chemistry

Figure 4b. JCR citation based indicators according to ISI multi-assignation of journals into categoriesNote: pro= percentage of citations outside category; pro=percentage of references outsidecategory; prattc=Pratt index for the concentration of citation over categories;prattr=Pratt index for the concentration of references over categories.

Data concerning interdisciplinarity as measured through CA indicators are shown inTable 4. In POLYM, three-category journals showed the highest values of differentsections, both main and secondary, and this group of journals also displayed the highestscattering of documents by sections, as expected. On the other hand, there were nosignificant differences between one-category and two-category journals.

214 Scientometrics 51 (2001)

F. MORILLO et al.: Interdisciplinarity through bibliometric indicators

Table 4CA based indicators according to ISI multi-assignation of journals into categories

CA basedPolymer Science Applied Chemistry

indicators

No.Categories No.Categories

1

(n = 37)

2

(n = 18)

3

(n = 6)

1

(n = 3)

2

(n = 18)

3

(n = 12)

psec (%) 11.9±8.0M = 9.7

12.2±8.3M = 8.9

16.9±13.4M = 11.4

45.8±24.9M = 58.6

20.0±11.9M = 18.4

11.2±7.7M = 9.3

pcross (%) 21.1±13.3M = 18.5

17.8±11.2M = 13.9

26.7±19.4M = 20.4

61.9±28.6M = 75.9

31.4±13.9M = 32.0

19.8±10.9M = 17.8

prattcross 0.788±0.097M = 0.767

0.771±0.136M = 0.817

0.706±0.124M = 0.758

0.480±0.209M = 0.372

0.657±0.149M = 0.694

0.743±0.101M = 0.754

Data expressed as: Mean ± Standard Deviation; M= Median.Note: psec= percentage of different main sections per journal; pcross= percentage of different main andsecondary sections per journal; prattcross= Pratt index for the concentration of documents over sections.

In the case of ACHEM, the results obtained from CA indicators were clearlyinconsistent with those based on ISI multi-assignation of journals. In fact, one-categoryjournals showed the highest percentage of different sections, both main and secondaryand the highest scattering of documents over sections as had been observed with JCRbased indicators (Figure 4b).

Interdisciplinarity at the journal level

Within each category, variability of the indicators proposed was analysed to assessindicator usefulness in measuring interdisciplinarity of journals, and with the final aimof obtaining a typology of journals. Indicators derived from a specialised databaseclassification (CA) were considered as the most adequate for the analysis of journals.Specifically, the percentage of one-section documents and the percentage of differentsections by journal were identified as the most sensitive indicators to classify journals.

Scientometrics 51 (2001) 215

F. MORILLO et al.: Interdisciplinarity through bibliometric indicators

Figure 5. Journals of Polymer Science vs. Applied Chemistry as seen through CA databaseNote: pcross= percentage of different main and secondary sections per journal; p1sec= percentageof one-section documents. The lower case letters (a-f) indicate the different clusters of journals.

216 Scientometrics 51 (2001)

F. MORILLO et al.: Interdisciplinarity through bibliometric indicators

Table 5Description of CA indicators in Polymer Science and Applied Chemistry hierarchical clusters

Cluster(number ofjournals)

pcross p1sec

a (16)22.1±8.0

(11.3 - 36.5)M = 19.0

67.3±5.3(55.6 - 74.5)

M = 68.5

b (1) 9.7 37.9

Polymerc (25)

9.7±4.1(3.0 - 17.1)

M = 9.3

87.3±7.4(74.7 - 100.0)

M = 85.7

Scienced (6)

45.7±4.3(40.0 - 50.0)

M = 45.7

71.8±7.0(60.7 - 80.0)

M = 71.0

e (1) 63.2 36.8

f (12)26.4±3.5

(22.2 - 33.9)M = 25.3

85.7±8.4(74.5 - 100.0)

M = 86.1

a (12)37.0±8.9

(17.9 - 51.3)M = 35.1

54.9±10.9(39.3 - 71.8)

M = 53.7

b (1) 13.6 22.3

AppliedChemistry c (17)

18.6±8.7(5.6 - 33.3)M = 17.5

80.3±6.7(68.0 - 95.0)

M = 81.1

d (2)78.4±3.5

(75.9 - 80.9)M = 78.4

64.2±17.6(51.7 - 76.6)

M = 64.2

e (1) 57.1 10.7

Note: pcross= percentage of different main and secondary sections per journal; p1sec= percentage ofone-section documents.

Figure 5 shows POLYM and ACHEM journals according to both indicators andgrouped in different clusters, which were determined through hierarchical clusteranalysis. In Figure 5, the different clusters are identified by means of lower case letters(from “a” to “f” in POLYM and from “a” to “e” in ACHEM). The horizontal andvertical lines, which correspond to the mean values of each variable, enable us to

Scientometrics 51 (2001) 217

F. MORILLO et al.: Interdisciplinarity through bibliometric indicators

identify which clusters are above or below the average values in both categories.According to the two indicators, the following typology of journals was obtained:specialised, multidisciplinary or interdisciplinary journals.

The clusters of specialised journals (covering a reduced number of topics) arelocated at the bottom of the figure, whilst multidisciplinary journals (different topicsoriginating from different documents) are at the top right, and interdisciplinary journals(a majority of documents dealing with multiple topics) are at the top left. Description offinal clusters is shown in Table 5. The same pattern is observed in both disciplines, but amuch higher specialisation is detected for POLYM, because of its lower pcross averageand higher p1sec average, and because its most interdisciplinary journal shows apercentage of different sections slightly over 60%, while in ACHEM this value rises to80%.

Discussion

The first problem every study on interdisciplinarity has to face is to determine whatis to be considered as interdisciplinary or cross-disciplinary research. In our study,multi-assignation of journals into categories, multi-assignation of documents into CAsections, and distribution of citations/references over disciplines were theinterdisciplinary indicators used.

At the level of disciplines, all the indicators used in this study were convergent:ACHEM appears as a more interdisciplinary category than POLYM. Thus, ACHEMshowed a higher multi-assignation rate in ISI categories than POLYM (83% vs. 39%), ahigher percentage of citations outside category (65% vs. 41%), a higher percentage ofreferences outside category (71% vs.54%), a higher percentage of documents with morethan one CA section (34% vs. 21%), a higher average number of sections per document(1.5 vs.1.3) and a higher percentage of different sections, both main and secondarysections (19% vs.12% and 30% vs.21%, respectively). In relation to JCR indicators, theusefulness of the percentage of citations or references outside category was firstlypointed out by Porter and Chubin (1985). Their study showed differences between thethree categories they analysed: Toxicology was more interdisciplinary than Demographyand Management Science. However, chemical journals were not analysed.

Although both CA and JCR based indicators showed a higher interdisciplinarity forACHEM than for POLYM, the most significant differences between the two categorieswere found through CA indicators. The latter indicators, based on a specialiseddatabase, emerged as being more sensitive to detect inter-category differences.

218 Scientometrics 51 (2001)

F. MORILLO et al.: Interdisciplinarity through bibliometric indicators

However, it should be noted that the small size of the sample of journals analysedthrough JCR (9 journals in each category) can also explain why the differences were notstatistically significant in some of the cases.

Within a given category, the sensitivity of CA and JCR indicators to discriminatebetween one-ISI-category and multi-ISI-category journals was assessed. In POLYM,one-category journals were less interdisciplinary, according to CA and JCR indicators,than two and three category journals, as was expected. Specifically, JCR-basedindicators were the most useful in separating one category POLYM journals fromseveral category journals: the higher the number of categories a given journal wasassigned to, the higher the percentage of citations outside category it received. In fact,this relationship is not surprising, since for the classification of journals into categoriesISI considers not only the title or main topic of journals, but also cluster analysis ofciting/cited journal patterns. Thus, if a journal assigned to one category showed a largepercentage of references/citations to a second category, ISI would probably assign it tothis second category.

However, this relationship did not apply to ACHEM. Surprisingly, one-categoryACHEM journals showed higher interdisciplinarity, as measured by CA and JCRindicators, than multi-assigned journals. One possible explanation can be found in thedifferent nature of both categories: POLYM is a well-delimited and specialiseddiscipline, while ACHEM is a typical “horizontal discipline”. Interdisciplinarity isinherent to the ACHEM category. According to the ISI definitions of categories,ACHEM “contains journals that report on the application of basic chemical sciences toother sciences, engineering, and industry” (ISI). Probably, all the journals of thiscategory could be multi-assigned to other related categories and, in fact, it shows a veryhigh multi-assignation rate. Moreover, not only ACHEM shows a higher percentage ofmulti-assigned journals than POLYM (83% vs. 39%), but also a higher external trend inthe multi-assignation pattern (55% vs. 33% of journals assigned to the categoriesoutside area), and a higher percentage of more than two-category journals (40% vs.25%of multi-assigned journals in each category). However, an excessive multi-assignation isavoided in the ISI scheme, and it should affect especially these horizontal categoriessuch as ACHEM. It seems that different kinds of categories coexist in the ISI classifica-tion scheme, and the degree of multi-assignation of journals (usually from two to fourISI categories) is not correlated with interdisciplinarity for these horizontal categories.Accordingly, we have doubts about the validity of ISI multi-assignation as aninterdisciplinary indicator in the case of broad categories, such as Chemistry or Physics.

Scientometrics 51 (2001) 219

F. MORILLO et al.: Interdisciplinarity through bibliometric indicators

These coexist with other more specific chemical or physical categories (for example,Inorganic Chemistry or Nuclear Physics) and since they are “mixed bags”, multi-assignation is probably artificially limited.

In our study, the ISI classification scheme proved useful to study interdisciplinarityat the category level, although with some limitations. ISI based indicators can providean interesting general overview of science, the relations between disciplines can beanalysed, and comparison among disciplines can be undertaken. Moreover, ISI basedindicators are less time-consuming than other approaches such as citation basedindicators, and primary outcomes are easily obtained. However, it presents theshortcoming of the heterogeneity of its categories, some being very specialised whilstothers are less defined. In this last case, multi-assignation in categories is not anappropriate indicator, since it is artificially limited by the needs of the classification.

At the level of particular journals, more specific indicators are needed. CAindicators, obtained from the analysis of all the documents published in each journal, arethe most precise. In fact, other authors have previously noted the convenience of usingspecialised databases (Tomov and Mutafov, 1996). According to CA indicators, ageneral typology of journals is presented, which classifies journals as specialised,multidisciplinary and interdisciplinary. This classification enables us to comparedifferent journals within a discipline, and to make comparisons between disciplinesaccording to the whole map of journals obtained for each discipline.

Our study also sheds some light on the question of whether interdisciplinarity is fieldor journal-dependent. We think that there is a field-dependent component, since averagevalues of the indicators for each category enable us to describe and compare differentcategories effectively. However, differences among journals within a category can beidentified in a more detailed analysis. Moreover, the weight of the different types ofjournals may probably vary from field to field. Therefore, interdisciplinarity is also ajournal-dependent feature.

In summary, our results support the usefulness of the bibliometric indicators hereproposed for the study of interdisciplinarity in the categories analysed. It should benoted that our study focuses only in two chemical categories, and differences may existamong areas. However, and being aware of those limitations, the results of our studysuggest that the indicators proposed are useful, and that the most appropriate indicatorsdiffer according to the level of analysis, whether it be areas, categories or journals.• At the macro level, ISI multi-assignation-based indicators are useful due to their

easy application. JCR and CA based indicators are discarded as being too laboriousto obtain at this level.

220 Scientometrics 51 (2001)

F. MORILLO et al.: Interdisciplinarity through bibliometric indicators

• At the category level, convergent results were obtained through ISI-multi-assignation indicators, JCR citing/cited patterns and CA based indicators. ISI-basedindicators were the most easily obtained, but JCR and CA-based indicators weremore sensitive.Two different cases should be distinguished: a) specific disciplines, which can bestudied through ISI multi-assignation-based indicators or document-based ones(JCR cited/citing patterns or those based on a specialised database); b) horizontaldisciplines such as ACHEM, in which document-based indicators are moreappropriate, since ISI considers these categories directly as multidisciplinary.

• Finally, for the study of journals the most appropriate indicators are those obtainedfrom specialised database classifications, such as Chemical Abstracts, from which atypology of journals may be obtained.

*

This study was partly financed by the Spanish research project SEC97-1375 and the research grantFPI-95 included in the research project SEC95-0082 supported by the Ministry of Education and Culture.We thank Laura Barrios for her help in the statistical analyses of data. We are also grateful to twoanonymous referees for their valuable suggestions.

References

Chemical Abstracts Service (CAS)(2000). CAS, Chemical Abstracts Service Home Page. http://www.cas.org(07-03-2000)

CRONIN, B., PEARSON, S., The export of ideas from information science, Journal of Information Science, 16(1990) 381–391.

Institute for Scientific Information. Journal Citation Reports 1996. Philadelphia, ISI, 1997.Institute for Scientific Information. Science Citation Index scope notes.

http://www.isinet.com/products/citations/scope/sci96b.htmKATZ, J.S., HICKS, D., The classification of interdisciplinary journals: A new approach, In: KOENIG, M.E.D.,

BOOKSTEIN, A. (Eds), Proceedings of the Fifth International Conference of the International Societyfor Scientometrics and Informetrics. Learned Information, Medford, 1995. pp. 245–254.

MORILLO, F., Estudio de la interdisciplinariedad en la ciencia a través de indicadores bibliométricos. Ph.D.Universidad Carlos III de Madrid. Getafe, 2000.

PORTER, A.L., CHUBIN, D.E., An indicator of cross-disciplinary research, Scientometrics, 8 (1985)161–176.

PRATT, A.D., A measure of class concentration in Bibliometrics. Journal of the American Society ofInformation Science, 28 (1977) 285–292.

Scientometrics 51 (2001) 221

F. MORILLO et al.: Interdisciplinarity through bibliometric indicators

QIN, J., LANCASTER, F.W., ALLEN, B., Types and levels of collaboration in interdisciplinary research in thesciences, Journal of the American Society for Information Science, 48 (1997) 893–916.

QIU, L., A study of interdisciplinary research collaboration. Research Evaluation, 2 (1992) 169–175.TIJSSEN, R. J. W., A quantitative assessment of interdisciplinary structures in science and technology: co-

classification analysis of energy research. Research Policy, 21(1992) 27–44.TOMOV, D. T., MUTAFOV, H. G., Comparative indicators of interdisciplinarity in modern science,

Scientometrics, 37 (1996) 267–278.URATA, H., Information flows among academic disciplines in Japan, Scientometrics, 18 (1990) 309–319.

Received February 16, 2001.

Address for correspondence:FERNANDA MORILLO

Centro de Información y Documentación Científica (CINDOC)Consejo Superior de Investigaciones Científicas (CSIC),Joaquín Costa 22, 28002 Madrid, SpainE-mail: [email protected]

222 Scientometrics 51 (2001)