forskningsanalyse? kvalitetsindikatorer? hvad er det for noget?
DESCRIPTION
Forskningsanalyse? Kvalitetsindikatorer? Hvad er det for noget?. Jeppe Nicolaisen Lektor, Ph.d. Danmarks Biblioteksskole. - PowerPoint PPT PresentationTRANSCRIPT
Forskningsanalyse? Kvalitetsindikatorer?
Hvad er det for noget?
Jeppe NicolaisenLektor, Ph.d.
Danmarks Biblioteksskole
Det kan forekomme nærved blasfemisk at påstå, at man kan sige noget centralt om videnskab bare ved at tælle antal sider eller antal tidsskrifter på et felt. Er man ikke nødt til at læse afhandlingerne, for at se om de er noget værd?
Tor Nørretranders (2007, s. 126)
Fremover bør vi anlægge internationale, objektive kriterier for tildeling af forskningsmidler. Det vil bringe dansk forskning op i verdensklasse. Objektiv, international vurdering af forskningens kvalitet gør det meget nemt at kende excellent forskning fra den jævne forskning.
Børge Grønne Nordestgaard (2007, s. 9)
Bibliometri
All studies which seek to quantify processes of written communication
Pritchard (1969, s. 348)
Bibliometriens anvendelser Informationssøgning
- Referencer, citationer, links
Vidensorganisation- Klyngeanalyse, bibliometriske kort
Forskningsevaluering- Forfattere, tidsskrifter, institutioner, lande- ’Collection management’
KvalitetsindikatorerPublikationsanalyseCitationsanalyse
Publikationsanalyse Kvantitative målinger på dokumentsamlinger.
Oftest optællinger af indholdet i et eller flere af de bibliografiske felter i en afgrænset del af en bibliografisk database.
Bl.a. optællinger af forfatteres, institutioners eller landes videnskabelige produktion fordelt på emne, tidsskrifter osv.
Citationsanalyse Kvantitative studier af dokumenters litteraturlister
Man skelner mellem to former for citationsanalyser:
- Referencer fra en samling dokumenter- Citationer til en samling dokumenter
reference A citation B Tid
Dokument B refererer til dokument A. Dokument A modtager en citation fra dokument B.
Bradfords lov
En type af matematisk-statistisk fordeling, opstillet på grundlag af iagttagelser over spredningen af artikler om et givent emne i tidsskrifter - nemlig at nogle få af tidsskrifterne rummer hovedparten af artiklerne, f.eks:
8 tss. producerer 110 artikler29 tss. producerer 133 artikler 1: 4: 16127 tss producerer 152 artikler
}
Bradfords lov
Management of Serials in Libraries:… de følgende pointer er nogle af de mest åbenbare potentialer af Bradford analyser:- Definition af emnets kerne- Selektion/kassation- Samlingsevaluering- Beregning af omkostninger ved forskellig
dækningsgrad- Prioritering af tidsskrifter
Nisonger (1998, s. 139-140)
Bradfords lov
Problemer (bl.a.):
Kvantitet/kvalitet Definitionen af ”emne”
- Ikke tidligere diskuteret ifht. Bradford’s lov
Nicolaisen, J. & Hjørland, B. (2007)Practical potentials of Bradford's law: A critical examination of the received view. Journal of Documentation, 63(3): 359-377.
Hjørland, B. & Nicolaisen, J. (2005). Bradford's law of scattering: ambiguities in the concept of "subject". Proceedings of the 5th International Conference on Conceptions of Library and Information Science: 96-106.
Hvornår er noget videnskabeligt?
Price (1970):
0 - 9 referencer…….. Unscholarly10 - 22 referencer.….. Scholarly23+ referencer……..... Non-creative
scholarship
Hvornår er noget videnskabeligt?
Paul Ehrlich (1909): Salvarsan
Alexander Fleming (1929): Penicilin
Hvornår er noget videnskabeligt?
Disciplinafhængig (Hyland, 2000):
Discipline Average per paper 1.000 words
Sociology 104,0 12,5Marketing 94,9 10,1Philosophy 85,2 10,8Molecular biology 82,7 15,5Applied linguistics 75,3 10,8Electronic engineering 42,8 8,4Mechanical engineering 27,5 7,3Magnetic physics 24,8 7,4
Den første citationsanalyseConsideration of the method of investigation [citation counts] here employed will show that we are concerned not merely with the quantity of work published […], but that in reality we are concerned only with the good work, the work which has survived and which has proved of value to the investigators who followed.
(Gross 1927, s. 641)
Citationer = Kvalitet
Citationsdatagenerelle problemer
Seglen (1996) [translated from Norwegian]:
References are selected because of their usefulness for the author, which is something different from their quality
Only a small fraction of all used material is cited General knowledge is not cited Knowledge is often cited from secondary sources Documents supporting an authors arguments are cited more often than
other documents Flattering (citing editors, potential referees and other authorities) Show-of (citing hot new “in”-articles) Reference copying (references provided by other authors) Conventions. In biochemistry, for example, methods are cited but not
reagents Self citations
Bibliometriens forsvar
Small (1987, s. 339):The issue is not whether we can rely on reference lists in individual cases as complete sets of influences (we cannot), but rather whether references can be used statistically, in the aggregate, as an indicator of influence.
Nederhof & Van Raan (1987, s. 326):… by no means the assumption is necessary, that scientists cite in their papers all work used in their research but still, citations can be used to monitor scientific influence.
Bibliometriens forsvar
Cawkell (1976, s. 3):Citation anomalies [in sence of ”excessive self-citations, plagiarism of references, careless or omitted references etc.] have little effect - they are like random noise in the presence of strong repetitive signals.
Gennemsnitsmantraet
Citation analysts assume that the biases and deficiencies of individual citers are repaired to a tolerable degree by the combined activity of the many.
White (2001, s. 102)
Gennemsnitsmantraet
Altså:Hvis citationsanalyser foretages på store datamængder vil man kunne eftervise en nøje sammenhæng mellem kvalitet OG antal citationer.Jo flere citationer – jo større kvalitet…
Citationer
Kvalitet
Bornsteins hypotese
If a relationship between citation frequency and research quality does exist, this relationship is not likely to
be linear. The relationship between research quality and citation
frequency probably takes the form of a J-shaped curve.
(Bornstein 1991, s. 31)
Ekspertvurdering
Citations-frekvens
J-kurve?1. En samling dokumenter2. En samling ekspertvurderinger af 13. En samling citationer til 1
Stikprøven:
1. Bøger anmeldet i tidsskriftet Contemporary Sociology mellem 1985-1994
2. Ratings fra databasen Sociological Abstracts3. Citationer søgt i Social Science Citation Index (5-års
citationsvindue)
J-kurve?Stikprøve:
Rating Antal anmeldelser i CS 5%Stikprøve (1985-1994)
Very favorable 275 13,75 40Favorable 4.907 245,35 246Neutral 996 49,80 50Unfavorable 879 43,95 44Very unfavorable70 3,50 40
Total 7.127 356,35 420
J-kurve?Gennemsnit og median citationsfrekvenser
Rating Gn.snit Median
Very Favorable 17,53 17,50Favorable 19,27 8,00Neutral 15,42 6,00Unfavorable 8,48 4,50Very unfavorable 15,78 6,00
Total 17,18 8,00
Resultater (gennemsnit)
Very unfavorableUnfavorable
NeutralFavorable
Very favorable
Evaluation (dependent variable)
0,00
5,00
10,00
15,00
20,00
Cita
tions
(ind
epen
dent
var
iabl
e)
Resultater (median)
Very unfavorableUnfavorable
NeutralFavorable
Very favorable
Evaluation (dependent variable)
0,00
5,00
10,00
15,00
20,00
Cita
tions
(ind
epen
dent
var
iabl
e)
Nicolaisen, J. (2002).The J-shaped distribution of citedness.Journal of Documentation, 58(4): 383-395.
Indikatorteori
The current holy grail in scientometrics is the development of indicator theories rather than citation theories.
Van der Veer Martens (2001, online)
Paul Wouters (1999)
The quest for a citation theory it is a dead end (s. 211).
Science and technology indicators create a “formalized representation” of science which initially neglects meaning. Of course, to interpret these representations one needs to attribute meaning again. The main point is, however, that this attribution of meaning can be postponed (s. 209).
Fred Dretske (1981)
Informationel semantik
Et signal bærer betydningen p hvis og kun hvis signalet indikerer p.
Røg - - - - - - Brand!
The error problem
What it indicates is mostly not what it represents…
Tænk på en lille fugl!
Sikker mekanisme/ Sikkerhedsmekanisme
Informational semantics seeks to uncover signals, which are reliably correlated with specific situations and hence indicate these situations. In other words, informational semantics attempt to establish certain mechanisms, but only safe ones are logically possible.
Nicolaisen (2007, s. 631)
Eugene Garfield (1979)A highly cited work is one that has been found useful by a relatively large number of people, or in a relatively large number of experiments […]. The citation count of a particular piece of scientific work does not necessarily say anything about its elegance or its relative importance to the advancement of science or society […]. The only responsible claim made for citation counts as an aid in evaluating individuals is that they provide a measure of the utility or impact of scientific work. They say nothing about the nature of the work, nothing about the reason for its utility or impact (s. 246).
Fortolkning
Citation analysis is not a shortcut to be used as a substitute for thinking
(f.eks. Egghe & Rousseau, 1990, s. 226;
Garfield, 1985, s. 408)
Introduction to Informetrics (1990)
We expect this book to be of help to the informetrics teacher in organising his or her course and to be interesting and useful both as a course book and as a background reading for students in library and information science (Egghe & Rousseau 1990, s. v).
Fortolkning
Citationsteori
Kulturel disciplinering
Begreber, mening, viden, informationsbehov, relevanskriterier, etc. formes i diskursfællesskaber, f.eks. i videnskabelige discipliner, fag, domæner.
Domæneanalyse
Forstå domænets
- Ontologi- Epistemologi- Sociologi
Fortolkningens grundlagVidenskabsfilosofiVidenskabshistorieVidenskabssociologi
Referencer Bornstein, R.F. (1991). The predictive validity of peer review: A neglected issue.
Behavioral and Brain Sciences, 14(1): 138-139. Cawkell, A.E. (1976). Understanding science by analysing its literature. The
Information Scientist, 10(1): 3-10. Dretske, F. (1981). Knowledge and the Flow of Information. Cambridge, MA: MIT
Press. Egghe, L. & Rousseau, R. (1990). Introduction to Informetrics: Quantitative Methods
in Library, Documentation and Information Science. Amsterdam, NL: Elsevier. Garfield, E. (1979). Citation Indexing: Its theory and Applications in Science,
Technology and Humanities. New York, NY: Wiley. Garfield, E. (1985). Uses and misuses of citation frequency. Current Contents, 43
(October 28), 3- 9. Gross, P.L.K. (1927). Fundamental science and war. Science, 66: 640-645. Hyland, K. (2000). Disciplinary Discourses: Social Interactions in Academic Writing.
Harlow, UK: Pearson Education Ltd. Martens, B. Van der Veer. (2001) Do citation systems represent theories of truth?
Information Research, 6(2). [Available at: http://informationr.net/ir/6-2/paper92.html]. Visited September 18, 2005.
Nederhof, A.J. & Van Raan, A.F.J. (1987). Citation theory and the Ortega hypothesis. Scientometrics, 12(5-6): 325-328.
Nordestgaard, B. G. (2007). Et strejf af nepotisme [Kronik]. Jyllands-Posten, 27. august. Nicolaisen, J. (2007). Citation analysis. Annual Review of Information Science and
Technology, 41: 609-641. Nisonger, T.E. (1998). Management of Serials in Libraries. Englewood, CO: Libraries
Unlimited. Nørretranders, T. (2007). Civilisation 2.0: Miljø, fællesskab og verdensbillede i linkenes
tidsalder. [Frederiksberg]: Thaning & Appel. Price, D.J.S. (1970). Citation measures of hard science, soft science, technology, and
nonscience. In: Nelson, C.E. & Pollock, D.K. (eds.), Communication Among Scientists and Engineers. Lexington, MA: Heath: 3–22.
Pritchard, A. (1969). Statistical bibliography or Bibliometrics?. Journal of Documentation, 24: 348-349.
Seglen, P.O. (1996). Bruk av siteringer og tidsskriftimpaktfaktor til forskningsevaluering. Biblioteksarbejde, 48: 27-34.
Small, H. (1987). The significance of bibliographic references. Scientometrics, 12(5-6): 339-341.
White, H.D. (2001). Authors as citers over time. Journal of the American Society for Information Science and Technology, 52(2): 87-108.
Wouters, P. (1999). The Citation Culture. Amsterdam, NL: Universiteit van Amsterdam. Ph.D.