2

Journals in the arts and humanities: their role and

evaluation

Professor Geoffrey Crossick Warden

Goldsmiths, University of London

3

4

Distinctive place of journals in arts & humanities

research Diversity of output monographs, edited collections edited texts, journal articles practice outputs

No clear hierarchy of esteem amongst as well as within each challenge for RAE and promotion panels esp problems for practice outputs

5

Why do we want to know about journals? Explore the

terms Use of journals to indicate:

activity and productivity

impact - relevance and usequality - peer judgment

6

Measurements and proxies

Activity &Productivity Impact Quality

Quantitative Tangible & Qualitative methods & intangible methods &

data data evidence

& data

Increasing use of proxy metrics to infer impact and quality

Especially in context of changes to RAE

7

Why is it so hard to rank journals or use citation data?

Rank according to what criteria? Impact factors very difficult Citation behaviour is very different

not cumulative cf science old work remains highly cited [See ISI list]critical discourses as mode of researchcitation not clear sign of quality/influencethe culture of the footnote

Diverse outputs - not just paper-based Print output different arts & humanities Lower % paper outputs in ISI-Journals

8

Highest citations in ISI humanities journals 2000

1 Karl Marx2 Lenin3 Shakespeare4 Aristotle5 The Bible6 Plato7 Freud8 Chomsky9 Hegel10 Cicero

9

RAE submissions 1996 & 2001

Humanities, Languages and Arts Journal articles? 1996 - 33% 2001 - 37%

sciences 90% 96%

engineering 57% 78% social sciences 42% 54%

Books? 1996 - 51% 2001 - 52% sciences 6% 3%

engineering 8% 5%social sciences 32% 28%

Book chapters 1996 - 4% 2001 - 3%

Other 1996 - 11% 2001 – 9%

10

Journal publication & RAE quality

Journal articles as % outputs to RAE2001 by disciplineUoA 3b 3a 4 5 5*

Law 83 71 61 53 43

Asian studies 43 56 55 42 29

History 37 36 33 33 34

Art & design 7 7 10 9

Music 11 15 15 23 27

11

ISI journals & the arts & humanities Publication in ISI journals often small %

overall outputs RAE 2001 Philosophy highest at 52% Library & information management 40% Most other subjects in 20%-29% range English & French just 21% Below 20% in Italian, Theology, Art &

Design and Middle Eastern & African Studies

12

Current projects on quality & journals Issue of evaluating quality very current

So too is assessing standing of journals But they’re not the same thing at all And decreasingly so arts & humanities Reflect on these:

RAE and metrics European Science Foundation’s ERIH Humanities Indicators Project

13

RAE metrics Metrics-driven RAE

‘Neither citations nor RC income’ AHRC expert group

outputs but no proxies for their quality Post-2008: STEM cf rest of disciplines

‘robust indicators’ being sought – for STEM

primarily HESA and bibliometric data arts & humanities 2013+? earlier impact consultation non-science 2009-10 national bibliometrics consortium

14

European Reference Index for the Humanities

European Science Foundation “The ERIH lists will help to identify

excellence in Humanities scholarship & should prove useful for the aggregate bench-marking of national research systems…in determining the international standing of the research activity in a given field in a country. However, as they stand, the lists are not a bibliometric tool”

15

Methods & goals of ERIH

Expert Panels in 15 disciplinary areas ESF member lists, iterative consultation Categorisation (lists emerging this

year) A = high-ranking international level B = standard international level C = important local/regional level

Resistance to hierarchy: why? Say above all to strengthen peer review How, if at all, will it be used?

16

Humanities Indicators Project

American Academy of Arts & Sciences many variables on state of

humanities Publications element:

Focus is monograph publications data no quality indicators sought

No plans to look at journal publishing Making humanities count: the

importance of data

17

Arts & humanities journals:the challenge of bibliometrics

Is there a challenge? Does anyone want to do bibliometrics with

arts & humanities journals? Little interest UK or elsewhere In many ways for good reasons set out here Yet looming is RAE post-2013….

can we build ‘robust measures’ without them? can it be done with them?