Journal Ranking Metrics new perspective in Journal performance

management

Professor Aboul Ella HassanienCairo Univesrity

Faculty of Computers and Information

Chairman of Scientific Research Group in Egypt (SRGE) http://www.egyptscience.net @ Aboitcairo@gmail.com

Using the Impact factor alone to judge a journal is like using weight to judge a person health

The title is a sentence taken from a report on citation statistics prepared by the International Mathematical Union (IMU). Another such take-home message is "Research is too important to measure its value with only a single coarse tool." Given that citation statistics are heavily used in assessing research and researchers, young scientists might gain a lot from investing some time to familiarize themselves with this subject.

Science is a game

Science is a game and the best players appreciate not only the beauty of a discovery by a precise and logical reasoning, but also the taste of making a guess

There is no way to get experience except through experience

Writing a paper is …

A lot like chip design

You need Conceptualization Floor planning and

layout Interconnections Testing

Why write and publish research papers?

Ideally to share research

findings and discoveries with the hope of improving some things.

Practically –

to get funding to get promoted to get a job to keep your job!

Academic publishing

Academic publishing describes a system that is necessary in order for academic scholars to peer review the work and make it available for a wider audience

Multiple Trends Impact The Research Landscape

Competition / Collaboration

Accountability / Government policies

Improve research outcomes

Multidisciplinarity

High mobilityResearchStrategy

& Research

Performance

What makes a good research paper?

Good science Good writing Publication in good journals

Deadly Sins - Plagiarism

Plagiarism

destroyed your

carri

er.

"Plagiarism is defined as the act of taking the specific substance of another and offering it as one's own without giving credit to the sources. 

Major Metrics: Research Impact

Publication count Number of publications produced by individual,

school or university Citation count

Number of times publication is cited by other publications

Major Metrics: Journal quality

Impact factor – average number of times

published papers are cited up to two years after publication

Immediacy Index average number of times

published papers are cited during year of publication.

Journal Evaluation Some challenges

I’m a publishing author in a niche area, how can I compare with researchers in another scientific field?

How can I get a value that reflects citation behaviour in my research area?

It’s 2010 – why can I only get a ranking relevant to 2008?

Where can I find the data that have been used to generate this number?

The Impact Factor does not cover the journal I’m publishing in. What now?

Difficult to compare

A Not recent enough

Lack of clarity of data origin

Lack of coverage

Journal Impact Factors, Immediacy Index and the Author h-index: tools used for benchmarking the quality of research

Journal Impact Factors: Why?

Evaluate the scholarly worth of a journal Rank journals within a discipline Help you decide where to publish your article for

maximum impact Evaluation for promotion / tenure / grants, or in

some countries, even government funding of an institution

Frequently used as an evaluation source by librarians during journal cancellations or new purchases

Journal Impact Factors: Why?

In Egypt, researchers in Cairo University are rewarded for publishing in journals defined by ISI as prestigious

In England, hiring panels routinely consider impact factors

By Spanish law, researchers are rewarded for publishing in journals defined by ISI as prestigious

In China, scientists get cash bonuses for publishing in high-impact journals. In some schools, physics students must publish at least 2 articles with a combined Impact Factor of 4 to get their PhD

Journal Impact Factors: Where?

Impact factors are listed in Journal Citation Reports JCR

You can easily get to the JCR from the Web of Science, so let’s start there, since understanding the Web of Science will help us better understand where the data for the JCR comes from.

Tip: Use Web of Science for article-level information; JCR for journal-level information. Both the Web of Science and the JCR are based on the same database of journal citations and cited references

Web of Science®

Thomson Reuters (formerly ISI) has one, huge database, Web of Science, that… Indexes selected journals

> 8,000 science; > 3,000 social science journals; > 1,800 Arts & Humanities

Tracks “cited references” and “times cited” Sample topic/author search: “impact factor”

Activity: Search for an article in your field that has been highly cited.

Journal Citation Reports

Journal Citation Reports (JCR) keeps track of citations at the journal level (same data as in Web of Science, just a different presentation of the data). A new edition comes out every Spring.

Tip: In addition to accessing the JCR via a Web of Science citation, you can access the JCR from our Online Journals & Databases resource: openurl.library.uiuc.edu/sfxlcl3/az? Activity: Get into the JCR and then:

Choose the Science or Social Sciences edition, and year Search by individual journal title, or by subject category Example:

Subject: Biochemistry & Molecular Biology Then, “View Journal Data”, sorted by Impact Factor

Journal Impact Factors: What?

What is the Journal Impact Factor? How is it calculated?

Only references in articles within the MORE than 13,000 journals indexed in Web of Science are counted; does not include citations that may cite the articles in Cell from book chapters, proceedings, or other journals that are not indexed in Web of Science

Citable articles are just research articles and reviews – not news articles, commentary, etc.

20

Impact Factor =Cites in 2009 to 2007 or 2008 papers

Papers published in 2007 or 2008

The impact factor is calculated by dividing the number of citations in the JCR year by the total number of articles published in the two previous years.

Impact Factor CalculationJournal: ACADEMY OF MANAGEMENT JOURNAL 

21

The immediacy index is calculated by dividing the number of citations to articles published in a given year by the number of articles published in that year.

Immediacy Index CalculationJournal: ACADEMY OF MANAGEMENT JOURNAL 

EFFICIENCY JOURNAL IMPACT FACTOR

2008 Impact Factor

200820072006

Source paper – published in 2008

Cited reference – published in 2006 or 2007

Citations

All Previous Years

2005 2009

23

Implications of Impact Factor

Tell us how frequently has the average article in a journal been cited in a particular year.

Tell us something about a journal as a whole e.g. the extent to which its recently published papers were cited in a given year.

Impact factor > 1 implied a journal is frequently cited Higher citations rate means your article has higher

chances of getting cited or read by researchers. Tells us NOTHING concrete about any specific paper or

specific author.

Implications of Impact Factor

Only a limited subset of journals is indexed by ISI Only uses the articles cited by the ~13,000 “ISI journals” Some disciplines are especially poorly covered

Biased toward English-language journals ISI has recently added several hundred non-English journals

Short (two year) snapshot of journal Some disciplines use older material more or take time to cite new

research JCR now also includes the 5-year data

Is an average; not all articles are equally well-cited

Implications of Impact Factor

Includes self-citations, that is articles in which the article cites other papers in the same journal

Only includes “citable” articles in the denominator of the equation, i.e., articles and reviews Editors may skew IF by increasing the number of review

articles, which bring in more citations (increases the numerator)

Or by increasing the number of “news” items (e.g., Science, general medical journals) , which are cited (appear in numerator) but not considered “citable” (and so aren’t in the denominator)

It is expensive to subscribe to the JCR

http://www.slideshare.net/danrholden/bibliometrics-12605810?next_slideshow=1

Publish or perish (POP)

Publish or Perish is a free analysis software that retrieves and analyzes academic citations.

The results are available on-screen and can also be copied to the Windows clipboard (for pasting into other applications) or saved to a variety of output formats

Download and install Publish or Perish

http://www.harzing.com/pop_win.htm

Google Scholar disadvantage

Scopus

Scopus - Most comprehensive Abstracts and Citations database

Increased prestige, ranking and funding because you . . .

. . . improve your institutional productivity . . .

. . . by enabling the management to make better decisions . . .

. . . and improving student and researcher productivity . . .

…because Scopus (a) enables you to evaluate performance and quality accurately . . .

. . . (b) offers you the most efficient and effective search . . .

. . . and (c) provides access to the most comprehensive content . . .

. . . thereby unlocking the library’s full potential enabled by . . .

Best evaluation of performance and quality:• Broadest

coverage of peer-reviewed, high quality journals

• Globally as well as locally

• More accurate citation count, fairer recognition for work

Improve productivity of Researcher and Student:• Most efficient

and effective search

• Access to most comprehensive content

Secure institutional success . . .

Scopus vs Web of Science

http:/www.slideshare.net/danrholden/bibliometrics-12605810?next_slideshow=1

h-Index… for Evaluating Authors (or Journals)

Proposed by JE Hirsch as an index to quantify an individual's scientific research output

Remember, Impact Factor is just for journals, though it’s often used to evaluate authors.

Combines an assessment of both quantity (number of papers) and quality (impact, or citations to these papers).

h-index is automatically calculated: Web of Science Scopus Publish or Perish (free download), based on data in Google Scholar

h-index can also be manually calculated for an author based on the number of papers authored and the number of times each paper has been cited.

See Wikipedia article for overview of h-index including criticisms, alternatives http://en.wikipedia.org/wiki/H-index

Determining h-index Manually

From h-index, http://en.wikipedia.org/wiki/H-index

Finding a h-index value in Web of Science

Scopus new bibliometric indicators to measure the influence of journals

IPP (Impact per Publication) The Impact per Publication measures the ratio of

citations per article published in the journal.

SNIP (Source Normalized Impact per Paper) Source Normalized Impact per Paper measures

contextual citation impact by weighting citations based on the total number of citations in a subject field

SJR (SCImago Journal Rank). SCImago Journal Rank is a prestige metric based on the

idea that 'all citations are not created equal'. With SJR, the subject field, quality and reputation of the journal have a direct effect on the value of a citation. It is a size-independent indicator and it ranks journals by their 'average prestige per article' and can be used for journal comparisons in the scientific evaluation process

SJR (SCImago Journal Rank).

Developed by Professors Félix de Moya, Research Professor at Consejo Superior de Investigaciones Científicas and Vicente Guerrero Bote at University of Extremadura,

SCImago Journal Rank (SJR) is a prestige metric based on the idea that 'all citations are not created equal'.

With SJR, the subject field, quality and reputation of the journal have a direct effect on the value of a citation.

SJR is a measure of scientific influence of scholarly journals that accounts for both the number of citations received by a journal and the importance or prestige of the journals where such citations come from.

It is a variant of the eigenvector centrality measure used in network theory.

Research Paper: "The SJR indicator: A new indicator of journals' scientific prestige"

SJR (SCImago Journal Rank).

The SCImago Journal & Country Rank is a portal that includes the journals and country scientific indicators developed from the information contained in the Scopus® database  

These indicators can be used to assess and analyze scientific domains.

This platform takes its name from the SCImago Journal Rank (SJR) indicator, developed by SCImago from the widely known algorithmGoogle PageRank™. This indicator shows the visibility of the journals contained in the Scopus® database from 1996.

IPP (Impact per Publication)

The IPP measures the ratio of citations in a year (Y) to scholarly papers published in the three previous years (Y-1, Y-2, Y-3) divided by the number of scholarly papers published in those same years (Y-1, Y-2, Y-3). The IPP metric is using a citation window of three years which is considered to be the optimal time period to accurately measure citations in most subject fields.

SNIP (Source Normalized Impact per Paper)

Source Normalized Impact per Paper measures contextual citation impact by weighting citations based on the total number of citations in a subject field.

http://www.sciencedirect.com/science/article/pii/S1751157710000039

SNIP/IPP

Source-Normalized Impact per Paper (SNIP) measures contextual citation impact by weighting citations based on the total number of citations in a subject field.

This unique aspect allows direct comparison of sources in different subject fields. The impact of a single citation is given higher value in subject areas where citations are less likely, and vice versa.

It is a ratio, with a numerator and a denominator. SNIP's numerator gives a journal's impact per publication (IPP). This is simply the average number of citations received in a particular year (e.g. 2013) by papers published in the journal during the three preceding years (e.g. 2010, 2011 and 2012).

SNIP/IPP

SNIP's denominator, the Database Citation Potential (DCP) is calculated as follows. We know that there are large differences in the

frequency at which authors cite papers between various scientific subfields. In view of this, for each journal an indicator is calculated of the citation potential in the subject field it covers. This citation potential is included in SNIP's denominator.

SNIP is IPP divided by DCP.

It aims to allow direct comparison of sources in different subject fields. Citation potential is shown to vary not only between journal subject categories - groupings of journals sharing a research field - or disciplines (e.g., journals in Mathematics, Engineering and Social Sciences tend to have lower values than titles in Life Sciences), but also between journals within the same subject category. For instance, basic journals tend to show higher citation potentials than applied or clinical journals. Likewise, journals covering emerging topics tend to be higher than periodicals in classical subjects, or more general journals.

SNIP

SNIP helps authors to identify which journals are performing best within their subject field and where to publish. Mouse over the circles in the visualization and click on the years to view the metrics. The size of the circles are compared to the highest values in the 5-year range. This highest value has a closed circle whereas the open circles indicate the value compared to this highest value.

Example

References

http://journalinsights.elsevier.com/journals/0031-9422/snip

http://www.sciencedirect.com/science/article/pii/S1751157710000039

http://www.harzing.com/pop.htm#about http://www.journalmetrics.com/ipp.php http://

pitt.libguides.com/c.php?g=12107&p=64718 http://

lib.guides.umd.edu/content.php?pid=569182&sid=5022406