Beyond CitationsDemonstrating your Impact through

Alternative Metrics

October 14, 2014

Kelly Grossmann, Science Librarian

MS Information, MS Bioinformatics

Outline

1. How and why do we assess impact?

2. Methods and Metrics for Impact Assessment

- Traditional

- Alternative

3. Improving your Impact

How do we assess the impact of

research?

… and why should we?

The scholarly publication world is

rapidly changing.

Growth of Publications

Growth of PublicationsDevelopment of Online Publishing - PLOS One analysis

Growth of Retractions

Reduction of Funding

What does this mean for you?

For Faculty & Researchers

“Publish or Perish”

- Showing academic value in a sea of

information.

Increased importance of relevance and

significance of research

For Students

Evaluate sources and identify core papers

Follow key research trends

Perhaps even demonstrate your scholarly impact for

graduate school

For Fellow Data Nerds

Traditional Metrics

Citation Counts

Google Scholar

Citation Counts

Biological Abstracts/Web of Science

Impact Factor

Journal level metric

Average number of citations to articles within journal

Easily Googled

Allows comparison by subject in Journal Citation Reports

Proprietary

Eigenfactor

Journal level

Scored .01-100

Accounts for importance of citations

More robust than IF

Open Source

Data party fun times: http://www.eigenfactor.org/map/maps.php

Copyright © 2012 University of Washington

h-index

Measured at the author level

h-index = number of papers h

that have been cited h times

i10 Index

Number of publications cited by more > 10 other articles

Calculating h & i10 Index

Google to Find h-index

http://scholar.google.com

Use author profile to find author h-index

Use Journal metrics to find journal h-index (average)

Allows you to compare impact within your field

Google Scholar Profile

Criticism of Traditional Methods

Depend on Counters

“Gaming the system”

Slow

Vary by time and length of

career

Gender bias in referring

practices

Skew towards favoring popular

Do not connotate

negative/positive references

Vary greatly by discipline

Often only available by Journal

(IF & EF)

Traditional Metrics only tell one part

of the story. That’s a lot

of Sharks!

Altmetrics

Altmetrics

Short for Alternative Metrics

Methods of analyzing impact beyond citations

Made possible by our ability to quickly, electronically share

information.

A Brief History

Altmetrics

Clicks

Captures

Retweets

Blog citations

Shares

+1

Views

Bookmarks

Downloads

Abstract views

Wikipedia links

Comments

Examples of Altmetrics

Meneley Readership Statistics

Citation manager

Collects statistics on article usage

http://www.mendeley.com/research-papers/

Altmetric Bookmarklet

http://www.altmetric.com/bookmarklet.php

http://www.pnas.org/content/101/suppl_1/5200.

short

Altmetrics from Publishers

Impact Aggregation Tools

Impact Story - https://impactstory.org/

Plum Analytics - http://www.plumanalytics.com/

Publish or Perish - http://www.harzing.com/pop.htm

Benefits to Using Altmetrics

Provide us with the ‘rest of the story’

Show public impact, too

View up-to-date trends in research

Improve your Impact. Share.

Where to Share

GitHub - Code

FigShare - Datasets, images, videos

SlideShare - Presentations

Twitter - Social networking

Facebook - Social networking

In summation...

Many metrics, many tools.

Combination of metrics is best.

Be sure to compare within your discipline.

Be sure to share your work to improve your

impact.

Get the whole story.

References[1] Sci2 Team. Science of Science (Sci2) Tool. User Manual. http://wiki.cns.iu.edu/pages/viewpage.action?pageId=2200066

[5] Priem, J. "MEDLINE-indexed articles published per year." R Chart. Jason Priem/blog 18 Oct. 2010. Accessed 10 Oct. 2014 <http://jasonpriem.org/2010/10/medline-literature-growth-chart/>.

[6] Laakso M, Welling P, Bukvova H, Nyman L, Björk B-C,et al. (2011) “The Development of Open Access Journal Publishing from 1993 to 2009.” Figure 2: The development of open access publishing.

PLoS ONE 6(6): e20961. doi:10.1371/journal.pone.0020961.

[7] Van Noorden, R. “Science publishing: The trouble with retractions.” Box: Rise of the retractions. Nature. Published online 5 Oct. 2011. Accessed 10 Oct. 2014. doi: 10.1038/478026a

<http://www.nature.com/news/2011/111005/full/478026a/box/2.html>

[7] Bohannon, J. “Who’s Afraid of Peer Review?” Science. 4 Oct. 2013. 342(6154). pages 60-65. doi: 10.1126/science.342.6154.60 <http://www.sciencemag.org/content/342/6154/60.full>

[8] McGovern, V. “Foundation funding and chemical biology.” Trends in research funding by agency. Nature Chemical Biology. 4, 519-522. 2008. doi: 10.1038/nchembio0908-519.

<http://www.nature.com/nchembio/journal/v4/n9/fig_tab/nchembio0908-519_F1.html>

[12, 13] Sci2 Team. Science of Science (Sci2) Tool. Indiana University and SciTech Strategies, (2009). https://sci2.cns.iu.edu.

[19] University of Washington. “Overview”. Eigenfactor information page. <http://www.eigenfactor.org/methods.php> Image: A model of research. <http://www.eigenfactor.org/images/animatedfigure.gif>

Accessed on 10 Oct. 2014.

[20] Vulpecula (User name). "h-index" h-index from a plot of decreasing citations for numbered papers. Wikipedia. 02 Jan. 2008. Accessed 10 Oct. 2014 <http://en.wikipedia.org/wiki/H-index>.

[25] Wendl, M.C. “H-index: however ranked, citations need context.” Nature. (2007) 449: 403.

[25] Kelly, C. D. , Jennions, M.D. “H-index: age and sex make it unreliable.” Nature. (2007) 449: 403.

[25] Pagel, P.S., Hudetz, J.A. “H-index is a sensitive indicator of academic activity in highly productive anaesthesiologists: results of a bibliometric analysis.” Acta Anaesthesiologica Scandinavica. (2011).

55:9. 1085-1089. dio: 10.111/j.1399-6576.2011.02508.x <http://onlinelibrary.wiley.com/doi/10.1111/j.1399-6576.2011.02508.x/abstract>

[26] West Side Story. Dir. Jerome Robbins and Robert Wise. Perf. Natalie Wood. Comp. Leonard Bernstein. Mirisch Corporation, 1961. Film.

[29] Careless, J. (2013). Altmetrics 101: A Primer. (cover story). Information Today, 30(2), 1-36. [Accessed October, 2014].

[32] Lowman, M. (2014). How to Raise a Woman Scientist. Retrieved 13 Oct. 2014, from http://www.huffingtonpost.com/meg-lowman/how-to-raise-a-woman-scie_b_5928644.html

[34, 35] Mendeley. <http://www.mendeley.com/catalog/modularity-community-structure-networks-27/> Accessed 10 Oct. 2014

[37] Thomson, J. “Altmetrics added to Royal Society of Chemistry Journals”. RSC Publishing Blog. 12 Sept. 2013. Accessed on 10 Oct. 2014. http://blogs.rsc.org/rscpublishing/2013/09/12/altmetrics-added-

to-royal-society-of-chemistry-journals/

Further Reading

Altmetrics Manifesto http://altmetrics.org/

University of Maryland Altmetrics LibGuide

http://lib.guides.umd.edu/altmetrics

HLWiki International:

http://hlwiki.slais.ubc.ca/index.php/Author_impact_metrics

Full citations available in presentation notes.

Questions?