the infrastructure for alternative metrics
DESCRIPTION
The Infrastructure for Alternative Metrics: What do we need to compare digital apples to digital apples? Todd Carpenter, Executive Director, NISO Presented at AAAS Annual Meeting: February 16, 2014 Chicago, ILTRANSCRIPT
![Page 1: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/1.jpg)
The Infrastructure for Alternative Metrics
What do we need to compare digital apples to digital apples?
!Todd Carpenter, Executive Director, NISO
AAAS Annual Meeting Chicago, IL | February 16, 2014
![Page 2: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/2.jpg)
We hate discussing infrastructure
![Page 3: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/3.jpg)
Photo: Minneapolis Collegeof Art and Design Library
![Page 4: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/4.jpg)
• Non-profit industry trade association accredited by ANSI with 190+ members
• Mission of developing and maintaining technical standards related to information, documentation, discovery and distribution of published materials and media
• Volunteer driven organization: 400+ spread out across the world
• Responsible in various wasy for standards like ISSN, DOI, Dublin Core metadata, DAISY digital talking books, OpenURL, MARC records, and ISBN
About
![Page 5: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/5.jpg)
Photo by dumbledad on Flickr http://www.flickr.com/photos/dumbledad/298650884/!
![Page 6: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/6.jpg)
But what is the “infrastructure” for
assessment?
![Page 7: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/7.jpg)
We stand on the shoulders of giants...
![Page 8: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/8.jpg)
The Mighty Citation
![Page 9: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/9.jpg)
Impact factor
![Page 10: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/10.jpg)
Thomson-Reuters IMPACT FACTOR
![Page 11: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/11.jpg)
Two Stanford Students
![Page 12: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/12.jpg)
Two Stanford Students
![Page 13: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/13.jpg)
They wrote a paper
![Page 14: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/14.jpg)
They started a company in someone’s garage
Photo: AP
![Page 15: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/15.jpg)
![Page 16: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/16.jpg)
Citations are awesome, aren’t they?
A perfect solution to all the world’s problems, no?
A simple conclusion can be drawn
![Page 17: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/17.jpg)
Back to Sir Isaac Newton“What Des-Cartes did
was a good step. You have added much several ways, & especially in
taking ye colours of thin plates into philosophical consideration. If I have seen further it is by
standing on ye shoulders of Giants."
![Page 18: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/18.jpg)
Citations aren’t always
what we think they
are
Caveat emptor
![Page 19: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/19.jpg)
Another problem with citations
![Page 20: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/20.jpg)
So where to from here?
![Page 21: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/21.jpg)
Define altmetrics, please
![Page 22: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/22.jpg)
Social Media Metrics
Image: Danny Brown, The State of Social Media Marketing 2012
![Page 23: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/23.jpg)
Usage Based Metrics
![Page 24: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/24.jpg)
Click Map Analysis
![Page 25: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/25.jpg)
Clickstream data yields high-resolution maps of science
Johan Bollen, Herbert Van de Sompel, et al. PLoS One, February 2009
Network Analysis
![Page 26: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/26.jpg)
Clickstream data yields high-resolution maps of science
Johan Bollen, Herbert Van de Sompel, et al. PLoS One, February 2009
Network Analysis
![Page 27: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/27.jpg)
Behavioral Metrics
![Page 28: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/28.jpg)
Sentiment Analysis
![Page 29: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/29.jpg)
Image: Domenico, Caron, Davis, et al.
Non-traditional content
Assessing impact of new forms of communication isn’t as simple as one might think.
![Page 30: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/30.jpg)
Image: Domenico, Caron, Davis, et al.
Non-traditional content
Assessing impact of new forms of communication isn’t as simple as one might think.
![Page 31: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/31.jpg)
We’ve got what, now how?
![Page 33: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/33.jpg)
How do we measure things?
![Page 34: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/34.jpg)
At what granularity do we measure things?
![Page 35: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/35.jpg)
Consistency in what is counted
Source: Scott Chamberlain, Consuming Article-‐Level Metrics: Observations And Lessons From Comparing Aggregator Provider Data, Information Standards Quarterly, Summer 2013, Vol 25, Issue 2.
![Page 36: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/36.jpg)
Which measurements are valuable?
![Page 37: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/37.jpg)
How do we ensure the validity of those measurements?
![Page 38: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/38.jpg)
How to (or even if we should) deal with other issues
such as gaming?
![Page 39: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/39.jpg)
What infrastructure do we need for
alternative metrics?
![Page 40: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/40.jpg)
Basic Definitions !
(So we are all talking about the same thing)
![Page 41: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/41.jpg)
Comparison across providers
Source: Scott Chamberlain, Consuming Article-‐Level Metrics: Observations And Lessons From Comparing Aggregator Provider Data, Information Standards Quarterly, Summer 2013, Vol 25, Issue 2.
![Page 42: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/42.jpg)
Authorship -----
Disambiguation &
Contributor Roles
![Page 43: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/43.jpg)
Element Identification
![Page 44: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/44.jpg)
Open exchange of component data
![Page 45: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/45.jpg)
TRUST
=
![Page 46: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/46.jpg)
Where does NISO fit in?
![Page 47: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/47.jpg)
![Page 48: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/48.jpg)
![Page 49: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/49.jpg)
Alterna(ve Assessment Ini(a(ve
!
Phase 1 Mee(ngs October 9, 2013 -‐ San Francisco, CA December 11, 2013 -‐ Washington, DC
January 23-‐24 -‐ Philadelphia, PA !
Phase 1 report expected in May 2014
![Page 50: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/50.jpg)
Just a few ideas from in-‐person mee(ngs Disambigua)on problems
Create a central repository for data Define provenance for metrics
Foster culture of DOI use in mass media Establish defini)ons for metric types Define data extract/gathering methods
Standardize units of measure Define levels of transparency/privacy
!
!
![Page 51: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/51.jpg)
Alterna(ve Assessment Ini(a(ve !
Phase 2 Presenta(ons of report (June 2014) Priori(za(on Effort (June -‐ Aug, 2014)
Project approval (Sept 2014) Working group forma(on (Oct 2014)
Consensus Development (Nov 2014 -‐ Dec 2015) Trial Use Period (Dec 15 -‐ Mar 16)
Publica(on of final recommenda(ons (Jun 16)
![Page 52: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/52.jpg)
May 15, 2013 74
![Page 53: The Infrastructure for Alternative Metrics](https://reader034.vdocuments.site/reader034/viewer/2022052619/555a9055d8b42a991b8b48ab/html5/thumbnails/53.jpg)
Thank you! Todd Carpenter, Executive Director
[email protected] @TAC_NISO
!National Information Standards Organization (NISO)
3600 Clipper Mill Road, Suite 302 Baltimore, MD 21211 USA
+1 (301) 654-2512 www.niso.org