june 18, 2014 niso virtual conference: transforming assessment: alternative metrics and other trends
DESCRIPTION
June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends What Do Alternative Metrics and the 'Flight of the Bumblebee' Have in Common? - Martha Kyrillidou, PhD, Senior Director of Statistics and Service Quality Programs, Association of Research Libraries (ARL)TRANSCRIPT
![Page 1: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/1.jpg)
What Do Alternative Metrics and the 'Flight of the Bumblebee'
Have in Common?
Martha KyrillidouARL Statistics and Service Quality Programs
Association of Research Libraries (ARL)NISO June Virtual Conference:
Transforming Assessment: Alternative Metrics and Other TrendsJune 18, 2014
![Page 2: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/2.jpg)
e-Pubs downloads (May 16, 2014)
“I made the Libraries presentation to the Purdue Board of Trustees this morning, including a video of the downloads from Purdue e-Pubs, speeded up to show downloads in 2 hours in 30 seconds (nearly 3,000). The sound track was “Flight of the Bumblebee.” The greatest compliment I had from a Trustee after the presentation was when he said, his only complaint was “that the presentation didn’t go longer, he found it very interesting, totally unexpected!”
James L. Mullins, PhDDean of Libraries and
Esther Ellis Norton ProfessorPurdue University
![Page 3: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/3.jpg)
ENABLING WORLD-CHANGING RESEARCH Libraries’ Purdue e-Pubs
As of May 8, 2014
37,580 documents
6,908,130 total downloads
2,596,249 past year
![Page 4: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/4.jpg)
Transformation I• "Flight of the Bumblebee" is an orchestral
interlude written by Nikolai Rimsky-Korsakov for his opera The Tale of Tsar Saltan, composed in 1899–1900. The piece closes Act III, Tableau 1, during which the magic Swan-Bird changes Prince Gvidon Saltanovich (the Tsar's son) into an insect so that he can fly away to visit his father (who does not know that he is alive).
![Page 5: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/5.jpg)
Association of Research Libraries
ARL Statistics and Assessment
…To describe and measure the
performance of research libraries and their contribution to teaching, research, scholarship and community service …
![Page 6: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/6.jpg)
Association of Research Libraries
ARL Statistics and Assessment
…To describe and measure the performance of research libraries and their
contribution to
teaching, research, scholarship and community service …
![Page 7: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/7.jpg)
The Printed Library Materials I Need for My Work
![Page 8: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/8.jpg)
Print and/or Electronic Journal Collections I Require for My Work
www.libqual.org
![Page 9: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/9.jpg)
A Library Website Enabling Me to Locate Information on My Own
![Page 10: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/10.jpg)
StatsQUAL®A gateway to library assessment tools that describe the role, character, and impact of physical and digital libraries.
LibQUAL+® is a rigorously tested Web-
based survey that libraries use to solicit, track, understand, and
act upon users‘ opinions of service quality.
LibQUAL+®
MINES for Libraries®
Measuring the Impact of Networked Electronic
Resources (MINES) is an online transaction- based
survey that collects data on the purpose of use of
electronic resources and the demographics of users.
ClimateQUAL®
ClimateQUAL® :
Organizational Climate and Diversity Assessment is
an online survey that measures staff perceptions
about: (a) the library's commitment to the
principles of diversity, (b) organizational policies and
procedures, and (c) staff attitudes.
ARL Statistics®
ARL Statistics™ is a series of annual publications that
describe the collections, expenditures, staffing, and
service activities for Association of Research
Libraries (ARL) member libraries.
LibValue
LibValue is a series of tested methods that capture Return on
Investment and Value of library services.
![Page 11: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/11.jpg)
The data and the models
• I hesitate to hang the future of libraries on a demonstrable effect size for the equation,
L = $$$
– Rachel Applegate , Indiana University SLIS
ARL Statistics®
ARL Statistics™ is a series of annual publications that
describe the collections, expenditures, staffing, and
service activities for Association of Research
Libraries (ARL) member libraries.
![Page 12: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/12.jpg)
Why study value
• Faced with difficult economic times and university budget cuts, the value of the library to the wider goals of the university is increasingly questioned.
– Introduction to the Lib-Value Project
» www.libvalue.org
Association of Research Libraries
![Page 13: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/13.jpg)
Association of Research Libraries
Use ……Value
![Page 14: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/14.jpg)
MINES for Libraries – Scholars Portal @ OCUL
9th Northumbria Conference, 23rd August 201114
In the Library
Off-campus
On-campus but not in the library
In the Library Off-campus On-campus but not in the library
![Page 15: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/15.jpg)
LibValue
Special Collections – Google Analytics application
• LibValue: Digitized Special Collections (video on YouTube)Thursday, August 15, 2013, 1:00–2:00 p.m. eastern
Presenters: Gayle Baker, Professor and Electronic Resources Coordinator, University of Tennessee Libraries Ken Wise, Associate Professor, University of Tennessee Libraries
www.arl.org
![Page 16: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/16.jpg)
A Values scorecard
• The value which stakeholders seek from libraries is transcendent, in the sense that the impact demonstrated must be beyond the library and immediate satisfaction or fulfillment. Library assessment effort to date has been (mainly) about quality rather than value.
• J. Stephen Town, Martha Kyrillidou, (2013) "Developing a values scorecard", Performance Measurement and Metrics, Vol. 14 Iss: 1, pp.7 - 16http://www.emeraldinsight.com/journals.htm?articleid=17086961
Association of Research Libraries
![Page 17: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/17.jpg)
A Value Scorecard
• Relationship & reputation capital• Organisational capital
– Tangible assets and resources– Intangible and meta-assets
• Library virtue– Impact and social capital
• Library momentum …. Leadership … Culture
![Page 18: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/18.jpg)
Article-level metrics – Relation/Reputation
• Usage• Captures• Mentions• Social Media• Citations
– Tananbaum, Greg. (2013, April 16). Article-level metrics: a SPARC primer. Scholarly Publishing and Academic Research Coalition. http://www.sparc.arl.org/sites/default/files/sparc-alm-primer.pdf
– Bollen, J. et al. (2008). Towards alternative metrics of journal impact: A comparison of download and citation data. http://arxiv.org/pdf/cs.DL/0503007.pdf
![Page 19: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/19.jpg)
![Page 20: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/20.jpg)
ARL Strategic Thinking and Design
• By 2033 the research library will have shifted from its role as a knowledge service provider within the university to become a collaborative partner within a rich and diverse learning ecosystem
![Page 21: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/21.jpg)
System of action components
– Coordinated management of collective collections– Scholarly publishing at scale– ARL Academy– Building a boundless symposium– A first suite of smart libraries– Innovation lab
![Page 22: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/22.jpg)
Transformation II
Questioning assumptions
Shaping the experience
![Page 23: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/23.jpg)
La danse (I) by Matisse, 1909, The Museum of Modern Art, New York City
![Page 24: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/24.jpg)
Henri Matisse The Dance, 1910, Hermitage Museum, St. Petersburg, Russia
![Page 25: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/25.jpg)
Dancing to the end of Love
Henri Matisse Music, 1910, Hermitage Museum, St. Petersburg, Russia
![Page 26: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/26.jpg)
Why I Love ….
• Stay connected• Get coffee• Take a break• Check out my class book• Reserve a conference room for a meeting• Use my own computer and plug in• Use the library’s computers• Study with a friend or on my own• Share laughs and hang with friends• Studying
![Page 27: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/27.jpg)
Final Blessings
• Indulge your curiosity• Influence your friends• Enable your organization.
– May your analytics always be good – and your judgment even better.
» Robert Morison, Analytics at Work
![Page 28: June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends](https://reader035.vdocuments.site/reader035/viewer/2022070313/554bc5aab4c9053a298b55f5/html5/thumbnails/28.jpg)