Download - Discovery impact scelc colloquium 2014mar05
![Page 1: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/1.jpg)
Discovery or Displacement? A Major Longitudinal Study of the Effect of Web-‐Scale Discovery Services on
Online (Journal) Usage
SCELC Colloquium March 5, 2014
Michael Levine-‐Clark, University of Denver
John McDonald, University of Southern California Jason Price, SCELC ConsorNum
![Page 2: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/2.jpg)
“…a steep increase in full text downloads and link resolver click‐throughs suggests Summon had a dramatic impact on user behavior and the use of library collections during this time period.” The Impact of Web-scale Discovery on the Use of a Library Collection Doug Way (2010) http://scholarworks.gvsu.edu/library_sp/9/
![Page 3: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/3.jpg)
h"p://www.oclc.org/partnerships/econtent/solu4ons.en.html
Vendor marke5ng
![Page 4: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/4.jpg)
Does implementa4on of a discovery service impact usage of publisher-‐
hosted journal content?
![Page 5: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/5.jpg)
What did we measure?
• Whether there is an effect
• NOT why that effect exists (that’s a future study!)
![Page 6: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/6.jpg)
• “Society will need to shed some of its obsession for causality in exchange for simple correla5ons: not knowing why, but only what”
• Cukier & Mayer-‐Schonberger. 2013. Big data: A revolu4on that will transform how we live, work, and think.
![Page 7: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/7.jpg)
![Page 8: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/8.jpg)
![Page 9: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/9.jpg)
Data collec5on • List of libraries with discovery services
> Searched on lib-‐web-‐cats • Surveyed Libraries
> Discovery service Implemented > ImplementaNon Date (month/year) > Search box locaNon > MarkeNng effort
• 149 Libraries Gave Approval > 33 libraries selected for this phase > 6 for each of the 4 major discovery services and a group of 9 libraries with no service
![Page 10: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/10.jpg)
Dataset • 33 Libraries – 28 US, 2 CA, 1 each from UK, AUS, NZ
– WorldCat book holdings > Average: 1,114,193 ; Range: ~300k to ~2.6mil
• ImplementaNon dates (Discovery Libraries): > 2010 (3), 2011 (19), 2012 (2)
• 6 Publishers • 9,206 Journals • 163,545 Usable ObservaNons
![Page 11: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/11.jpg)
Methodology Compared COUNTER JR1 total full text arNcle views for the 12 months before vs 12 months aeer implementaNon date
June
2010
Start
Implem
entaNo
n May 2011
May 2012
End
Year 1 Year 2
Included implementaNon month in Year 1 to ensure that both periods included an enNre academic year
![Page 12: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/12.jpg)
![Page 13: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/13.jpg)
Examine Data for Outliers
![Page 14: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/14.jpg)
Observa5ons by Library & Service
![Page 15: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/15.jpg)
Observa5ons by Publisher
![Page 16: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/16.jpg)
![Page 17: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/17.jpg)
Average Usage Change By Discovery & Publisher
![Page 18: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/18.jpg)
Analyzing Usage Change: % vs Total
Use 12 months before
Use 12 months aRer
% Change Total Change
Journal A 500 600 20% 100 Journal B 5 15 200% 10
Which is the beier measure?
Is it the same for publisher-‐ & journal-‐level data?
![Page 19: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/19.jpg)
Reducing varia5on due to ins5tu5on size Currently converNng to change per FTE Values are shown as x 1,000 to bring the change metric back per journal-‐library combinaNon to a minimum of 0.1 2013 JISC Discovery study took a similar approach
![Page 20: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/20.jpg)
Average Usage Change By Discovery & Publisher
Per Journal & Per 10,000 FTE
![Page 21: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/21.jpg)
Full Model
Including Discovery Service, Publisher, and Library
Including Discovery Service, Publisher, and Library
![Page 22: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/22.jpg)
Does the effect of discovery service differ across libraries?
Library 10-‐15 Library 16-‐21 Library 22-‐27 Library 28-‐33 Library 1-‐9
![Page 23: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/23.jpg)
Nested ANOVA Model
[all three factors – preliminary results]
![Page 24: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/24.jpg)
Does usage change vary across libraries?
Institution (sorted by Mean Change)
![Page 25: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/25.jpg)
Does usage change vary across publishers?
Publisher (sorted by Mean Change)
![Page 26: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/26.jpg)
Does usage change vary across discovery services?
![Page 27: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/27.jpg)
Publisher
Does the effect of discovery service differ across publishers?
![Page 28: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/28.jpg)
Results Can we detect differences between Discovery Services, Publishers, and/or Libraries and/or their interac4ons? • Library – Yes • Publisher – No • Discovery Service – Yes
• DifferenNal discovery service effect by publisher – Yes
![Page 29: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/29.jpg)
Next Steps • Design & test for effects of:
– Aggregator full text availability – Publisher Size – Journal Subject – Overall usage trends (Requires Disc Srvc ‘control’) – ConfiguraNon opNons in Discovery services
• Expand pool of libraries • Perhaps explore WHY
![Page 30: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/30.jpg)
Sharing Data • With par5cipa5ng libraries
– Customized reports for each library
• With par5cipa5ng publishers – Customized reports for each publisher – PresentaNons as requested
• With discovery vendors – PresentaNons as requested
• In publica5ons and presenta5ons – Maintaining anonymity of data
![Page 31: Discovery impact scelc colloquium 2014mar05](https://reader033.vdocuments.site/reader033/viewer/2022051816/54463459afaf9f65178b45a7/html5/thumbnails/31.jpg)
Doing “Resarch”, SCELC Style!
• Why SCELC? • SCELC Funding
– staNsNcs consultant – research & wriNng retreats – See hip://bit.ly/1dNMDL3 for more detail
• SCELC libraries encouraged to parNcipate in next round – Survey: hip://bit.ly/DSparNcipaNon